Timeout Pattern

Overview

The Timeout Pattern is a resilience pattern that prevents operations from hanging indefinitely by enforcing a maximum execution time. Without timeouts, a single slow downstream service can hold up threads, connections, and user requests indefinitely, causing cascading failures across the system.

It sounds trivially simple — “stop waiting after N seconds” — but getting it right involves three questions most teams skip: how long is long enough, what happens to the work you abandoned, and who downstream still thinks the request is alive.

When to Use

Use the Timeout Pattern when:

• You call external services, databases, or APIs that may become unresponsive
• You need to guarantee maximum response times to users or upstream callers
• Hanging operations could exhaust thread pools, connection pools, or memory
• You want to fail fast rather than wait indefinitely for a response

When not to use it: legitimate long-running operations don’t belong under a short timeout. Batch exports, report generation, video processing, and large file uploads need a different shape — an async job with a queue and a progress endpoint, not a request that dies at 30 seconds. Streaming responses also need care: a blanket timeout on the whole stream will kill slow consumers, so timeout the individual reads, not the response.

Solution

Python

import functools
from concurrent.futures import ThreadPoolExecutor, TimeoutError as FutureTimeout

def with_timeout(seconds: float):
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            with ThreadPoolExecutor(max_workers=1) as executor:
                future = executor.submit(func, *args, **kwargs)
                try:
                    return future.result(timeout=seconds)
                except FutureTimeout:
                    raise TimeoutError(f"Operation timed out after {seconds}s")
        return wrapper
    return decorator

@with_timeout(seconds=2.0)
def fetch_slow_data():
    import time
    time.sleep(5)
    return "data"

# Usage
try:
    result = fetch_slow_data()
    print(result)
except TimeoutError as e:
    print(f"Failed: {e}")

JavaScript

function withTimeout(fn, timeoutMs) {
  return function(...args) {
    return new Promise((resolve, reject) => {
      const timer = setTimeout(() => {
        reject(new Error(`Operation timed out after ${timeoutMs}ms`));
      }, timeoutMs);

      Promise.resolve(fn(...args))
        .then(resolve)
        .catch(reject)
        .finally(() => clearTimeout(timer));
    });
  };
}

async function fetchSlowData() {
  await new Promise(r => setTimeout(r, 5000));
  return "data";
}

const timedFetch = withTimeout(fetchSlowData, 2000);

// Usage
timedFetch()
  .then(console.log)
  .catch(e => console.log("Failed:", e.message));

Java

import java.util.concurrent.*;

public class Timeout {
    public static <T> T execute(Callable<T> task, long timeoutMs) throws Exception {
        ExecutorService executor = Executors.newSingleThreadExecutor();
        try {
            Future<T> future = executor.submit(task);
            return future.get(timeoutMs, TimeUnit.MILLISECONDS);
        } catch (TimeoutException e) {
            throw new RuntimeException("Operation timed out after " + timeoutMs + "ms");
        } finally {
            executor.shutdownNow();
        }
    }

    public static void main(String[] args) {
        try {
            String result = execute(() -> {
                Thread.sleep(5000);
                return "data";
            }, 2000);
            System.out.println(result);
        } catch (Exception e) {
            System.out.println("Failed: " + e.getMessage());
        }
    }
}

Explanation

The Timeout Pattern enforces a hard deadline on operations, built on three moving parts:

• Deadline: the maximum time an operation is allowed to run.
• Abandonment: when the deadline expires, the caller stops waiting. Whether the underlying work actually stops is a separate question — see Cancellation below.
• Propagation: timeouts should shrink as they travel down the call chain. If an API call has a 5s budget and it calls a database that takes 4s, the database call needs a shorter timeout — say 2s — so the database isn’t still burning time after the API gave up.

This prevents thread pool exhaustion, connection leaks, and the particular misery of a user-facing request that hangs for ninety seconds because a dependency is slow.

Each layer’s timeout leaves headroom for the layer above to decide what to do with the failure — retry, fall back, or return an error. That’s why the outer budget is always bigger than the sum of inner timeouts: the difference is the caller’s time to react.

Timeout vs cancellation

A subtlety that bites in production: timing out the wait isn’t the same as stopping the work. In the Python and Java examples, future.result(timeout) and future.get(timeout) raise an exception for the caller, but the worker thread keeps running until it finishes. If your timeout fires a thousand times a minute on a sick database, you’re still holding a thousand connections.

Real cancellation depends on the platform:

• JavaScript: AbortController actually signals the underlying fetch/socket to stop.
• Java: future.cancel(true) sends an interrupt — it works only if the code inside checks for interruption (blocking I/O on sockets usually ignores it).
• Python: threads can’t be killed safely; the standard answer is setting timeouts on the I/O itself (requests.get(url, timeout=3), psycopg2 statement_timeout) so the abandoned thread exits on its own.
• gRPC / HTTP/2: deadlines travel in metadata, and the server can hear the client cancel — true end-to-end cancellation.

The safest combination is a timeout at every level: the call-level wrapper above plus socket-level timeouts in the HTTP client and driver, so the abandoned work dies quickly instead of leaking connections.

External calls with AbortController

// Timeout pattern with AbortController (Node.js 18+)
async function fetchWithTimeout(url: string, timeoutMs: number): Promise<Response> {
  const controller = new AbortController();
  const timeout = setTimeout(() => controller.abort(), timeoutMs);
  try {
    const response = await fetch(url, { signal: controller.signal });
    return response;
  } catch (err) {
    if (err instanceof DOMException && err.name === "AbortError") {
      throw new Error(`Request timeout after ${timeoutMs}ms`);
    }
    throw err;
  } finally {
    clearTimeout(timeout);
  }
}

// Usage: call external API with a 5s timeout
try {
  const res = await fetchWithTimeout("https://api.stripe.com/v1/charges", 5000);
  const data = await res.json();
} catch (err) {
  if (err.message.includes("timeout")) {
    // Handle timeout: retry, fallback, or error to user
    console.error("Stripe API timeout, using fallback");
  }
}

Unlike Promise.race-style wrappers, controller.abort() tears down the underlying socket — the request doesn’t keep occupying the connection after the caller has moved on.

Choosing timeout values

The usual advice — “set it to 2-3x p99” — is a starting point, not the answer. A better mental model:

Service	Timeout	Retries	Backoff
Stripe API	5s	3	Exponential
DB query	10s	0	N/A
Redis	1s	2	Fixed 500ms
S3 upload	30s	2	Exponential
Internal API	3s	2	Exponential
Email service	10s	3	Exponential

A few rules of thumb behind the numbers:

• Fast, cheap dependencies get short timeouts. Redis at p99 = 50ms doesn’t deserve 5s — a 1s ceiling surfaces a dead node quickly.
• Retries only make sense inside the budget. Three retries at 5s each is a 15s worst case; make sure the layer above can afford it.
• Non-idempotent calls get zero retries by default. A timed-out POST may have succeeded server-side; the retry decision belongs to the caller, not the generic wrapper.
• Re-measure quarterly. Latency drifts; a timeout set for last year’s p99 will be wrong now.

Variants

Variant	Description	Use Case
Fixed Timeout	Same timeout for all calls	Simple, predictable behavior
Adaptive Timeout	Timeout based on historical latencies (P99)	Dynamic response to service health
Deadline Propagation	Pass remaining time through the call chain	End-to-end request latency budgets
Partial Results	Return what was fetched before timeout	Streaming, search, aggregation

What Works

• Always set timeouts on external calls — network I/O, database queries, HTTP requests
• Propagate deadlines through your call chain (gRPC context, HTTP headers, explicit parameters)
• Set timeouts shorter at lower levels — leave headroom for retries and fallbacks
• Log timeout events with the target service name; a timeout rate per dependency is a leading indicator of trouble
• Combine with Circuit Breaker — if timeouts are frequent, stop calling the failing service — and with Retry for transient issues
• Prefer real cancellation over abandoned waits — AbortController, context cancellation, socket timeouts

Common Mistakes

• Not setting any timeout, allowing operations to hang forever
• Setting timeouts too long, defeating the purpose of failing fast — 60s defaults are a bug, not a safety margin
• Setting timeouts too short, causing unnecessary failures during normal latency spikes
• Not canceling the underlying operation when the timeout fires — the caller returns but the work, and its connections, live on
• Ignoring timeout propagation, causing cascading deadline misses
• Retrying non-idempotent operations after a timeout — you may execute the side effect twice

Troubleshooting

• Timeouts firing during normal load: your ceiling is inside the latency distribution. Measure p99 (not p50) and move the timeout above it, or investigate the latency regression first.
• Timeout but the work still runs: the wrapper abandoned the wait but never cancelled the operation. Add socket-level timeouts or use a primitive with real cancellation.
• Everything times out when one dependency is slow: headroom is inverted — an inner timeout that’s longer than the outer budget lets the downstream hold resources after the caller gave up.
• Retries amplify the outage: timeouts + retries with no jitter create a thundering herd against a recovering service. Add jitter, and cap retries inside the call budget.

See Also

• MDN: AbortController
• AWS Builders’ Library: Timeouts, retries, and backoff with jitter
• Retry Pattern
• Circuit Breaker Pattern
• Bulkhead Pattern
• Graceful Degradation Pattern