Retry Logic with Exponential Backoff

Overview

Transient failures — network timeouts, rate limits, temporary outages — are part of every distributed system. Retrying right away makes things worse: every client keeps hammering a service that’s already struggling to recover. Exponential backoff spreads retries out (1s, 2s, 4s, 8s…) and jitter randomizes each wait so thousands of clients don’t retry in lockstep — the thundering herd problem. This recipe builds a configurable retry decorator in Python, JavaScript, and Java, then wires it to a circuit breaker and keeps retried writes safe with idempotency keys.

When to Use

Reach for this recipe when:

• Calling external APIs that may rate-limit or experience temporary outages. See Call REST API for API client patterns.
• Publishing messages to queues or event buses that may be temporarily unavailable. See RabbitMQ Task Queue for message broker resilience.
• Performing database operations that might hit deadlock or connection timeouts. See Connection Pooling for managing DB connections.
• Building microservices that need resilience against downstream service degradation. See Event-Driven Microservices for resilient architecture.
• Don’t reach for retries on non-idempotent writes that lack deduplication — retrying a payment POST or a stock decrement blindly creates duplicates. Make the operation idempotent first.

Solution

Python

import time
import random
from functools import wraps
from typing import Callable, TypeVar, Tuple

T = TypeVar("T")

def retry(
    max_retries: int = 3,
    base_delay: float = 1.0,
    max_delay: float = 60.0,
    exceptions: Tuple[type, ...] = (Exception,),
    jitter: bool = True,
    on_retry: Callable[[Exception, int, float], None] = None
):
    def decorator(fn: Callable[..., T]) -> Callable[..., T]:
        @wraps(fn)
        def wrapper(*args, **kwargs) -> T:
            for attempt in range(max_retries + 1):
                try:
                    return fn(*args, **kwargs)
                except exceptions as e:
                    if attempt == max_retries:
                        raise
                    delay = min(base_delay * (2 ** attempt), max_delay)
                    if jitter:
                        delay = random.uniform(0, delay)
                    if on_retry:
                        on_retry(e, attempt + 1, delay)
                    time.sleep(delay)
        return wrapper
    return decorator

# Usage
@retry(max_retries=3, base_delay=1.0, exceptions=(ConnectionError, TimeoutError))
def fetch_data(url: str) -> dict:
    import requests
    response = requests.get(url, timeout=5)
    response.raise_for_status()
    return response.json()

# With idempotency key — generate it once, send it on every attempt
import uuid

idempotency_key = str(uuid.uuid4())

@retry(max_retries=3, exceptions=(ConnectionError, TimeoutError))
def create_order(payload: dict) -> dict:
    import requests
    response = requests.post(
        "https://api.example.com/orders",
        json=payload,
        headers={"Idempotency-Key": idempotency_key},
        timeout=5,
    )
    response.raise_for_status()
    return response.json()

JavaScript

async function retry(fn, options = {}) {
  const {
    maxRetries = 3,
    baseDelay = 1000,
    maxDelay = 60000,
    jitter = true,
    shouldRetry = () => true,
    onRetry = () => {},
  } = options;

  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      return await fn();
    } catch (error) {
      if (attempt === maxRetries || !shouldRetry(error)) {
        throw error;
      }
      let delay = Math.min(baseDelay * (2 ** attempt), maxDelay);
      if (jitter) {
        delay = Math.random() * delay;
      }
      onRetry(error, attempt + 1, delay);
      await new Promise((resolve) => setTimeout(resolve, delay));
    }
  }
}

// Usage
async function fetchWithRetry(url) {
  return retry(
    () => fetch(url).then((r) => r.json()),
    {
      maxRetries: 3,
      baseDelay: 1000,
      shouldRetry: (err) => err.status >= 500 || err.code === "ETIMEDOUT",
      onRetry: (err, attempt, delay) => {
        console.warn(`Retry ${attempt} after ${delay}ms: ${err.message}`);
      },
    }
  );
}

// Decorator-style with idempotency
function withRetry(fn, options) {
  return (...args) => retry(() => fn(...args), options);
}

Java

import java.time.Duration;
import java.util.Random;
import java.util.concurrent.Callable;
import java.util.function.BiConsumer;
import java.util.function.Predicate;

public class RetryExecutor {
  private final int maxRetries;
  private final Duration baseDelay;
  private final Duration maxDelay;
  private final boolean jitter;
  private final Predicate<Throwable> shouldRetry;
  private final BiConsumer<Throwable, Integer> onRetry;
  private final Random random = new Random();

  public RetryExecutor(int maxRetries, Duration baseDelay, Duration maxDelay,
                       boolean jitter, Predicate<Throwable> shouldRetry,
                       BiConsumer<Throwable, Integer> onRetry) {
    this.maxRetries = maxRetries;
    this.baseDelay = baseDelay;
    this.maxDelay = maxDelay;
    this.jitter = jitter;
    this.shouldRetry = shouldRetry;
    this.onRetry = onRetry;
  }

  public <T> T execute(Callable<T> action) throws Exception {
    for (int attempt = 0; attempt <= maxRetries; attempt++) {
      try {
        return action.call();
      } catch (Exception e) {
        Throwable cause = e instanceof java.util.concurrent.ExecutionException
          ? e.getCause() : e;
        if (attempt == maxRetries || !shouldRetry.test(cause)) {
          throw e;
        }
        long delayMs = Math.min(
          baseDelay.toMillis() * (1L << attempt),
          maxDelay.toMillis()
        );
        if (jitter) {
          delayMs = (long) (random.nextDouble() * delayMs);
        }
        onRetry.accept(cause, attempt + 1);
        Thread.sleep(delayMs);
      }
    }
    throw new IllegalStateException("Unreachable");
  }

  // Usage
  public static void main(String[] args) throws Exception {
    RetryExecutor retry = new RetryExecutor(
      3, Duration.ofSeconds(1), Duration.ofSeconds(60),
      true,
      t -> t instanceof java.net.ConnectException,
      (err, attempt) -> System.out.println("Retry " + attempt + ": " + err.getMessage())
    );

    String result = retry.execute(() -> {
      // Simulated flaky call
      if (Math.random() < 0.7) throw new java.net.ConnectException("timeout");
      return "success";
    });
  }
}

Explanation

• Exponential backoff doubles the wait after each failure: 1s → 2s → 4s → 8s. The service gets time to recover instead of a fixed drumbeat of retries.
• Jitter randomizes each wait (random * delay) so clients recovering at the same moment don’t fire retries in sync. “Full jitter” picks a random point between 0 and the exponential cap and spreads load the most; “decorrelated jitter” picks a random point between the base delay and 3× the previous sleep, which converges faster on busy services.
• Max delay and total deadline are two different caps. Max delay bounds a single wait; the deadline bounds the whole operation. Without a deadline, a caller can burn minutes in retries while the user has already left.
• Exception filtering limits retries to transient errors (timeouts, 5xx, connection refused). A 4xx fails the same way every time — retrying it just burns latency.
• Retry-After beats any formula. When a 429 or 503 carries that header, wait exactly what it says — the server is telling you its recovery time.
• Idempotency keys (a UUID sent as a header) let the server deduplicate retried requests, so a retried POST doesn’t create the record twice.

Variants

Strategy	Backoff Formula	Best For
Fixed	delay constant	Predictable retry intervals, debugging
Linear	delay * attempt	Moderate load on recovering services
Exponential	delay * 2^attempt	Most common, good balance of persistence vs load
Exponential + Jitter	random * exponential	Production APIs, prevents thundering herd
Decorrelated jitter	min(cap, rand(base, prev * 3))	High-contention APIs, AWS-style clients
Circuit Breaker	Fail-fast after N errors	Cascading failure protection

What Works

1. Always add jitter in production. Coordinated retries from thousands of clients can DDoS a service that’s just coming back.
2. Retry only idempotent operations. A POST without an idempotency key — or a non-transactional write — can create duplicates on retry. If you can’t make the operation idempotent, put it behind a queue or deduplicate on the server.
3. Cap the single delay and the total budget separately. Waiting 60s on one retry is fine for a batch job and fatal for a checkout request — decide the budget per call site.
4. Log every retry with the attempt number, computed delay, and exception type. Without that, intermittent failures stay invisible until they become outages.
5. Pair retries with a circuit breaker. When a dependency fails consistently, open the circuit for a cooldown and fail fast instead of paying the retry cost on every request. See the Circuit Breaker Pattern.

Common Mistakes

1. Retrying instantly with zero delay and piling even more load on a struggling service.
2. Retrying non-idempotent operations like payments or inventory decrements without deduplication keys.
3. Not capping max delay, causing requests to hang for minutes before the user sees an error.
4. Retrying all exceptions indiscriminately, including 4xx client errors that will never succeed.
5. Omitting jitter, leading to thundering herd problems during service recovery.

Additional Implementations

Circuit Breaker Integration

import time
from enum import Enum
from functools import wraps

class CircuitState(Enum):
    CLOSED = "closed"
    OPEN = "open"
    HALF_OPEN = "half_open"

class CircuitBreaker:
    def __init__(self, failure_threshold=5, recovery_timeout=30):
        self.failure_threshold = failure_threshold
        self.recovery_timeout = recovery_timeout
        self.failure_count = 0
        self.state = CircuitState.CLOSED
        self.last_failure_time = None

    def __call__(self, fn):
        @wraps(fn)
        def wrapper(*args, **kwargs):
            if self.state == CircuitState.OPEN:
                if time.time() - self.last_failure_time > self.recovery_timeout:
                    self.state = CircuitState.HALF_OPEN
                else:
                    raise Exception("Circuit breaker is OPEN")

            try:
                result = fn(*args, **kwargs)
                if self.state == CircuitState.HALF_OPEN:
                    self.state = CircuitState.CLOSED
                    self.failure_count = 0
                return result
            except Exception as e:
                self.failure_count += 1
                self.last_failure_time = time.time()
                if self.failure_count >= self.failure_threshold:
                    self.state = CircuitState.OPEN
                raise
        return wrapper

# Usage: retry + circuit breaker
cb = CircuitBreaker(failure_threshold=5, recovery_timeout=30)

@retry(max_retries=3, base_delay=1.0)
@cb
def call_external_api():
    import requests
    return requests.get("https://api.example.com/data").json()

Go with Standard Library

package main

import (
    "context"
    "fmt"
    "math"
    "math/rand"
    "time"
)

func withRetry(ctx context.Context, fn func() error, maxRetries int, baseDelay time.Duration) error {
    for attempt := 0; attempt <= maxRetries; attempt++ {
        err := fn()
        if err == nil {
            return nil
        }
        if attempt == maxRetries {
            return fmt.Errorf("after %d retries: %w", maxRetries, err)
        }

        delay := float64(baseDelay) * math.Pow(2, float64(attempt))
        jitter := rand.Float64() * delay * 0.5
        select {
        case <-time.After(time.Duration(delay + jitter)):
        case <-ctx.Done():
            return ctx.Err()
        }
    }
    return nil
}

// Usage
func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()

    err := withRetry(ctx, func() error {
        // simulated flaky call
        return nil
    }, 5, time.Second)
    fmt.Println("Result:", err)
}

Python with Tenacity Library

from tenacity import (
    retry, stop_after_attempt, wait_exponential_jitter,
    retry_if_exception_type, before_sleep_log
)
import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

@retry(
    stop=stop_after_attempt(5),
    wait=wait_exponential_jitter(initial=1, max=60),
    retry=retry_if_exception_type((ConnectionError, TimeoutError)),
    before_sleep=before_sleep_log(logger, logging.WARNING),
    reraise=True,
)
def fetch_with_tenacity(url: str) -> dict:
    import requests
    response = requests.get(url, timeout=5)
    response.raise_for_status()
    return response.json()

Performance Tips

1. Prefer async retries for I/O-bound work. A thread blocked on time.sleep holds its pool slot for the whole backoff:

import asyncio

async def async_retry(fn, max_retries=3, base_delay=1.0):
    for attempt in range(max_retries + 1):
        try:
            return await fn()
        except Exception:
            if attempt == max_retries:
                raise
            await asyncio.sleep(base_delay * (2 ** attempt))

2. Cache successful results. Once a retried call goes through, there’s no reason to pay the retry cost again on the next read:

from functools import lru_cache

@lru_cache(maxsize=128)
@retry(max_retries=3)
def fetch_config(key: str) -> str:
    return requests.get(f"https://config.example.com/{key}").text

3. Pool connections. Without pooling, every retry pays a fresh TCP and TLS handshake:

import requests
from requests.adapters import HTTPAdapter

session = requests.Session()
session.mount("https://", HTTPAdapter(pool_connections=10, pool_maxsize=100))

See Also

• AWS Builder’s Library — Timeouts, retries, and backoff with jitter — the canonical reference on full and decorrelated jitter.
• RFC 9110 § Retry-After — the header that should override any computed backoff.
• Google SRE Book — Handling Overload — how unbounded retries amplify outages.
• Retry Backoff — resilience4j, Polly, retry budgets, and deadlines at the architecture level.
• Companion repository — runnable retry helpers in Python, JavaScript, Java, and Go, with tests that verify the backoff sequence using an injected fake sleep.