Retry Logic with Exponential Backoff

Overview

Transient failures — network timeouts, rate limits, temporary service outages — are inevitable in distributed systems. Naively retrying immediately can overload struggling services. Exponential backoff spaces out retries exponentially (1s, 2s, 4s, 8s…) while jitter randomizes wait times to prevent thundering herds. This approach handles building a reliable retry decorator with configurable backoff strategies, circuit breaker integration, and idempotency awareness in Python, JavaScript, and Java.

When to Use

Use this resource when:

• Calling external APIs that may rate-limit or experience temporary outages. See Call REST API for API client patterns.
• Publishing messages to queues or event buses that may be temporarily unavailable. See RabbitMQ Task Queue for message broker resilience.
• Performing database operations that might hit deadlock or connection timeouts. See Connection Pooling for managing DB connections.
• Building microservices that need resilience against downstream service degradation. See Event-Driven Microservices for resilient architecture.

Solution

Python

import time
import random
from functools import wraps
from typing import Callable, TypeVar, Tuple

T = TypeVar("T")

def retry(
    max_retries: int = 3,
    base_delay: float = 1.0,
    max_delay: float = 60.0,
    exceptions: Tuple[type, ...] = (Exception,),
    jitter: bool = True,
    on_retry: Callable[[Exception, int, float], None] = None
):
    def decorator(fn: Callable[..., T]) -> Callable[..., T]:
        @wraps(fn)
        def wrapper(*args, **kwargs) -> T:
            for attempt in range(max_retries + 1):
                try:
                    return fn(*args, **kwargs)
                except exceptions as e:
                    if attempt == max_retries:
                        raise
                    delay = min(base_delay * (2 ** attempt), max_delay)
                    if jitter:
                        delay = random.uniform(0, delay)
                    if on_retry:
                        on_retry(e, attempt + 1, delay)
                    time.sleep(delay)
        return wrapper
    return decorator

# Usage
@retry(max_retries=3, base_delay=1.0, exceptions=(ConnectionError, TimeoutError))
def fetch_data(url: str) -> dict:
    import requests
    response = requests.get(url, timeout=5)
    response.raise_for_status()
    return response.json()

# With idempotency key
import uuid

def call_api_with_retry():
    idempotency_key = str(uuid.uuid4())
    headers = {"Idempotency-Key": idempotency_key}
    return fetch_data("https://api.example.com/data")

JavaScript

async function retry(fn, options = {}) {
  const {
    maxRetries = 3,
    baseDelay = 1000,
    maxDelay = 60000,
    jitter = true,
    shouldRetry = () => true,
    onRetry = () => {},
  } = options;

  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      return await fn();
    } catch (error) {
      if (attempt === maxRetries || !shouldRetry(error)) {
        throw error;
      }
      let delay = Math.min(baseDelay * (2 ** attempt), maxDelay);
      if (jitter) {
        delay = Math.random() * delay;
      }
      onRetry(error, attempt + 1, delay);
      await new Promise((resolve) => setTimeout(resolve, delay));
    }
  }
}

// Usage
async function fetchWithRetry(url) {
  return retry(
    () => fetch(url).then((r) => r.json()),
    {
      maxRetries: 3,
      baseDelay: 1000,
      shouldRetry: (err) => err.status >= 500 || err.code === "ETIMEDOUT",
      onRetry: (err, attempt, delay) => {
        console.warn(`Retry ${attempt} after ${delay}ms: ${err.message}`);
      },
    }
  );
}

// Decorator-style with idempotency
function withRetry(fn, options) {
  return (...args) => retry(() => fn(...args), options);
}

Java

import java.time.Duration;
import java.util.Random;
import java.util.concurrent.Callable;
import java.util.function.BiConsumer;
import java.util.function.Predicate;

public class RetryExecutor {
  private final int maxRetries;
  private final Duration baseDelay;
  private final Duration maxDelay;
  private final boolean jitter;
  private final Predicate<Throwable> shouldRetry;
  private final BiConsumer<Throwable, Integer> onRetry;
  private final Random random = new Random();

  public RetryExecutor(int maxRetries, Duration baseDelay, Duration maxDelay,
                       boolean jitter, Predicate<Throwable> shouldRetry,
                       BiConsumer<Throwable, Integer> onRetry) {
    this.maxRetries = maxRetries;
    this.baseDelay = baseDelay;
    this.maxDelay = maxDelay;
    this.jitter = jitter;
    this.shouldRetry = shouldRetry;
    this.onRetry = onRetry;
  }

  public <T> T execute(Callable<T> action) throws Exception {
    for (int attempt = 0; attempt <= maxRetries; attempt++) {
      try {
        return action.call();
      } catch (Exception e) {
        Throwable cause = e instanceof java.util.concurrent.ExecutionException
          ? e.getCause() : e;
        if (attempt == maxRetries || !shouldRetry.test(cause)) {
          throw e;
        }
        long delayMs = Math.min(
          baseDelay.toMillis() * (1L << attempt),
          maxDelay.toMillis()
        );
        if (jitter) {
          delayMs = (long) (random.nextDouble() * delayMs);
        }
        onRetry.accept(cause, attempt + 1);
        Thread.sleep(delayMs);
      }
    }
    throw new IllegalStateException("Unreachable");
  }

  // Usage
  public static void main(String[] args) throws Exception {
    RetryExecutor retry = new RetryExecutor(
      3, Duration.ofSeconds(1), Duration.ofSeconds(60),
      true,
      t -> t instanceof java.net.ConnectException,
      (err, attempt) -> System.out.println("Retry " + attempt + ": " + err.getMessage())
    );

    String result = retry.execute(() -> {
      // Simulated flaky call
      if (Math.random() < 0.7) throw new java.net.ConnectException("timeout");
      return "success";
    });
  }
}

Explanation

• Exponential backoff doubles the wait time after each failure: 1s → 2s → 4s → 8s. This gives recovering services breathing room while still retrying promptly.
• Jitter adds randomness (random * delay) to prevent synchronized retries from many clients simultaneously hitting a recovering server (thundering herd problem).
• Max delay cap prevents unbounded waits — essential for user-facing operations where latency matters.
• Exception filtering ensures you only retry transient errors (timeouts, 5xx, connection refused), not permanent ones (4xx client errors, validation failures).
• Idempotency keys (UUID sent as header) guarantee that retried requests don’t create duplicate side effects. The server uses the way to deduplicate.

Variants

Strategy	Backoff Formula	Best For
Fixed	delay constant	Predictable retry intervals, debugging
Linear	delay * attempt	Moderate load on recovering services
Exponential	delay * 2^attempt	Most common, good balance of persistence vs load
Exponential + Jitter	random * exponential	Production APIs, prevents thundering herd
Circuit Breaker	Fail-fast after N errors	Cascading failure protection

What Works

1. Always add jitter in production — without it, coordinated retries from thousands of clients can DDoS a recovering service.
2. Only retry idempotent operations — POST without idempotency key or non-transactional writes can create duplicates on retry.
3. Set a max delay and total timeout — a user waiting 60+ seconds for a retry cascade is worse than failing fast with a clear error.
4. Log every retry — include attempt number, delay, and exception type for debugging intermittent issues.
5. Consider circuit breakers for downstream dependencies — if a service fails consistently, stop retrying for a cooldown period instead of hammering it.

Common Mistakes

1. Retrying immediately (0 delay), amplifying load on an already struggling service.
2. Retrying non-idempotent operations like payments or inventory decrements without deduplication keys.
3. Not capping max delay, causing requests to hang for minutes before the user sees an error.
4. Retrying all exceptions indiscriminately, including 4xx client errors that will never succeed.
5. Omitting jitter, leading to thundering herd problems during service recovery.

Performance Tips

1. Use async retries for I/O-bound operations. Don’t block a thread while waiting:

import asyncio

async def async_retry(fn, max_retries=3, base_delay=1.0):
    for attempt in range(max_retries + 1):
        try:
            return await fn()
        except Exception:
            if attempt == max_retries:
                raise
            await asyncio.sleep(base_delay * (2 ** attempt))

2. Cache successful results. If a retried call succeeds, cache the result to avoid future retries:

from functools import lru_cache

@lru_cache(maxsize=128)
@retry(max_retries=3)
def fetch_config(key: str) -> str:
    return requests.get(f"https://config.example.com/{key}").text

3. Use connection pooling with retries. Reusing connections avoids the TCP handshake overhead on each retry:

import requests
from requests.adapters import HTTPAdapter

session = requests.Session()
session.mount("https://", HTTPAdapter(pool_connections=10, pool_maxsize=100))