Rate Limiting

Overview

Rate limiting controls how many requests a client can make to your API in a given time window. It prevents abuse, ensures fair resource allocation, and protects downstream services from overload — and it’s one of the few defenses that works at every scale, from a single-process toy API to a globally distributed one.

This recipe implements the three algorithms you’ll actually meet: token bucket (Python, in-memory), fixed window (JavaScript + Redis, distributed), and sliding window (Java, in-memory). The comparison table below the code tells you which to pick, and the distributed section covers what changes when your API runs on more than one server.

When to Use

Use this recipe when:

• Protecting public APIs from abuse or DDoS
• Enforcing tiered usage limits (free vs paid plans)
• Preventing brute-force attacks on authentication endpoints — see JWT Authentication for the auth side
• Managing capacity for resource-intensive operations
• Implementing fair-use policies across users

Don’t rate limit by gut feel: pick the limit from real usage data (p95 of legitimate clients plus headroom), not a round number that “sounds safe”.

Solution

Python (Token Bucket)

Token bucket is the most common choice for APIs that tolerate bursts: tokens refill at a steady rate, and a client can spend them faster than they refill until the bucket empties.

import time
from threading import Lock

class TokenBucket:
    def __init__(self, capacity: int, refill_rate: float):
        self.capacity = capacity
        self.tokens = capacity
        self.refill_rate = refill_rate
        self.last_refill = time.time()
        self.lock = Lock()

    def allow(self) -> bool:
        with self.lock:
            now = time.time()
            elapsed = now - self.last_refill
            self.tokens = min(self.capacity, self.tokens + elapsed * self.refill_rate)
            self.last_refill = now
            if self.tokens >= 1:
                self.tokens -= 1
                return True
            return False

bucket = TokenBucket(capacity=10, refill_rate=1)
print(bucket.allow())  # True

JavaScript (Fixed Window with Redis)

Fixed window is the simplest algorithm that works across instances: a counter per key per window, incremented atomically in Redis. It’s the right starting point for most distributed APIs — see Rate Limiting with Redis for the production-hardened version.

const redis = require('redis');
const client = redis.createClient();

async function rateLimit(key, limit, windowSeconds) {
  const windowKey = `${key}:${Math.floor(Date.now() / 1000 / windowSeconds)}`;
  const current = await client.incr(windowKey);
  if (current === 1) {
    await client.expire(windowKey, windowSeconds);
  }
  return current <= limit;
}

// Express middleware — see the middleware recipe for the pattern
async function limiter(req, res, next) {
  const key = `ratelimit:${req.ip}`;
  const allowed = await rateLimit(key, 100, 60);
  if (!allowed) return res.status(429).json({ error: 'Too many requests' });
  next();
}

Java (Sliding Window)

Sliding window log keeps a timestamp per request and drops the boundary-burst problem of fixed windows, at the cost of storing every timestamp.

import java.util.concurrent.*;

public class SlidingWindow {
    private final int capacity;
    private final long windowMs;
    private final ConcurrentLinkedDeque<Long> timestamps = new ConcurrentLinkedDeque<>();

    public SlidingWindow(int capacity, long windowMs) {
        this.capacity = capacity;
        this.windowMs = windowMs;
    }

    public synchronized boolean allow() {
        long now = System.currentTimeMillis();
        while (!timestamps.isEmpty() && now - timestamps.peekFirst() > windowMs) {
            timestamps.pollFirst();
        }
        if (timestamps.size() < capacity) {
            timestamps.addLast(now);
            return true;
        }
        return false;
    }
}

Algorithm Comparison

Algorithm	Pros	Cons	Best For
Fixed Window	Simple, low memory	Burst at window boundary (up to 2× limit)	Basic protection, most APIs
Sliding Window	Smooth rate, no boundary bursts	Higher memory/compute	Precise rate control
Token Bucket	Allows bursts up to capacity	Harder to implement correctly	APIs with burst tolerance
Leaky Bucket	Strict constant output rate	Queues or drops excess requests	Protecting slow downstreams

Choosing an Algorithm

The table compresses into three questions. Do bursts matter? If a client legitimately needs 10 requests in a burst once a minute, token bucket absorbs it where fixed window rejects it. Is precision worth the memory? Sliding window log stores a timestamp per request — fine for hundreds of requests per window, wasteful for millions; sliding window counter (weighting the previous window) is the usual middle ground. Where does state live? In-memory is correct only when one process serves the traffic or limits are per-instance; the moment a second instance exists, the counter must move to Redis or the edge.

For most APIs the answer is: fixed window in Redis first, token bucket when burst complaints arrive, sliding window only when someone can name the boundary-burst incident they’re solving.

Distributed Rate Limiting

The JavaScript example above is the distributed case: Redis holds the counter, so every instance sees the same number. Three details decide whether it survives production:

• Fail open or fail closed? If Redis is down, do you allow everything (open — availability wins, abuse slips through during the outage) or reject everything (closed — protection wins, the API dies with Redis)? Most APIs fail open for read traffic and fail closed on login/payment endpoints. Decide explicitly; don’t let the default exception handler decide for you.
• Clock drift. Window math uses server time; NTP skew between app servers shifts window boundaries slightly. It’s harmless at minute windows and only matters at sub-second precision.
• Key design. ratelimit:{ip} punishes everyone behind a NAT or corporate proxy. Prefer API key or user ID, falling back to IP only for anonymous traffic — and give authenticated users a higher tier.

Per-Endpoint Limits

A single global limit treats a cheap GET /status the same as an expensive POST /reports/export. In practice you want weighted or per-endpoint budgets: cheap reads get a high ceiling, expensive writes get a low one, and authentication endpoints (/login, /password-reset) get the tightest limit of all — they’re the ones attackers actually hammer. Implement it as a map from route to limit, not as N separate limiters, so adding an endpoint is a config change rather than new middleware.

const LIMITS = {
  'POST:/login': { limit: 5, window: 60 },      // brute-force target
  'POST:/reports/export': { limit: 10, window: 3600 }, // expensive
  default: { limit: 100, window: 60 },
};

async function limiter(req, res, next) {
  const cfg = LIMITS[`${req.method}:${req.route?.path}`] ?? LIMITS.default;
  const key = `ratelimit:${req.user?.id ?? req.ip}:${req.method}:${req.route?.path}`;
  const allowed = await rateLimit(key, cfg.limit, cfg.window);
  if (!allowed) {
    res.set('Retry-After', String(cfg.window));
    return res.status(429).json({ error: 'Too many requests' });
  }
  next();
}

Testing Your Limiter

Rate limiters fail quietly — they either let too much through or block real users, and neither throws an exception. Three tests cover the failure modes: burst to the limit and confirm the 429 arrives with Retry-After; wait out the window and confirm the counter resets (in tests, shrink the window to 1-2 seconds rather than sleeping a minute); and kill Redis mid-test to confirm your fail-open/fail-closed decision is the one you actually made. A load test that ramps past the limit also verifies the limiter itself isn’t the bottleneck — the check should add single-digit milliseconds, not a full extra round trip per request.

Returning 429 Correctly

A rate limit rejection is still an API response and deserves the same care as a 200. Return 429 Too Many Requests with a Retry-After header (seconds until retry, per RFC 6585), and ideally the IETF RateLimit header fields — RateLimit-Limit, RateLimit-Remaining, RateLimit-Reset — so well-behaved clients can self-throttle instead of discovering the limit by tripping over it. Include the limit in your API docs; a limit nobody knows about reads as flakiness, not policy.

What Works

• Return 429 with Retry-After — clients that know when to retry stop hammering; clients that don’t get blocked again anyway
• Use Redis for distributed limiting — in-memory counters silently double the limit per instance you add
• Differentiate by client — API key or user ID, not just IP; shared NATs punish innocent users
• Tier by trust — authenticated users get more than anonymous; paid plans get more than free
• Log limit events — a spike in 429s is either an attack or a limit set too tight; you want to know which
• Tell clients the limit — RateLimit-* headers turn reactive backoff into proactive pacing

Common Mistakes

• Rate limiting by IP only — corporate NATs and mobile carrier gateways concentrate thousands of users behind one address
• In-memory counters behind a load balancer — N instances means N times the intended limit
• No Redis failure plan — an unhandled exception on Redis-down either kills the API (fail closed by accident) or disables the limiter (fail open by accident)
• Limits chosen by vibes — 100 req/min blocks a legitimate dashboard page that fans out 40 calls; measure real client usage first
• 429 without Retry-After — clients retry immediately, making the problem you’re defending against worse

Troubleshooting

• Legitimate users hitting limits: check whether you’re keying on IP behind a NAT/proxy — switch to API key or user ID, and raise the anonymous tier only after measuring.
• 429s spike at the top of the minute: clients synchronized their retries to the window boundary — add Retry-After with jitter, or move to sliding window so the boundary disappears.
• Limit doubled after scaling: in-memory counter + second instance — move the counter to Redis; each process was enforcing its own copy.
• Redis down and API rejecting everything: the limiter failed closed unintentionally — wrap the Redis call in a timeout and decide fail-open vs fail-closed per endpoint class.
• Limiter adds visible latency: the Redis round-trip is on the hot path — use a Lua script to make check+increment one round trip, or allow a small local burst and reconcile asynchronously.

Further Reading

• RFC 6585 — Additional HTTP Status Codes — defines 429 Too Many Requests and Retry-After semantics
• IETF RateLimit header fields — the RateLimit-Limit/Remaining/Reset standard
• Redis INCR documentation — the primitive behind fixed-window counters
• Related on this site: Rate Limiting with Redis for production Redis patterns, token bucket with Redis for the Lua-scripted variant, and the API gateway recipe for edge-level enforcement.