API Rate Limiting

Overview

A scraper starts hitting your public API at 2,000 requests per second. Your database connection pool saturates, legitimate users start timing out, and the bill for the autoscaling group triples overnight. Rate limiting is the line between that incident and a shrug: a middleware that answers “how many?” before your handlers answer anything else.

Three algorithms cover most production needs. Token bucket refills a token pool at a fixed rate. Each request spends one token, so short bursts are allowed while the average rate stays bounded. Sliding window counts requests inside a rolling time window and rejects everything past the limit. Strict, with no boundary spikes. Leaky bucket drains queued requests at a constant rate, smoothing bursty input into predictable output.

This recipe gives you working implementations in Python, JavaScript, and Java, the trade-offs between them, and the production details that decide whether your limiter protects you or becomes the outage: Redis failover policy, per-tenant limits, correct 429 responses, and what to monitor. For a deeper design walkthrough, see the API Rate Limiting Guide; for a Redis-only deep dive, see API Rate Limiting with Redis.

When to Use

Use rate limiting when:

• Public APIs need protection against brute force, scraping, and credential stuffing
• Different user tiers (anonymous, authenticated, premium) require different quotas
• Several API nodes must share rate limit state consistently
• Expensive endpoints (exports, reports) need tighter limits than cheap reads

Skip it for trusted internal service-to-service traffic and webhook receivers that already verify signatures — see When Not to Use below.

In practice most public APIs end up with a tiered model: anonymous requests get a tight IP-based ceiling, authenticated users get a per-account ceiling sized to their plan, and internal services get an exempted or much higher limit. Design the key and the tier table first; the algorithm choice comes second.

Solution

Python (Token Bucket with Redis)

import time
import redis

r = redis.Redis(host='localhost', port=6379, db=0)

def token_bucket(key, capacity=10, refill_rate=1.0):
    """Token bucket using a Redis hash with expiry.

    NOTE: hgetall + hset is not atomic. Two concurrent requests can read
    the same counter and both pass. For strict enforcement use a Lua
    script (see companion resources) — the non-atomic version drifts
    by ~1 request under concurrency, fine for most APIs.
    """
    now = time.time()
    bucket_key = f"rate_limit:{key}"
    tokens = r.hgetall(bucket_key)

    if not tokens:
        r.hset(bucket_key, mapping={"tokens": capacity - 1, "last_refill": now})
        r.expire(bucket_key, int(capacity / refill_rate) + 1)
        return True

    current_tokens = float(tokens.get(b"tokens", 0))
    last_refill = float(tokens.get(b"last_refill", now))
    elapsed = now - last_refill
    current_tokens = min(capacity, current_tokens + elapsed * refill_rate)

    if current_tokens < 1:
        r.hset(bucket_key, mapping={"tokens": current_tokens, "last_refill": now})
        return False

    r.hset(bucket_key, mapping={"tokens": current_tokens - 1, "last_refill": now})
    return True

JavaScript (Sliding Window with Redis)

import { createClient } from 'redis';

const client = createClient({ url: 'redis://localhost:6379' });
await client.connect();

async function slidingWindow(key, limit = 100, windowSec = 60) {
  const now = Date.now();
  const windowStart = now - windowSec * 1000;
  const sortedSetKey = `rate_limit:${key}`;

  // Remove entries outside the window
  await client.zRemRangeByScore(sortedSetKey, 0, windowStart);

  // Count current entries
  const count = await client.zCard(sortedSetKey);

  if (count >= limit) {
    return { allowed: false, retryAfter: windowSec };
  }

  // Add current request
  await client.zAdd(sortedSetKey, [{ score: now, value: `${now}` }]);
  await client.expire(sortedSetKey, windowSec);

  return { allowed: true, remaining: limit - count - 1 };
}

The sorted set stores one member per request scored by timestamp. It’s memory-heavier than a counter: a 100-req/s limiter holds ~128 bytes per key, but it gives you a true rolling window instead of the fixed-window boundary spike. Wire it as middleware before your route handlers so rejected requests never touch your business logic, and return retryAfter in both the JSON body and the Retry-After header.

Java (Leaky Bucket)

import java.util.concurrent.ConcurrentHashMap;

public class LeakyBucket {
    private static final ConcurrentHashMap<String, Bucket> buckets = new ConcurrentHashMap<>();

    static class Bucket {
        final int capacity;
        final double leakRatePerSec;
        double water;
        long lastLeak;

        Bucket(int capacity, double leakRatePerSec) {
            this.capacity = capacity;
            this.leakRatePerSec = leakRatePerSec;
            this.water = 0;
            this.lastLeak = System.currentTimeMillis();
        }

        synchronized boolean allow() {
            long now = System.currentTimeMillis();
            double elapsed = (now - lastLeak) / 1000.0;
            water = Math.max(0, water - elapsed * leakRatePerSec);
            lastLeak = now;
            if (water < capacity) {
                water += 1;
                return true;
            }
            return false;
        }
    }

    public static boolean allowRequest(String clientId, int capacity, double leakRate) {
        return buckets.computeIfAbsent(clientId, k -> new Bucket(capacity, leakRate)).allow();
    }
}

The synchronized on allow() keeps each bucket consistent under concurrent threads; ConcurrentHashMap handles the per-client lookup. This version lives in memory; fine for a single-node deployment or a gateway sidecar. For multi-node consistency you’d move the same leak math into Redis (or just use the sliding-window approach above).

Explanation

The request path never changes. The limiter just sits in front of it:

Token bucket keeps a pool of tokens refilling at a fixed rate. Each request consumes one; empty pool means rejection. Bursts up to capacity pass, while the long-run average stays at the refill rate. Pick it for general public APIs where clients occasionally spike.

Sliding window keeps the request timestamps for the current rolling window and rejects once the count hits the limit. Unlike fixed windows (which let 2× the limit through at the boundary), the window slides with every request. Pick it for payment or sensitive endpoints where the limit must be exact.

Leaky bucket queues incoming requests and drains them at a constant rate, like a bucket with a hole. Arrival bursts don’t matter; output is always smooth. Pick it when protecting a downstream service that can’t absorb bursts at all.

A concrete example makes the difference obvious. A client fires 20 requests in one second against a limit of 10/second. With a token bucket of capacity 10, the first 10 pass instantly and the next 10 are rejected. The burst was legal, the sustained rate wasn’t. With a sliding window at the same limit, the first 10 pass and the rest wait out the window. With a leaky bucket, the 20 requests queue and drain at one per 100ms; none are rejected, but the last one finishes 2 seconds later. Same limit, three different user experiences.

The other decision is the key. rate_limit:ip:203.0.113.7 throttles an address; rate_limit:user:u_4812 throttles an account; rate_limit:key:ak_live_9f3 throttles an API credential. The right key is usually the most specific identity you can trust: an API key for paid APIs, user ID for authenticated traffic, IP only as a fallback for anonymous endpoints.

Variants

Algorithm	Burst Handling	Memory	Best For
Token bucket	Allows bursts up to capacity	Low	APIs with bursty traffic
Sliding window	Strict per-window limit	Medium	Payment APIs, sensitive endpoints
Leaky bucket	Smooths to constant rate	Low	Downstream service protection

For the token-bucket algorithm alone in Redis, see Redis Rate Limiting Token Bucket; for a FastAPI-specific implementation, Python API Rate Limiting covers the framework hooks.

Edge cases worth knowing before you pick. Fixed window (not shown) is the cheapest implementation, one INCR plus EXPIRE, but lets a client fire the full quota at the end of one window and again at the start of the next, doubling the limit at the boundary in effect. Sliding log is the strictest and the most expensive, storing every timestamp. The three algorithms above are the middle ground that production systems actually run.

What Works — and What Breaks

Do:

1. Use token bucket for controlled bursts and sliding window for strict limits
2. Return Retry-After with 429 responses so clients know when to back off
3. Rate limit by user ID or API key, not just IP. Corporate NAT puts thousands of users behind one address
4. Log rate limit violations for security monitoring and abuse detection
5. Fail open around Redis: a dead rate limiter blocking all traffic is a worse outage than a temporarily unlimited API
6. Differentiate limits by endpoint cost: a GET /users is cheap; a POST /reports/export isn’t
7. Store counters in Redis, not memory; in-memory counters don’t survive multi-instance deployments
8. Exempt internal service-to-service calls or give them a much higher ceiling

Don’t:

1. Rate limit only by IP and you’ll block legitimate users behind NAT and miss rotating attackers
2. Return 429 without Retry-After, so clients can’t implement correct backoff and will hammer you harder
3. Use one rate limit for all endpoints regardless of cost or sensitivity
4. Let the limiter fail closed silently. Redis down + fail closed = self-inflicted outage
5. Ignore the violations log. A spike in 429s is either a misconfigured limit or an attack in progress
6. Use fixed windows without jitter, so all clients retry simultaneously at the window boundary (thundering herd)
7. Forget the proxy layer. Behind a load balancer, req.ip is the proxy’s IP until you configure X-Forwarded-For trust
8. Apply limits to GET but not other methods, and attackers switch to POST/PUT to bypass them

Production Checklist

• Rate limits enforced before request processing, not after
• 429 responses include Retry-After, X-RateLimit-Limit, X-RateLimit-Remaining
• Counters scoped per API key or user, not per IP only
• Expensive endpoints have separate, lower limits
• Limits applied to all HTTP methods including OPTIONS and HEAD
• trust proxy configured; X-Forwarded-For validated against trusted proxy IPs
• Redis failover policy chosen deliberately (fail open for most APIs) with alerting
• Internal service-to-service traffic exempt or on a higher ceiling
• Jitter (0-500ms random delay) in client retry logic to prevent boundary stampedes
• Rate limit config externalized (env vars, config service) for hot-reload
• Limits tested in staging with realistic concurrent traffic
• 429 rate monitored; alert above ~5% of total traffic
• Limits documented in API docs and developer portal

Scaling and Performance

Strategy	Overhead per request	Max throughput	Memory per key
In-memory token bucket	0.01ms	500K req/s	64 bytes
Redis fixed window	1-2ms	50K req/s	72 bytes
Redis sliding window	2-3ms	30K req/s	128 bytes
Redis token bucket	1-2ms	50K req/s	64 bytes
Cloudflare edge	0ms (at origin)	Unlimited	0 bytes

In-memory is 100× faster than Redis but doesn’t work across instances. For single-instance deployments it’s the right call; for anything distributed, Redis is the standard. Edge rate limiting (Cloudflare, Fastly) absorbs abuse before it reaches your origin. For public APIs under attack, it’s usually the first thing to buy.

The math at scale: each check costs 1-3ms (a Redis round trip), and each check takes ~2 operations. At 1M requests/second you need Redis Cluster sharded by API key (~2M ops/s at peak). If your latency budget is under 10ms, use local token buckets with periodic Redis sync every ~100ms. You trade ~10% precision under contention for speed. In multi-region deployments, Redis must be co-located with each region’s instances; cross-region calls add 50-200ms and defeat the purpose.

Component	Cost	Notes
Redis (single, 1GB)	$10-$25/month	AWS ElastiCache, GCP Memorystore
Redis Cluster (3 shards)	$75-$200/month	For >100K ops/second
Cloudflare Rate Limiting	$5-$25/month	Edge-level, no Redis needed
AWS API Gateway throttling	$0	Built-in, per-stage limits

Security Considerations

• Key spoofing: clients who can fake their IP or user ID bypass the limit. Validate X-Forwarded-For against your trusted proxy IPs. Trust it blindly and you’ve made things worse than ignoring it.
• Under-limit DDoS: attackers send just-under-limit traffic from many IPs to exhaust resources without ever triggering a 429. Combine rate limits with connection limits and body-size limits; OWASP’s API Security Top 10 covers unrestricted resource consumption for a reason.
• Redis as a single point of failure: run Sentinel or Cluster mode. A limiter that blocks all traffic when the store dies is a load-bearing piece of availability.
• Header leakage: X-RateLimit-Limit and X-RateLimit-Remaining reveal your configuration to attackers. Consider omitting or approximating them for unauthenticated traffic. The emerging RateLimit header standard gives you a cleaner alternative to custom X- headers.
• Method switching: limits applied only to GET are trivially bypassed via POST, PUT, or even OPTIONS.
• Path variation: /api/users?x=1 and /api/users?x=2 are different keys to a naive limiter. Normalize URLs and rate limit by endpoint pattern, not full path.
• Timing attacks: if denied requests resolve measurably faster than allowed ones, attackers can infer limit state. Return early decisions through the same code path.

Monitoring and Testing

Watch four signals. 429 response rate: a sudden jump means a misconfigured limit or an abusive client. Alert above ~5% of traffic. Redis latency for limit checks: if p95 exceeds 5ms, your limiter is now a bottleneck itself. Key count growth: unbounded growth means clients are generating unique IDs to evade limits; cap it and expire aggressively. Limiter failures: a crashed limiter or unreachable Redis needs an immediate page, because you just lost either your protection or your availability.

Test four scenarios. Enforcement: send N+1 requests rapidly; the (N+1)th must return 429 with correct headers. Window reset: request at the limit boundary (59.9s, 60.1s) and verify the reset behaves as documented. Redis failover: kill Redis mid-test and confirm the limiter fails the way your policy says (open or closed), not the other way. Distributed consistency: run instances in parallel and verify the aggregate limit holds across all of them.

Troubleshooting

• Limits not enforced across instances. Counters are in memory, not Redis, so each node has its own limit. Move state to a shared store.
• Everyone gets 429 behind a proxy. req.ip is the load balancer’s IP; configure trust proxy and read X-Forwarded-For.
• Legitimate corporate users blocked. They’re behind NAT sharing one IP. Switch the key to user ID or API key.
• API goes down when Redis does. The limiter fails closed. Decide your policy deliberately: fail open for most public APIs, fail closed only for endpoints where unlimited access is worse than downtime.
• Thundering herd at window boundaries. Fixed windows reset for everyone at once and all clients retry simultaneously. Use sliding windows or add jitter to retries.
• 429s but no Retry-After honored. Some clients ignore the header by design. Log violators and consider longer blocks for repeat offenders.

When Not to Use This Approach

• Internal microservices with trusted callers — rate limiting inter-service traffic adds latency and cascade-failure risk for no security gain. Use service quotas at the gateway instead.
• Webhook receivers with signature verification — if every request passes an HMAC check, invalid traffic is already rejected before the limiter runs.
• Batch processing APIs — a batch job sending 10K requests in 10 seconds is expected behavior, not abuse. Use queue-based throttling instead of per-request limits.

See Also

• API Rate Limiting Guide — the design walkthrough behind this recipe
• API Rate Limiting with Redis — Redis-specific implementation deep dive
• API Security Checklist Guide — rate limiting as one layer of API defense
• API Error Response Template — format the 429 body correctly
• Redis commands — INCR, EXPIRE, ZADD, ZREMRANGEBYSCORE used in these implementations
• RateLimit header fields draft — the emerging standard replacing custom X-RateLimit-* headers
• OWASP API Security Top 10 — unrestricted resource consumption and what limits alone can’t stop

Companion code: API rate limiting resources — atomic Lua token bucket, Express middleware, and a pytest suite for the scenarios above.