Caching & Memoization in Python, JavaScript, and Java

Overview

Caching is one of the cheapest ways to speed up repeated work. You compute something once, store the result, and serve the next request without doing the work again. Memoization is just caching for function return values, keyed by the arguments you passed in. The catch is more complexity: stale data, invalidation, consistency headaches — the kind of stuff that bites you in distributed systems.

I’ve reached for caching on projects where a single database query took 800ms and got hammered 200 times per second. Slapping a 60-second TTL cache in front of it dropped the average response time to 2ms. That’s the kind of win caching hands you — but only if you stay on top of invalidation. For a deeper dive into the cache-aside pattern with Redis, check our Redis cache-aside recipe. If you’re looking at multi-level setups, our multi-level cache L1/L2 recipe walks through the architecture in detail.

When to Use

• An expensive database query or API call is hit again and again.
• A function does heavy math or statistical computation that you don’t want to repeat.
• The data changes slowly, such as configuration or reference data.
• Latency is a problem and your system does way more reads than writes.
• You want to take load off a downstream service.

When NOT to Use

• The underlying data changes faster than you can invalidate the cache.
• Strong consistency is required and even a short stale read is unacceptable.
• The working set is bigger than the cache memory you’ve got, and there’s no eviction policy.
• You haven’t measured the bottleneck. Cache only after profiling.

Solution

Python

from functools import lru_cache
from cachetools import TTLCache

# Built-in LRU memoization
@lru_cache(maxsize=128)
def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

print(fibonacci(100))  # Instant, cached

# TTL cache with expiration
api_cache = TTLCache(maxsize=100, ttl=300)  # 5 minutes

def fetch_user(user_id):
    if user_id in api_cache:
        return api_cache[user_id]
    user = db.query("SELECT * FROM users WHERE id = %s", user_id)
    api_cache[user_id] = user
    return user

JavaScript

// Simple memoization
function memoize(fn) {
  const cache = new Map();
  return (...args) => {
    const key = JSON.stringify(args);
    if (cache.has(key)) return cache.get(key);
    const result = fn(...args);
    cache.set(key, result);
    return result;
  };
}

const fib = memoize((n) => (n < 2 ? n : fib(n - 1) + fib(n - 2)));
console.log(fib(100)); // Instant

// LRU cache with size limit
class LRUCache {
  constructor(capacity) {
    this.capacity = capacity;
    this.cache = new Map();
  }
  get(key) {
    if (!this.cache.has(key)) return undefined;
    const value = this.cache.get(key);
    this.cache.delete(key);
    this.cache.set(key, value);
    return value;
  }
  set(key, value) {
    if (this.cache.has(key)) this.cache.delete(key);
    else if (this.cache.size >= this.capacity) {
      const first = this.cache.keys().next().value;
      this.cache.delete(first);
    }
    this.cache.set(key, value);
  }
}

Java with Caffeine

import com.github.benmanes.caffeine.cache.*;

Cache<String, User> userCache = Caffeine.newBuilder()
    .maximumSize(100)
    .expireAfterWrite(Duration.ofMinutes(5))
    .build();

// Get or compute
User user = userCache.get(userId, id -> db.findById(id));

// Manual put
userCache.put(userId, updatedUser);

// Invalidate
userCache.invalidate(userId);

Redis cache-aside

import redis
import json

r = redis.Redis(host="localhost", port=6379, decode_responses=True)

def get_user(user_id):
    cached = r.get(f"user:{user_id}")
    if cached:
        return json.loads(cached)
    user = db.find(user_id)
    r.setex(f"user:{user_id}", 300, json.dumps(user))
    return user

Cache stampede prevention

A cache stampede happens when a bunch of requests hit a missing or expired key at the same time and all of them fetch from the source. The fix: let only one request fetch, and have the rest wait for it.

Python with a per-key lock:

import threading
import time

_locks = {}
_locks_guard = threading.Lock()

def get_with_lock(key, fetch_fn, ttl=300):
    cached = r.get(key)
    if cached:
        return json.loads(cached)
    with _locks_guard:
        if key not in _locks:
            _locks[key] = threading.Lock()
    with _locks[key]:
        # Double-check after acquiring lock
        cached = r.get(key)
        if cached:
            return json.loads(cached)
        value = fetch_fn()
        r.setex(key, ttl, json.dumps(value))
        return value

JavaScript with a promise-based single-flight:

const inflight = new Map();

async function getWithSingleFlight(key, fetchFn, ttlSeconds = 300) {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  if (inflight.has(key)) return inflight.get(key);

  const promise = fetchFn().then((value) => {
    redis.setex(key, ttlSeconds, JSON.stringify(value));
    inflight.delete(key);
    return value;
  });
  inflight.set(key, promise);
  return promise;
}

I’ve seen stampedes take down a database during a traffic spike. Single-flight or per-key locking is cheap insurance — add it from the start if you expect any concurrency.

Multi-level cache (L1 in-memory + L2 Redis)

A multi-level cache keeps a small in-memory cache (L1) in front of a shared Redis cache (L2). L1 serves the hottest keys in under a millisecond; L2 picks up everything else and stays consistent across instances.

from cachetools import TTLCache

l1 = TTLCache(maxsize=100, ttl=30)  # 30s, per-instance
l2 = redis.Redis(host="localhost", port=6379, decode_responses=True)

def get_user(user_id):
    # L1 hit
    if user_id in l1:
        return l1[user_id]
    # L2 hit
    cached = l2.get(f"user:{user_id}")
    if cached:
        user = json.loads(cached)
        l1[user_id] = user
        return user
    # Miss — fetch from DB
    user = db.find(user_id)
    l2.setex(f"user:{user_id}", 300, json.dumps(user))
    l1[user_id] = user
    return user

The trade-off is complexity: you now have two caches to invalidate. Use pub/sub to propagate L1 invalidations across instances, or keep L1 TTLs short enough that staleness is bounded. For the full architecture, see our multi-level cache L1/L2 recipe.

Library comparison

Language	Library	Scope	TTL	LRU	Distributed
Python	functools.lru_cache	Memoization	No	Yes	No
Python	cachetools	In-memory	Yes	Yes	No
Python	redis-py	Distributed	Yes	No	Yes
JavaScript	lru-cache (npm)	In-memory	Yes	Yes	No
JavaScript	node-redis	Distributed	Yes	No	Yes
Java	Caffeine	In-memory	Yes	Yes	No
Java	Spring Cache + Redis	Distributed	Yes	Yes	Yes

I reach for functools.lru_cache for pure Python memoization, cachetools when I need TTL, and Redis the moment I need to share cache state across processes. In JavaScript, lru-cache (npm) is what I reach for — it’s fast, well-tested, and handles both TTL and LRU. In Java, Caffeine is the best in-memory option I’ve tried; Spring Cache abstracts the provider so you can swap Caffeine for Redis without touching business code.

Explanation

A cache sits between the caller and the expensive data source. On a hit, it returns the stored value. On a miss, it fetches, stores, and returns the value. TTL limits staleness, maximum size triggers eviction, and invalidation removes entries when the underlying data changes.

Variants

Strategy	When to use	Trade-off
TTL	Data changes predictably	May serve stale data briefly
Write-through	Consistency is critical	Slower writes, simpler reads
Write-behind	High write throughput	Risk of data loss on crash
Cache-aside	Flexibility, read-heavy	Application manages cache logic
Eviction (LRU/LFU)	Memory constraints	May evict hot data prematurely

Best Practices

• Cache the most expensive and most frequently accessed data, not every value in sight.
• Set TTLs carefully. Too short makes the cache useless; too long serves stale data.
• Monitor hit rates. A cache below 80% is usually not worth the trouble.
• Handle cache failures gracefully. If Redis goes down, fall back to the database.
• Version cache keys or include the app version to avoid stale data after deployments.
• Invalidate proactively when the source data changes, instead of waiting for TTL.

Common Mistakes

• Caching data that changes too frequently or is rarely requested.
• Not handling cache stampede when a popular key expires.
• Storing unbounded caches that grow until they eat all your memory.
• Ignoring cache consistency in distributed systems.
• Forgetting to invalidate the cache after writes.

See Also

• Redis documentation — official Redis docs, packed with data structures, commands, and caching patterns.
• Caffeine cache docs — Caffeine wiki with configuration, eviction policies, and performance benchmarks.
• cachetools docs — the Python caching library reference, with TTLCache, LRUCache, and LFUCache.
• lru-cache (npm) — JavaScript LRU cache with TTL support, the standard choice for Node.js in-memory caching.
• Java Caffeine recipe — our deep dive on Caffeine configuration for Java apps.
• Node.js in-memory LRU recipe — our guide to LRU caching in Node.js with the lru-cache package.