Two-Level Cache Pattern

Overview

A two-level cache combines an L1 in-memory cache (fast, local, limited size) with an L2 distributed cache (slower, shared, larger capacity). L1 serves hot keys at memory speed with zero network overhead. L2 provides shared cache state across application instances and handles the full dataset.

Reads check L1 first. On a miss, they check L2. On an L2 miss, they fetch from the database and populate both levels. Writes update both levels and invalidate L1 on other instances via pub/sub or TTL.

When to Use

• Hot keys are read frequently and network latency to Redis is noticeable
• Multiple application instances share cached data
• You need sub-millisecond read latency for specific keys
• L1 miss rate is low enough that the extra complexity is justified
• You want to reduce load on the distributed cache (Redis)

Solution

Python with LRU L1 and Redis L2

import redis
import json
import time
from functools import lru_cache
from collections import OrderedDict

class TwoLevelCache:
    def __init__(self, redis_client: redis.Redis, l1_max_size: int = 1000, l1_ttl: float = 60.0, l2_ttl: int = 3600):
        self.redis = redis_client
        self.l1_max_size = l1_max_size
        self.l1_ttl = l1_ttl
        self.l2_ttl = l2_ttl
        self._l1 = OrderedDict()  # key -> (value, expiry_timestamp)

    def get(self, key: str, loader: callable) -> any:
        # L1 check
        if key in self._l1:
            value, expiry = self._l1[key]
            if time.time() < expiry:
                self._l1.move_to_end(key)
                return value
            del self._l1[key]

        # L2 check
        cached = self.redis.get(key)
        if cached is not None:
            value = json.loads(cached)
            self._populate_l1(key, value)
            return value

        # Cache miss — load from database
        value = loader()
        self._populate_l1(key, value)
        self.redis.setex(key, self.l2_ttl, json.dumps(value))
        return value

    def _populate_l1(self, key: str, value: any):
        if len(self._l1) >= self.l1_max_size:
            self._l1.popitem(last=False)  # Evict LRU
        self._l1[key] = (value, time.time() + self.l1_ttl)

    def invalidate(self, key: str):
        self._l1.pop(key, None)
        self.redis.delete(key)

    def invalidate_all(self):
        self._l1.clear()


cache = TwoLevelCache(redis.Redis(host='localhost', port=6379))

def get_user(user_id: str) -> dict:
    return cache.get(
        f"user:{user_id}",
        lambda: db.query_one("SELECT * FROM users WHERE id = %s", [user_id])
    )

def update_user(user_id: str, name: str, email: str):
    db.execute("UPDATE users SET name=%s, email=%s WHERE id=%s", [name, email, user_id])
    cache.invalidate(f"user:{user_id}")

TypeScript with Map L1 and Redis L2

import { createClient } from 'redis';

class TwoLevelCache {
  private l1: Map<string, { value: any; expiry: number }> = new Map();
  private l1MaxSize: number;
  private l1TtlMs: number;
  private l2Ttl: number;
  private redis: ReturnType<typeof createClient>;

  constructor(
    redisClient: ReturnType<typeof createClient>,
    l1MaxSize = 1000,
    l1TtlMs = 60000,
    l2Ttl = 3600
  ) {
    this.redis = redisClient;
    this.l1MaxSize = l1MaxSize;
    this.l1TtlMs = l1TtlMs;
    this.l2Ttl = l2Ttl;
  }

  async get<T>(key: string, loader: () => Promise<T>): Promise<T> {
    // L1 check
    const l1Entry = this.l1.get(key);
    if (l1Entry && Date.now() < l1Entry.expiry) {
      // Refresh LRU position
      this.l1.delete(key);
      this.l1.set(key, l1Entry);
      return l1Entry.value as T;
    }
    this.l1.delete(key);

    // L2 check
    const l2Cached = await this.redis.get(key);
    if (l2Cached !== null) {
      const value = JSON.parse(l2Cached) as T;
      this.populateL1(key, value);
      return value;
    }

    // Cache miss — load from database
    const value = await loader();
    this.populateL1(key, value);
    await this.redis.set(key, JSON.stringify(value), { EX: this.l2Ttl });
    return value;
  }

  private populateL1(key: string, value: any): void {
    if (this.l1.size >= this.l1MaxSize) {
      // Evict oldest entry (first in Map)
      const oldestKey = this.l1.keys().next().value;
      if (oldestKey) this.l1.delete(oldestKey);
    }
    this.l1.set(key, { value, expiry: Date.now() + this.l1TtlMs });
  }

  invalidate(key: string): void {
    this.l1.delete(key);
    this.redis.del(key);
  }

  invalidateAll(): void {
    this.l1.clear();
  }
}

// Usage
const redisClient = createClient({ url: 'redis://localhost:6379' });
await redisClient.connect();
const cache = new TwoLevelCache(redisClient);

const user = await cache.get(`user:${userId}`, () =>
  db.query('SELECT * FROM users WHERE id = $1', [userId])
);

Java with Caffeine L1 and Redis L2

import com.github.benmanes.caffeine.cache.Cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import redis.clients.jedis.Jedis;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.util.concurrent.TimeUnit;

public class TwoLevelCacheManager {

    private final Cache<String, Object> l1;
    private final Jedis l2;
    private final ObjectMapper mapper = new ObjectMapper();
    private final int l2Ttl;

    public TwoLevelCacheManager(Jedis jedis, int l1MaxSize, int l1TtlMinutes, int l2TtlSeconds) {
        this.l2 = jedis;
        this.l2Ttl = l2TtlSeconds;
        this.l1 = Caffeine.newBuilder()
            .maximumSize(l1MaxSize)
            .expireAfterWrite(l1TtlMinutes, TimeUnit.MINUTES)
            .build();
    }

    @SuppressWarnings("unchecked")
    public <T> T getOrLoad(String key, java.util.function.Function<String, T> loader, Class<T> type) {
        // L1 check
        T l1Value = (T) l1.getIfPresent(key);
        if (l1Value != null) {
            return l1Value;
        }

        // L2 check
        String l2Cached = l2.get(key);
        if (l2Cached != null) {
            try {
                T value = mapper.readValue(l2Cached, type);
                l1.put(key, value);
                return value;
            } catch (Exception e) {
                // Deserialization failed, treat as miss
            }
        }

        // Cache miss — load from database
        T value = loader.apply(key);
        try {
            l2.setex(key, l2Ttl, mapper.writeValueAsString(value));
        } catch (Exception e) {
            // Serialization failed, skip L2
        }
        l1.put(key, value);
        return value;
    }

    public void invalidate(String key) {
        l1.invalidate(key);
        l2.del(key);
    }
}

Explanation

The two-level cache exploits locality of reference. Hot keys are served from L1 (in-memory), which has sub-microsecond access time. Cold keys fall through to L2 (Redis), which has network latency but serves all instances.

The L1 cache is per-instance. Each application instance maintains its own L1. This means L1 can have stale data if another instance updates the database. To mitigate this, L1 has a short TTL (e.g. 60 seconds). The L2 cache is shared and has a longer TTL (e.g. 1 hour).

On a write, both L1 and L2 are invalidated. Other instances still have the old value in their L1 until their L1 TTL expires. For tighter consistency, use Redis pub/sub to broadcast invalidation events so all instances clear their L1 immediately.

Variants

Approach	L1 Implementation	L2 Implementation	Consistency
LRU + Redis	OrderedDict / Map	Redis	Eventual (L1 TTL)
Caffeine + Redis	Caffeine	Redis	Eventual (L1 TTL)
Pub/sub invalidation	Any in-memory	Redis	Strong (on invalidate)
Write-through both	Any in-memory	Redis	Strong
L1-only (no L2)	LRU / Caffeine	None	Per-instance, no sharing

Best Practices

• For a deeper guide, see Read-Through Cache Pattern.

• Keep L1 small — L1 is per-instance and consumes application memory. 1000-5000 entries is typical. Use LRU eviction to bound size.

• Set a short L1 TTL — L1 should expire faster than L2 (e.g. 60s vs 1h). This limits staleness when other instances update data.

• Use pub/sub for L1 invalidation — when one instance invalidates a key, broadcast to all instances so they clear their L1. This reduces the staleness window from the L1 TTL to near-zero.

• Monitor L1 hit ratio — if L1 hit ratio is low, the extra complexity is not justified. L1 is only valuable for hot keys accessed many times per second.

• Serialize consistently — L1 stores objects in memory, L2 stores serialized JSON. Ensure the serialization round-trip preserves data types.

Common Mistakes

• L1 too large — consuming too much application memory causes GC pressure or OOM. Cap L1 size and use LRU eviction.
• L1 TTL same as L2 TTL — if both expire at the same time, L1 does not reduce L2 load. Set L1 TTL shorter (e.g. 1/10 of L2 TTL).
• No L1 invalidation across instances — instance A updates data and clears its L1. Instance B still serves stale data from its L1. Use pub/sub or accept the TTL-based staleness.
• Caching everything in L1 — L1 is for hot keys only. Caching cold keys in L1 wastes memory and evicts hot keys. Use access frequency to decide what enters L1.
• Not handling L2 failures — if Redis is down, L1 still works but L2 misses become database hits. Implement a circuit breaker or fallback to L1-only mode.

Troubleshooting

• Cache and database are out of sync: define a TTL or invalidation policy.
• Hit rate dropped after a deployment: check cache key generation and serialization changes. A new version may use different keys.
• Cold cache causes thundering herd: use cache warming, request coalescing, or single-flight patterns for hot keys.
• Memory usage grows uncontrollably: set max memory policies, eviction thresholds, and key expiration. Audit large values.
• Stale data served to users: implement cache invalidation on write and cache-bust URLs for static assets.

Production Notes

• Deploy gradually using canary or blue-green to catch regressions early.
• Configure alerts for error rate, p99 latency, and failure rate before enabling in production.
• Document the rollback in the runbook; test the procedure in staging at least once per quarter.
• Review structured logs with correlation IDs to trace requests end-to-end during incidents.

Key Takeaways

• Apply two-level cache pattern when you need a practical solution for your use case.
• Monitor performance after implementation; measure latency, errors, and resource usage before and after.
• Check the Troubleshooting section for common failures; most have documented root causes with fixes.
• Keep dependencies updated and run tests in CI to prevent production regressions.

Common Production Pitfalls

• Applying the pattern where no abstraction is needed, adding accidental complexity.
• Letting the pattern leak into unrelated modules and blur ownership boundaries.
• Over-engineering the first implementation instead of starting simple and measuring pain.
• Skipping contract tests, so refactors silently break consumers.
• Ignoring failure modes that the pattern does not cover.
• Using the pattern as a default instead of choosing the right tool for the current scale.
• Forgetting to document when to stop using the pattern and what replaces it.
• Missing observability around the pattern’s performance and error propagation.