Database Caching at Scale · Operating Caches in Production

Memory, Eviction Policies and Serialization

What happens when the cache fills up — and why the wrong maxmemory-policy turns a full cache into write errors.

Scenario

Example app: a Redis instance holds both the cache and session data and rate-limit counters. It runs with the default maxmemory-policy noeviction. Traffic grows; memory reaches maxmemory.

What you see in production

Suddenly every cache write fails with OOM command not allowed when used memory > 'maxmemory' — and so do logins (sessions) and the rate limiter. After switching to allkeys-lru, logins start failing randomly: sessions are being evicted.

The naive code

maxmemory 4gb
maxmemory-policy noeviction     # default: refuse writes when full
# sessions, rate limits and cache keys all share this instance

Reproduce it

A tiny 3-slot store under different eviction policies:

noeviction vs allkeys-lru vs volatile-lruTry it yourself
from collections import OrderedDict

def run(policy):
    store = OrderedDict()                       # key -> has_ttl (LRU order)
    log = []
    for key, ttl in [("session:ada", False), ("cache:p1", True), ("cache:p2", True), ("cache:p3", True)]:
        if len(store) >= 3:
            candidates = [k for k, t in store.items() if policy == "allkeys-lru" or (policy == "volatile-lru" and t)]
            if policy == "noeviction" or not candidates:
                log.append(f"SET {key} -> OOM error")
                continue
            victim = candidates[0]               # least recently used among candidates
            store.pop(victim)
            log.append(f"evicted {victim}")
        store[key] = ttl
    return log, list(store)

for policy in ("noeviction", "allkeys-lru", "volatile-lru"):
    log, keys = run(policy)
    print(f"{policy:12} {log} -> kept {keys}")

Output

noeviction   ['SET cache:p3 -> OOM error'] -> kept ['session:ada', 'cache:p1', 'cache:p2']
allkeys-lru  ['evicted session:ada'] -> kept ['cache:p1', 'cache:p2', 'cache:p3']
volatile-lru ['evicted cache:p1'] -> kept ['session:ada', 'cache:p2', 'cache:p3']

Root cause

When Redis reaches maxmemory, the eviction policy decides what happens: noeviction rejects writes; allkeys-* may evict anything, including data you can't lose; volatile-* evicts only keys with a TTL. Mixing caches (disposable) with sessions/counters (not disposable) in one instance makes every policy wrong for something.

The fix

  • Separate concerns: a cache instance with allkeys-lru (or allkeys-lfu) and a separate instance (or at least a separate policy) for sessions, locks and counters, sized with headroom and persistence as needed.
  • Every cache key has a TTL; durable-ish keys too where they naturally expire (sessions).
  • Serialize compactly and explicitly: JSON/MessagePack with only the fields you need, versioned keys; compress large values.
maxmemory 4gb
maxmemory-policy allkeys-lfu     # keep frequently used keys; evict cold ones

Trade-offs

  • LFU keeps popular keys better than LRU under scans, but reacts more slowly to shifting popularity.
  • Separate instances cost more but isolate failure modes (a cache flush can't log everyone out).
  • Compression saves memory and bandwidth at some CPU cost; use it for big values only.

Detect it in production

  • INFO memory (used_memory, fragmentation ratio) and INFO stats (evicted_keys, rejected writes via errors).
  • Alert at 80% of maxmemory; graph evictions — a sudden rise lowers hit ratio.
  • Key-size sampling (--bigkeys, MEMORY USAGE key).

Key points

  • The eviction policy decides what a full cache does; noeviction turns fullness into write errors.
  • Don't mix disposable cache data with sessions/counters under one policy.
  • TTLs everywhere, compact versioned serialization, and memory alerts.

Practice questions

What happens when Redis with `noeviction` reaches maxmemory?

Answer

Commands that would use more memory fail with an OOM error; reads still work, but cache fills, session writes and counters start erroring.

Why is `allkeys-lru` risky if sessions live in the same Redis as the cache?

Answer

It may evict any key, including sessions, logging users out at random when memory is tight. Keep sessions in a separate instance or use a policy that only evicts cache keys (with TTLs).

What's the difference between volatile-* and allkeys-* eviction policies?

Answer

volatile-* policies evict only keys that have a TTL; allkeys-* policies may evict any key.

Open in the app to mark lessons complete, take notes and track your progress.