Database Caching at Scale · Operating Caches in Production
Memory, Eviction Policies and Serialization
What happens when the cache fills up — and why the wrong maxmemory-policy turns a full cache into write errors.
Scenario
Example app: a Redis instance holds both the cache and session data and rate-limit counters. It runs with the default maxmemory-policy noeviction. Traffic grows; memory reaches maxmemory.
What you see in production
Suddenly every cache write fails with OOM command not allowed when used memory > 'maxmemory' — and so do logins (sessions) and the rate limiter. After switching to allkeys-lru, logins start failing randomly: sessions are being evicted.
The naive code
maxmemory 4gb
maxmemory-policy noeviction # default: refuse writes when full
# sessions, rate limits and cache keys all share this instance
r.set(f"session:{sid}", payload) # no TTL
r.set(f"cache:product:{pid}", data, ex=600)
Reproduce it
A tiny 3-slot store under different eviction policies:
from collections import OrderedDict
def run(policy):
store = OrderedDict() # key -> has_ttl (LRU order)
log = []
for key, ttl in [("session:ada", False), ("cache:p1", True), ("cache:p2", True), ("cache:p3", True)]:
if len(store) >= 3:
candidates = [k for k, t in store.items() if policy == "allkeys-lru" or (policy == "volatile-lru" and t)]
if policy == "noeviction" or not candidates:
log.append(f"SET {key} -> OOM error")
continue
victim = candidates[0] # least recently used among candidates
store.pop(victim)
log.append(f"evicted {victim}")
store[key] = ttl
return log, list(store)
for policy in ("noeviction", "allkeys-lru", "volatile-lru"):
log, keys = run(policy)
print(f"{policy:12} {log} -> kept {keys}")
Output
noeviction ['SET cache:p3 -> OOM error'] -> kept ['session:ada', 'cache:p1', 'cache:p2'] allkeys-lru ['evicted session:ada'] -> kept ['cache:p1', 'cache:p2', 'cache:p3'] volatile-lru ['evicted cache:p1'] -> kept ['session:ada', 'cache:p2', 'cache:p3']
Root cause
When Redis reaches maxmemory, the eviction policy decides what happens: noeviction rejects writes; allkeys-* may evict anything, including data you can't lose; volatile-* evicts only keys with a TTL. Mixing caches (disposable) with sessions/counters (not disposable) in one instance makes every policy wrong for something.
The fix
- Separate concerns: a cache instance with
allkeys-lru(orallkeys-lfu) and a separate instance (or at least a separate policy) for sessions, locks and counters, sized with headroom and persistence as needed. - Every cache key has a TTL; durable-ish keys too where they naturally expire (sessions).
- Serialize compactly and explicitly: JSON/MessagePack with only the fields you need, versioned keys; compress large values.
maxmemory 4gb
maxmemory-policy allkeys-lfu # keep frequently used keys; evict cold ones
payload = msgpack.packb({"id": p.id, "name": p.name, "price": str(p.price)}) # only what pages use
r.set(f"v3:product:{p.id}", payload, ex=ttl(600))
RedisCacheConfiguration.defaultCacheConfig()
.serializeValuesWith(RedisSerializationContext.SerializationPair.fromSerializer(new GenericJackson2JsonRedisSerializer()))
.prefixCacheNameWith("v3:")
.entryTtl(Duration.ofMinutes(10));
Trade-offs
- LFU keeps popular keys better than LRU under scans, but reacts more slowly to shifting popularity.
- Separate instances cost more but isolate failure modes (a cache flush can't log everyone out).
- Compression saves memory and bandwidth at some CPU cost; use it for big values only.
Detect it in production
INFO memory(used_memory, fragmentation ratio) andINFO stats(evicted_keys,rejectedwrites via errors).- Alert at 80% of maxmemory; graph evictions — a sudden rise lowers hit ratio.
- Key-size sampling (
--bigkeys,MEMORY USAGE key).
Key points
- The eviction policy decides what a full cache does;
noevictionturns fullness into write errors. - Don't mix disposable cache data with sessions/counters under one policy.
- TTLs everywhere, compact versioned serialization, and memory alerts.
Practice questions
What happens when Redis with `noeviction` reaches maxmemory?
Answer
Commands that would use more memory fail with an OOM error; reads still work, but cache fills, session writes and counters start erroring.
Why is `allkeys-lru` risky if sessions live in the same Redis as the cache?
Answer
It may evict any key, including sessions, logging users out at random when memory is tight. Keep sessions in a separate instance or use a policy that only evicts cache keys (with TTLs).
What's the difference between volatile-* and allkeys-* eviction policies?
Answer
volatile-* policies evict only keys that have a TTL; allkeys-* policies may evict any key.
Open in the app to mark lessons complete, take notes and track your progress.