Database Caching at Scale · Cache-Aside Done Right

Cache-Aside in Practice

The read path, the write path and the details (keys, TTLs, serialization, failures) that decide whether a cache helps or hurts.

Scenario

Example app: an online store's product page reads a product, its price and its stock from Postgres: 3 queries, ~12 ms. The page gets 8,000 requests per second during a sale; 95% of views hit the same 2,000 products.

What you see in production

The team adds Redis. Database load drops — but support reports wrong prices for up to an hour after merchandisers change them, and when Redis restarts the site slows to a crawl. One deploy that changed the Product class broke every product page with deserialization errors.

The naive code

Cache reads with a long TTL and update the cache on write:

def get_product(pid):
    data = r.get(f"product:{pid}")
    if data:
        return pickle.loads(data)                       # class layout baked into the cache
    product = Product.objects.get(pk=pid)
    r.set(f"product:{pid}", pickle.dumps(product), ex=3600)
    return product

def update_price(pid, price):
    Product.objects.filter(pk=pid).update(price=price)
    r.set(f"product:{pid}", pickle.dumps(Product.objects.get(pk=pid)))   # no TTL now, and racy

Reproduce it

A cache-aside read path with hit/miss accounting, then a price change that forgets to invalidate:

Cache-aside: hits, misses and a forgotten invalidationTry it yourself
db = {"p1": {"price": 100}, "p2": {"price": 50}}
cache, stats = {}, {"hit": 0, "miss": 0}

def get(pid):
    if pid in cache:
        stats["hit"] += 1
        return cache[pid]
    stats["miss"] += 1
    value = dict(db[pid])              # SELECT … (copy, like deserializing)
    cache[pid] = value                 # SET product:{pid} … EX 3600
    return value

for pid in ["p1", "p1", "p2", "p1", "p2", "p1"]:
    get(pid)
print("after 6 reads:", stats, f"hit ratio {stats['hit'] / 6:.0%}")

db["p1"]["price"] = 80                 # merchandiser changes the price; cache not invalidated
print("database price:", db["p1"]["price"], "| page shows:", get("p1")["price"], "<- stale until TTL")
cache.pop("p1")                        # the fix: DEL product:p1 after the commit
print("after invalidation, page shows:", get("p1")["price"])

Output

after 6 reads: {'hit': 4, 'miss': 2} hit ratio 67%
database price: 80 | page shows: 100 <- stale until TTL
after invalidation, page shows: 80

Root cause

Cache-aside (also called look-aside) puts the application in charge of keeping the cache correct. On reads it fills the cache on a miss; on writes it must invalidate whatever the write made wrong. Forgetting the write path — or updating the cache with a value read at the wrong moment — leaves stale data until the TTL. Language-specific serialization (pickle, Java serialization) ties cached bytes to class layouts, so a deploy can make every entry unreadable.

The fix

KEY = "v3:product:{}"                                 # schema version in the key

def get_product(pid):
    data = r.get(KEY.format(pid))
    if data is not None:
        return ProductDTO.model_validate_json(data)    # explicit, versioned JSON
    product = ProductDTO.from_model(Product.objects.get(pk=pid))
    r.set(KEY.format(pid), product.model_dump_json(), ex=300 + random.randint(0, 60))
    return product

def update_price(pid, price):
    with transaction.atomic():
        Product.objects.filter(pk=pid).update(price=price)
        transaction.on_commit(lambda: r.delete(KEY.format(pid)))   # delete AFTER commit

Rules of thumb:

  • Invalidate (delete) on write, after commit — don't compute and set the new value in the write path. Facebook's memcache paper describes the same choice: "We choose to delete cached data instead of updating it because deletes are idempotent" (Scaling Memcache at Facebook).
  • Always set a TTL — it bounds the damage of every missed invalidation.
  • Cache DTOs as JSON (or another explicit format) with a version in the key.
  • Treat the cache as optional: on a Redis error, fall back to the database (with protection — see stampedes).

Trade-offs

  • Cache-aside caches only what's read (memory-efficient) but the first read after each invalidation is slow.
  • Short TTLs reduce staleness but increase misses; pick per data type (prices minutes, product descriptions hours).
  • Caching whole ORM objects is convenient but brittle; cache the shape the page needs.

Detect it in production

  • Hit ratio per key prefix, cache latency, and database QPS from cache misses.
  • "Staleness probes": periodically compare a sample of cached values with the database.
  • Deserialization error counts after deploys.

Key points

  • Cache-aside: read → miss → load → set with TTL; write → commit → delete.
  • Delete instead of update; always use TTLs; version keys and use explicit serialization.
  • The cache must be optional and its correctness monitored.

Practice questions

Why invalidate (delete) the cache entry on write instead of setting the new value?

Answer

Deletes are idempotent and can't write an older value over a newer one; the next read loads the current row. Setting values from the write path can race with other writers and readers and leave stale data.

Why should every cache entry have a TTL even if you invalidate on write?

Answer

Invalidations get missed (bugs, crashes, network errors, new code paths). A TTL bounds how long a wrong value can live.

A deploy changed a class and every product page started failing. What caching practice prevents this?

Answer

Cache explicit, versioned representations (e.g. JSON DTOs) and include a schema version in the key, so new code reads new keys instead of deserializing old binary objects.

Open in the app to mark lessons complete, take notes and track your progress.