Choosing TTLs and Adding Jitter

Scenario

Example app: every morning at 09:00 a batch job warms the cache for 200,000 products with EX 3600. Product data otherwise changes a few times a day.

What you see in production

At 10:00:00 every day, database CPU spikes to 100% for about a minute and p99 latency triples — exactly one hour after the warm-up. Nothing else happens at 10:00.

The naive code

for product in Product.objects.all().iterator():
    r.set(f"product:{product.id}", serialize(product), ex=3600)   # all expire together

RedisCacheConfiguration.defaultCacheConfig().entryTtl(Duration.ofHours(1));   // same TTL for every entry

Reproduce it

Expiry times per minute bucket, with a fixed TTL and with ±10% random jitter:

import random
random.seed(1)
KEYS, TTL = 200_000, 3600

def expiries(jitter):
    buckets = {}
    for _ in range(KEYS):
        ttl = TTL + (random.randint(-int(TTL * jitter), int(TTL * jitter)) if jitter else 0)
        minute = ttl // 60
        buckets[minute] = buckets.get(minute, 0) + 1
    return buckets

for jitter in (0, 0.10):
    b = expiries(jitter)
    print(f"jitter ±{jitter:.0%}: keys expire across {len(b)} minute(s); "
          f"worst minute = {max(b.values()):,} misses")

jitter ±0%: keys expire across 1 minute(s); worst minute = 200,000 misses
jitter ±10%: keys expire across 13 minute(s); worst minute = 16,994 misses

Root cause

Keys written at the same time with the same TTL expire at the same time, turning a steady trickle of misses into a burst — a self-inflicted "cache avalanche". The TTL itself was also chosen without thinking about how stale each field may be.

The fix

1. Add jitter to TTLs (±10–20%) so expiries spread out.
2. Choose TTLs per data type from a staleness budget: how wrong may this be, and for how long? Prices: 1–5 minutes (plus invalidation). Descriptions: hours. Reference data: a day.
3. Refresh ahead for hot keys (next lessons) instead of letting them expire under load.

def ttl(base_seconds, jitter=0.15):
    return int(base_seconds * random.uniform(1 - jitter, 1 + jitter))

r.set(f"product:{pid}:price", price, ex=ttl(120))
r.set(f"product:{pid}:description", desc, ex=ttl(6 * 3600))

// Per-cache TTLs from a staleness budget
Map<String, RedisCacheConfiguration> caches = Map.of(
    "prices", base.entryTtl(Duration.ofMinutes(2)),
    "descriptions", base.entryTtl(Duration.ofHours(6)));
// add jitter with a custom RedisCacheWriter/TtlFunction (Spring Data Redis 3.2+: RedisCacheWriter.TtlFunction)

Cache::put("product:$id:price", $price, now()->addSeconds(random_int(102, 138)));   // 120 s ± 15%

const ttl = (base: number, jitter = 0.15) => Math.round(base * (1 - jitter + Math.random() * 2 * jitter));
await redis.set(`product:${id}:price`, price, "EX", ttl(120));

Trade-offs

• Jitter makes exact expiry times unpredictable — fine for caches, not for business deadlines.
• Short TTLs mean more misses and more database load; long TTLs mean more staleness when invalidation fails. Split objects so volatile fields get short TTLs and stable ones long TTLs.
• Bulk warm-ups should also spread their writes, not just their TTLs.

Detect it in production

• Plot cache misses per minute: regular spikes at fixed times point to synchronized expiry.
• Redis INFO stats → expired_keys rate; database QPS aligned with those spikes.

Key points

• Same write time + same TTL = same expiry time = miss storm.
• Add ±10–20% jitter and spread bulk warm-ups.
• Choose TTLs per data type from an explicit staleness budget.