Database Caching at Scale · Operating Caches in Production

Redis Data Structures and Atomic Operations

Use INCR, hashes, sorted sets and Lua scripts instead of racy GET-then-SET sequences.

Scenario

Example app: a game backend keeps per-player daily limits ("max 5 free spins per day") and a leaderboard in Redis. The limit check reads the counter, compares it with 5, then writes the incremented value.

What you see in production

Players with fast fingers (or scripts) get 7–9 free spins a day. The leaderboard, stored as one JSON blob per game, is rewritten on every score update, and concurrent updates occasionally drop scores.

The naive code

def try_spin(player):
    used = int(r.get(f"spins:{player}") or 0)      # read
    if used >= 5:
        return False
    r.set(f"spins:{player}", used + 1, ex=86400)   # write: concurrent requests both pass the check
    return True

def add_score(game, player, score):
    board = json.loads(r.get(f"board:{game}") or "{}")   # read-modify-write of a whole JSON blob
    board[player] = max(board.get(player, 0), score)
    r.set(f"board:{game}", json.dumps(board))

Reproduce it

Two requests interleave a GET-then-SET counter; then the same with an atomic increment-and-check (what a Lua script does inside Redis):

Racy GET/SET vs an atomic scriptTry it yourself
def racy(store):
    def spin():
        used = store.get("spins", 0)          # GET
        yield
        if used < 5:
            store["spins"] = used + 1          # SET
            yield "allowed"
        else:
            yield "denied"
    return spin

def atomic(store):
    def spin():
        yield                                  # the whole script runs as one step inside Redis
        if store.get("spins", 0) < 5:
            store["spins"] = store.get("spins", 0) + 1
            yield "allowed"
        else:
            yield "denied"
    return spin

for name, factory in (("GET then SET", racy), ("atomic script", atomic)):
    store = {"spins": 4}                       # one free spin left
    spin = factory(store)
    a, b = spin(), spin()
    next(a); next(b)                           # both start
    results = [next(a), next(b)]
    print(f"{name:13} -> {results}, spins used = {store['spins']}")

Output

GET then SET  -> ['allowed', 'allowed'], spins used = 5
atomic script -> ['allowed', 'denied'], spins used = 5

Root cause

Redis executes each command atomically, but a sequence of commands from a client is not atomic: other clients' commands interleave between your GET and SET. Storing whole aggregates as one blob also turns every small update into a read-modify-write of the entire value.

The fix

Use the data structure that makes the update a single command, or run the logic server-side in a script:

-- KEYS[1] = spins:{player}, ARGV[1] = limit, ARGV[2] = ttl
local used = tonumber(redis.call('GET', KEYS[1]) or '0')
if used >= tonumber(ARGV[1]) then return 0 end
redis.call('INCR', KEYS[1])
if used == 0 then redis.call('EXPIRE', KEYS[1], ARGV[2]) end
return 1

Trade-offs

  • Lua scripts block Redis while they run — keep them short and O(1)/O(log n).
  • In Redis Cluster, all keys a script touches must be in the same hash slot (use hash tags like {player42}:spins).
  • MULTI/EXEC transactions batch commands atomically but can't branch on values read inside them; use WATCH for optimistic checks or a script.

Detect it in production

  • Business invariants: count spins per player per day in the database/event log and compare with the limit.
  • Redis SLOWLOG for long scripts; latency monitoring.

Key points

  • Single Redis commands are atomic; sequences aren't.
  • Prefer native structures (INCR, hashes, sorted sets) and server-side scripts for check-and-set logic.
  • Keep scripts short; respect hash slots in clusters.

Practice questions

Why does a GET-then-SET limit check allow more than the limit under concurrency?

Answer

Other clients' commands interleave between the GET and the SET; two requests can both read 4 and both write 5, so both pass the check.

How do you store a leaderboard in Redis without read-modify-write races?

Answer

Use a sorted set: ZADD (optionally with GT to only raise scores) updates one member atomically, and ZREVRANGE returns the top N.

What constraint applies to Lua scripts in Redis Cluster?

Answer

All keys accessed by the script must hash to the same slot; use hash tags ({...}) to co-locate them.

Open in the app to mark lessons complete, take notes and track your progress.