Resilience Patterns · Resilient Data and Dependencies
Queue-Based Load Leveling
Absorb bursts by decoupling producers from consumers.
Put a durable queue between bursty producers and a constrained consumer (database, third-party API with rate limits). The API accepts work quickly (202 Accepted), consumers process at a sustainable rate, and backlogs drain after the peak. Bound the queue, give messages deadlines, autoscale consumers on queue age, and make processing idempotent.
Sync vs queue-based
| Criterion | Synchronous processing | Queue-based leveling |
|---|---|---|
| Peak handling | Fails or slows under bursts | Absorbs bursts |
| User feedback | Immediate result | 202 + status/notification |
| Downstream protection | None | Consumers control rate |
| Complexity | Low | Queues, workers, idempotency, monitoring |
Flash sale: 50,000 order requests in 2 minutes; payment provider allows 200 req/s
API → validate + reserve stock (fast) → enqueue 'charge' (202 Accepted, order pending)
Workers: 180 req/s to provider (under its limit), retries with backoff, idempotency keys
Backlog drains in ~5 minutes; users notified on completion
Common pitfalls
- Unbounded queues hiding a permanent capacity problem.
- No user-visible status for queued work.
- Processing work whose deadline passed.
Key points
- Durable queue absorbs bursts
- Consumers set the pace
- Status + idempotency + deadlines
Further reading: Azure Architecture: Queue-based load leveling
Practice questions
How does queue-based load leveling protect a rate-limited third-party API?
Answer
Consumers process messages at a controlled rate below the third party's limit, while the queue absorbs bursts from producers.
What's the user-experience trade-off?
Answer
Users get an accepted/pending status instead of an immediate final result, so you need status endpoints or notifications.
Open in the app to mark lessons complete, take notes and track your progress.