Rate Limiting, Quotas & Abuse Protection · Where and What to Enforce
Fairness and Noisy Neighbors
Keep one tenant from degrading everyone else.
In multi-tenant systems, limits are about fairness: per-tenant rate and concurrency limits, separate queues or weighted fair queuing for background jobs, priority classes (interactive > batch), and isolation for the largest customers (dedicated workers or shards). Monitor per-tenant usage to spot noisy neighbors before they cause incidents.
Fairness techniques
| Technique | Protects against | Example |
|---|---|---|
| Per-tenant rate limits | Request floods | 5,000 req/min per tenant |
| Per-tenant concurrency | Slow, heavy requests | 10 concurrent exports |
| Fair queuing for jobs | One tenant filling the queue | Round-robin across tenant queues |
| Priority classes | Batch starving interactive | Interactive pool vs batch pool |
| Cell / shard isolation | Blast radius | Large tenants on dedicated cells |
Weighted fair queuing for background jobs (concept)
queues: tenant-A (weight 1), tenant-B (weight 1), tenant-C enterprise (weight 3)
scheduler: pick next job round-robin by weight → C, C, C, A, B, C, C, C, A, B …
per-tenant max in-flight: 20; global workers: 100
Common pitfalls
- A single FIFO job queue shared by all tenants.
- Limits only per API key (tenants create more keys).
- No per-tenant dashboards.
Key points
- Per-tenant limits and concurrency
- Fair queuing and priorities
- Cells/shards for blast radius
Further reading: AWS Builders' Library: Fairness in multi-tenant systems
Practice questions
Why is a single FIFO queue unfair in a multi-tenant system?
Answer
One tenant enqueuing a huge batch delays every other tenant's jobs behind it; per-tenant queues with fair scheduling bound that impact.
What is a 'cell' in cell-based architecture?
Answer
An isolated, independent copy of the service stack serving a subset of tenants, limiting the blast radius of failures and noisy neighbors.
Open in the app to mark lessons complete, take notes and track your progress.