Resilience Under Load

Systems fail under overload by queueing until everything times out. Protect them with timeouts on every call, bounded queues and backpressure, load shedding (reject excess or low-priority requests early with 429/503), circuit breakers, bulkheads, and graceful degradation (disable non-essential features, serve cached data). Autoscale, but remember downstream systems may not scale as fast.

Admission control in a gateway/service
if inflight_requests > max_concurrency:            # e.g. 200 per instance
    if request.priority == "low":                  # recommendations, analytics
        return 503, {"Retry-After": "2"}           # shed early, cheaply
if dependency_breaker("recommendations").open:
    return cached_popular_items()                  # degrade gracefully

Common pitfalls

• Unbounded queues hiding overload until memory runs out.
• Retries amplifying overload.
• Autoscaling the app while the database is the bottleneck.

Key points

• Fail fast beats slow failure
• Shed low-priority load first
• Degrade non-essential features

Further reading: Google SRE: Cascading failures