API Architecture and Integration Design · Integration Patterns
Resilient Service Integrations
Timeouts, retries, circuit breakers, bulkheads and graceful degradation.
Every remote call can be slow or fail. Set timeouts on every call (from your latency budget), retry only idempotent operations with exponential backoff + jitter and a retry budget, use circuit breakers to fail fast when a dependency is unhealthy, isolate dependencies with bulkheads (separate pools/concurrency limits), and design fallbacks (cached data, defaults, feature degradation).
Latency budget for GET /product page (p99 target 800 ms)
- product-service timeout 200 ms, 1 retry (critical)
- pricing-service timeout 150 ms, 1 retry (critical; fallback: cached price < 5 min)
- reviews-service timeout 300 ms, no retry (optional; fallback: hide section)
- recommendations timeout 250 ms, breaker (optional; fallback: popular items)
Total worst case stays within budget; optional parts degrade instead of failing the page.
Common pitfalls
- Default infinite timeouts.
- Retries at every layer multiplying load (retry storms).
- Fallbacks that hide outages from monitoring.
Key points
- Timeouts from a latency budget
- Bounded retries with jitter
- Breakers, bulkheads, graceful degradation
Further reading: AWS Builders' Library: Timeouts, retries and backoff
Practice questions
What is a retry storm and how do you prevent it?
Answer
Retries at multiple layers multiply traffic to a struggling dependency, making the outage worse. Retry at one layer, use backoff with jitter, retry budgets and circuit breakers.
What is a bulkhead?
Answer
Isolating resources (thread pools, connection pools, concurrency limits) per dependency so one slow dependency can't exhaust capacity for everything else.
Open in the app to mark lessons complete, take notes and track your progress.