Idempotency & Retry-Safe Systems · Retries Done Right
Timeouts and Deadline Propagation
Set timeouts from latency budgets and pass deadlines downstream.
Every remote call needs a timeout; without one, slow dependencies exhaust threads and connections. Derive timeouts from the caller's latency budget, not defaults: if the API must answer in 800 ms, its downstream calls can't each wait 30 s. Propagate a deadline (absolute time) so downstream services stop work the caller has already given up on — gRPC does this natively; for HTTP pass a header (e.g., X-Request-Deadline or grpc-timeout-style) and check it before expensive work.
Kinds of timeouts
| Timeout | Purpose | Typical value |
|---|---|---|
| Connect | Fail fast on unreachable hosts | 100 ms – 2 s |
| Per-attempt / read | Bound one attempt | p99.9 of dependency + margin |
| Overall deadline | Bound total time incl. retries | Caller's budget |
| Server-side request | Stop runaway work | Slightly above client deadline |
const deadline = Date.now() + 800; // request budget
async function callInventory(sku: string) {
const remaining = deadline - Date.now() - 50; // keep 50 ms for our own work
if (remaining <= 0) throw new DeadlineExceeded();
return fetch(`http://inventory/skus/${sku}`, {
signal: AbortSignal.timeout(Math.min(remaining, 300)),
headers: { "x-request-deadline": String(deadline) },
});
}
import asyncio, time
async def call_inventory(client, sku: str, deadline: float):
remaining = deadline - time.monotonic() - 0.05
if remaining <= 0:
raise DeadlineExceeded()
async with asyncio.timeout(min(remaining, 0.3)):
return await client.get(f"http://inventory/skus/{sku}")
ctx, cancel := context.WithTimeout(r.Context(), 800*time.Millisecond) // request budget
defer cancel()
callCtx, cancelCall := context.WithTimeout(ctx, 300*time.Millisecond) // per-call cap, bounded by parent
defer cancelCall()
req, _ := http.NewRequestWithContext(callCtx, http.MethodGet, "http://inventory/skus/"+sku, nil)
resp, err := http.DefaultClient.Do(req) // cancelled when either deadline passes
Common pitfalls
- Library defaults of 'no timeout' or 30–60 s.
- Retries that ignore the overall deadline.
- Downstream services continuing work for callers that gave up.
Key points
- Timeouts from latency budgets
- Overall deadline caps retries
- Propagate deadlines
Further reading: gRPC: Deadlines
Practice questions
How do you choose a per-attempt timeout for a dependency?
Answer
From its observed latency distribution (e.g., slightly above p99.9) and within the caller's remaining budget, leaving room for retries if allowed.
Why propagate deadlines to downstream services?
Answer
So they can stop work the original caller has abandoned, saving resources during overload instead of finishing useless requests.
Open in the app to mark lessons complete, take notes and track your progress.