Call external APIs with retries and timeouts from FastAPI

Approach

Share one httpx.AsyncClient (created in lifespan) with explicit timeouts and connection limits. Retry idempotent calls on transient errors with exponential backoff + jitter using tenacity, honor Retry-After on 429, and translate failures into domain errors (e.g., 503 to your caller).

Solution

import httpx
from tenacity import AsyncRetrying, retry_if_exception, stop_after_attempt, wait_exponential_jitter

RETRYABLE_STATUS = {429, 502, 503, 504}


def is_transient(exc: BaseException) -> bool:
    if isinstance(exc, (httpx.ConnectError, httpx.ReadTimeout)):
        return True
    return isinstance(exc, httpx.HTTPStatusError) and exc.response.status_code in RETRYABLE_STATUS


async def get_exchange_rate(http: httpx.AsyncClient, base: str, quote: str) -> float:
    try:
        async for attempt in AsyncRetrying(
            retry=retry_if_exception(is_transient),
            wait=wait_exponential_jitter(initial=0.2, max=3),
            stop=stop_after_attempt(3),
            reraise=True,
        ):
            with attempt:
                resp = await http.get(f"/rates/{base}/{quote}")
                resp.raise_for_status()
                return resp.json()["rate"]
    except (httpx.HTTPError,) as exc:
        raise RatesUnavailable() from exc

# lifespan: app.state.http = httpx.AsyncClient(base_url=..., timeout=httpx.Timeout(5, connect=2),
#                                              limits=httpx.Limits(max_connections=50))

Pitfalls

• httpx default timeout (5s) is fine, but timeout=None hangs forever.
• Retrying non-idempotent POSTs.
• A new client per request (no connection pooling).

Variations

• Add a circuit breaker (e.g., purgatory or aiobreaker) for critical dependencies.
• Cache successful responses briefly to absorb spikes.

Further reading: HTTPX: Timeouts