WSGI, ASGI and the Event Loop

Scenario

Example app: a Python service was moved from Flask on Gunicorn (WSGI) to an async framework on Uvicorn (ASGI) "for performance". One endpoint calls a legacy partner API with a blocking HTTP client inside an async def handler.

What you see in production

• When the partner API is slow, every endpoint becomes slow, including /health, which does nothing.
• Kubernetes liveness probes time out and restart healthy pods.
• CPU is low; the process is mostly waiting.

The naive code

@app.get("/partner-prices")
async def partner_prices():
    response = requests.get(PARTNER_URL, timeout=10)   # blocking call inside async code
    return response.json()

Reproduce it

An event loop runs many requests on one thread by switching between them whenever one awaits. A blocking call never awaits, so nothing else runs until it finishes. The simulation models a loop with three requests.

def run_loop(requests_):
    # each request is a list of steps: ("await", seconds) yields to the loop, ("block", seconds) holds it
    clock, finished = 0.0, {}
    waiting = {name: (0.0, list(steps)) for name, steps in requests_.items()}  # name -> (ready_at, remaining steps)
    while waiting:
        name = min(waiting, key=lambda n: waiting[n][0])
        ready_at, steps = waiting.pop(name)
        clock = max(clock, ready_at)
        if not steps:
            finished[name] = clock
            continue
        kind, seconds = steps.pop(0)
        if kind == "block":
            clock += seconds                 # the loop thread is stuck; nobody else runs
            waiting[name] = (clock, steps)
        else:
            waiting[name] = (clock + seconds, steps)   # other requests run while this one waits
    return finished

health = [("await", 0.01)]
blocking_partner = [("block", 3.0)]
async_partner = [("await", 3.0)]

for label, partner in [("blocking client", blocking_partner), ("async client", async_partner)]:
    done = run_loop({"partner": partner, "health-1": health, "health-2": health})
    print(f"{label:<16} health finished at {max(done['health-1'], done['health-2']):.2f}s, partner at {done['partner']:.2f}s")

blocking client  health finished at 3.01s, partner at 3.00s
async client     health finished at 0.01s, partner at 3.00s

Root cause

ASGI servers and Node.js run requests concurrently on an event loop: a single thread that switches between tasks only at await points. A blocking call (synchronous HTTP client, time.sleep, heavy CPU work, a synchronous database driver) holds the thread, so every other request on that loop waits.

The fix

• Use async libraries inside async def handlers (httpx.AsyncClient, async database drivers) and await them.
• If you must call blocking code, run it in a thread pool (await asyncio.to_thread(...), Starlette's run_in_threadpool), or declare the handler as plain def: FastAPI runs def endpoints in a thread pool. See FastAPI: async and await.
• Move CPU-heavy work to worker processes or queues.
• In Node.js the same rule applies: never use synchronous APIs (fs.readFileSync, heavy loops) on the request path (the Node.js event loop).

The models compared:

Trade-offs

• Async code serves many slow I/O requests with little memory, but one mistake blocks everything on the loop.
• Sync (WSGI, PHP-FPM) workers are simpler and isolate slow requests, but need many processes or threads for high concurrency.

Detect it in production

• Measure event-loop lag (how late scheduled callbacks run); Node.js exposes it via perf_hooks.monitorEventLoopDelay, and asyncio's debug mode logs slow callbacks.
• If unrelated endpoints slow down together, suspect a blocked loop.

Key points

• Event loops switch between requests only at await; blocking calls stall everything.
• Use async libraries, or run blocking code in threads (def endpoints in FastAPI, asyncio.to_thread).
• WSGI and PHP-FPM isolate requests in workers; ASGI and Node.js multiplex them on a loop.