Latency for a service suddenly doubled. Walk me through how you investigate.

Summary

First assess impact and mitigate (is the SLO burning? was there a recent change? roll back if likely). Then narrow down: which endpoints, regions, instances and customers are affected; when exactly it started; what changed then; and where the time goes (traces, dependency latency, saturation of CPU, memory, connection pools, database).

Longer Answer

A structured approach:

1. Scope: all traffic or a subset (one endpoint, one zone, one tenant, one version)?
2. Timeline: correlate the start with deploys, config and flag changes, traffic shifts, dependency incidents, cron jobs.
3. Breakdown: traces show whether time is in our code, the database or a downstream call.
4. Resources: USE method on the constrained resources; garbage collection pauses; thread or connection pool exhaustion (queueing).
5. Hypothesis and test: one change at a time, compare against a healthy baseline.

Communicate status throughout, and capture findings for the postmortem.

Key Points

• Impact and mitigation first.
• Scope, timeline, what changed.
• Traces and dependency breakdown.
• Saturation and queueing checks.

What interviewers look for

A methodical narrowing process that starts with impact and recent changes, not random guessing.