Logging & Observability interview question · Most asked
Walk through how you would investigate a sudden latency spike in production.
Summary
Confirm the scope on dashboards (which endpoints, regions, customers), correlate with recent changes (deploys, flags, traffic), use traces to find the slow span, check dependency and resource saturation, then mitigate first (rollback, scale, disable feature) and find root cause after.
Longer Answer
A practical sequence: (1) check SLO/RED dashboards and when it started; (2) look for deploy or config markers; (3) open slow traces (exemplars from the latency histogram) to see whether time is spent in the DB, an external API, lock contention or the app itself; (4) check saturation: DB connections, thread pools, CPU throttling, GC pauses, queue backlog; (5) mitigate with the fastest safe action; (6) write a blameless postmortem with follow-ups.
Key Points
- Scope and timing first
- Correlate with changes
- Traces to find the slow span
- Check saturation of pools/CPU/GC
- Mitigate before root-causing
More most-asked Logging & Observability questions
- What are the three pillars of observability, and how do they differ?
- What is structured logging and why is it preferred over plain text logs?
- What is a correlation (request) ID and how do you propagate it?
- What is distributed tracing and how does OpenTelemetry fit in?
- What are the RED, USE and Four Golden Signals methods?
- What are SLIs, SLOs and error budgets?
- What should never be logged, and how do you prevent it?