Queues & Background Processing in Depth · Scaling Consumers
Lag, Queue Age and Backpressure
Measure how far behind consumers are and react.
The key health metrics are queue depth / consumer lag and, more importantly, age of the oldest message — it directly measures user-visible delay. Alert on age against the business SLO (e.g., emails sent within 5 minutes). When consumers fall behind: scale consumers, throttle producers, shed low-priority work, or extend retention to avoid data loss. Kafka lag must stay well within retention or data is lost before processing.
Queue health metrics
| Metric | What it tells you |
|---|---|
| Oldest message age | User-visible delay (alert on this) |
| Queue depth / consumer lag | Backlog size |
| Ingress vs egress rate | Whether backlog grows or shrinks |
| Processing errors / DLQ rate | Health of handlers |
| Lag vs retention (Kafka) | Risk of losing unprocessed data |
# Prometheus alert: emails delayed beyond SLO
- alert: EmailQueueDelayed
expr: aws_sqs_approximate_age_of_oldest_message_average{queue_name="emails"} > 300
for: 5m
labels: { severity: page }
Common pitfalls
- Alerting on depth alone (depth can be fine if it drains fast).
- Kafka lag approaching retention unnoticed.
- Unbounded producer rate during incidents.
Key points
- Alert on oldest-message age
- Compare ingress vs egress
- Keep lag within retention
Further reading: SQS: CloudWatch metrics
Practice questions
Why alert on age of oldest message instead of queue depth?
Answer
Age reflects actual delay experienced by the work; a deep queue draining quickly may be fine, while a small queue with stuck messages is not.
What happens if Kafka consumer lag exceeds topic retention?
Answer
Messages are deleted before the consumer reads them, causing silent data loss.
Open in the app to mark lessons complete, take notes and track your progress.