Queues & Background Processing in Depth · Messaging Foundations
Ordering Guarantees
What ordering brokers give you and how to design around it.
Global ordering doesn't scale; brokers order within a partition/queue/message group. Kafka orders per partition (same key → same partition), SQS FIFO orders per message group ID, RabbitMQ orders per queue with a single consumer. Parallel consumers, retries and redeliveries can still reorder processing. Design for it: key messages by entity, make handlers tolerate out-of-order events (version numbers, "last writer wins by version"), or carry full state instead of deltas.
Ordering by broker
| Broker | Ordering scope | Breaks when |
|---|---|---|
| Kafka | Per partition (key) | Changing partition count; retries to other topics |
| SQS standard | None (best effort) | Always possible |
| SQS FIFO | Per message group | Group id too coarse limits throughput |
| RabbitMQ | Per queue | Multiple consumers / requeues |
| Google Pub/Sub | Per ordering key (opt-in) | Without ordering keys |
async def apply_customer_update(event: dict):
# Ignore stale events delivered out of order
updated = await db.execute(
'''UPDATE customers SET name = $2, email = $3, version = $4
WHERE id = $1 AND version < $4''',
event["customer_id"], event["name"], event["email"], event["version"])
if updated == "UPDATE 0":
metrics.increment("events.stale_ignored")
// Same orderId → same partition → ordered for that order
kafkaTemplate.send("order-events", order.getId(), OrderEvent.paid(order));
Common pitfalls
- Assuming global order across partitions.
- Increasing partition count (keys move partitions).
- Delta events applied out of order corrupt state.
Key points
- Order is per partition/group
- Key by entity
- Make consumers order-tolerant
Further reading: Kafka: Concepts
Practice questions
How do you keep all events for one order in sequence in Kafka?
Answer
Use the order ID as the message key so all its events land in the same partition, and process each partition sequentially.
How can a consumer cope with out-of-order events?
Answer
Include a version or sequence number and ignore events older than the stored version, or carry full state so the latest version wins.
Open in the app to mark lessons complete, take notes and track your progress.