Messaging & Queues interview question · Most asked
Compare Kafka and RabbitMQ (and managed queues like SQS).
Summary
Kafka is a distributed, partitioned, replicated log: messages are retained and consumers track offsets, enabling replay, high throughput and ordered processing per partition. RabbitMQ is a message broker with flexible routing (exchanges, queues), per-message acks and work-queue semantics. SQS is a fully managed queue (standard or FIFO) with minimal operations.
Longer Answer
Use Kafka for event streaming, event sourcing, analytics pipelines and many consumer groups replaying data; RabbitMQ for task queues and complex routing; SQS/SNS or Google Pub/Sub for simple managed messaging on cloud. Redis Streams and NATS JetStream are lighter alternatives.
Key Points
- Kafka: retained log, replay, partitions
- RabbitMQ: routing, acks, task queues
- SQS: managed simplicity
- Choose by retention and routing needs
Comparison / Tradeoffs
| Kafka | RabbitMQ | SQS | |
|---|---|---|---|
| Model | Log + offsets | Broker queues | Managed queue |
| Replay | Yes | No (once consumed) | No |
| Ordering | Per partition | Per queue | FIFO queues |
| Ops | Heavier | Moderate | None |