Topic hub
Messaging & Queues
Queues vs streams, delivery guarantees, idempotent consumers, outbox, sagas and background jobs.
Lessons
- Delivery Guarantees and Idempotent Consumers · Messaging, Events and Email Systems
At-most-once, at-least-once and 'effectively once' processing. - Transactional Outbox and Change Data Capture · Messaging, Events and Email Systems
Publish events reliably together with database changes. - Queues, Streams and Background Jobs · Messaging, Events and Email Systems
Choose between task queues, pub/sub and event streams. - Designing Events · Messaging, Events and Email Systems
Events vs commands, event shapes and schema evolution. - Kafka in Practice · Messaging, Events and Email Systems
Topics, partitions, consumer groups, keys and retention. - Operating Queues and Streams · Messaging, Events and Email Systems
Dead-letter queues, backpressure, monitoring and replay. - Sagas and Long-Running Workflows · Messaging, Events and Email Systems
Coordinate multi-service business processes without distributed transactions. - Server-Sent Events · Real-Time APIs
Simple, HTTP-native server push with automatic reconnection. - WebSockets in Depth · Real-Time APIs
Handshake, frames, heartbeats, close codes and message protocols. - What Event Sourcing Is · Event Sourcing & CQRS
Store every change as an immutable event and derive state from them. - CQRS Explained · Event Sourcing & CQRS
Separate the write model from read models — with or without event sourcing. - Idempotent Consumers and the Inbox Pattern · Idempotency & Retry-Safe Systems
Process each message's effect once despite redelivery. - Payment Webhooks Done Right · Payments & Money APIs
Verify, dedupe, handle out-of-order events and re-fetch. - Choosing a Message Broker · Queues & Background Processing in Depth
Kafka, RabbitMQ, SQS/SNS, Google Pub/Sub, Redis Streams and NATS. - Queues, Pub/Sub and Streams · Queues & Background Processing in Depth
Three messaging models and what each is for. - Retries and Dead-Letter Queues · Queues & Background Processing in Depth
Retry transient failures, park permanent ones. - Durable Workflows with Temporal · Queues & Background Processing in Depth
Long-running, reliable business processes as code. - Designing Webhook Delivery (Sender Side) · Real-Time APIs
Event types, payloads, subscriptions and delivery pipeline.
Decision guides
- At-Most-Once vs At-Least-Once vs Effectively-Once
- Choreography vs Orchestration
- Event Sourcing vs CRUD + Audit Log vs CDC
- Kafka vs RabbitMQ vs SQS vs Redis Streams
- Streaming vs Synchronous vs Batch LLM Calls
- Temporal vs Step Functions vs Queue + State Machine
Cookbook recipes
- Run background jobs from FastAPI with ARQ · FastAPI
- Run scheduled tasks alongside FastAPI · FastAPI
- Process background jobs with BullMQ · Node.js
- Send emails asynchronously with a queue in PHP · PHP
- Receive and verify webhooks in PHP · PHP
- Run scheduled jobs (cron) reliably in PHP · PHP
- Run scheduled jobs once across replicas with ShedLock · Spring Boot
- Poll a long-running job's status · React
- Send emails after a transaction commits in Spring · Spring Boot
- Publish events reliably with the transactional outbox · Spring Boot
- Consume Kafka messages with retries and a dead-letter topic · Spring Boot
- Stream progress updates with Server-Sent Events · Spring Boot