The Outbox Pattern and Dual Writes — Idempotency & Retry-Safe Systems

Writing to your database and publishing to a broker are two independent operations; a crash between them loses or invents events (the dual-write problem). The transactional outbox writes the event to an outbox table in the same transaction as the business change; a relay (polling with FOR UPDATE SKIP LOCKED, or CDC like Debezium) publishes it afterwards. Relays may publish twice, so events carry stable IDs and consumers stay idempotent.

Publishing strategies

Criterion	Publish in request	Outbox + poller	Outbox + CDC (Debezium)
Atomic with DB write	No	Yes	Yes
Latency	Lowest	Poll interval	Low (log tailing)
Infrastructure	None	Small worker	Kafka Connect / Debezium
Duplicates	Possible + loss	Possible (no loss)	Possible (no loss)

BEGIN;
  UPDATE orders SET status = 'paid' WHERE id = 'ord_1';
  INSERT INTO outbox (id, aggregate_id, type, payload) VALUES (gen_random_uuid(), 'ord_1', 'order.paid', '{"order_id":"ord_1"}');
COMMIT;
-- relay: SELECT ... FROM outbox WHERE published_at IS NULL ORDER BY created_at LIMIT 100 FOR UPDATE SKIP LOCKED

Common pitfalls

Publishing to Kafka inside the DB transaction (blocks on broker outages).
Outbox rows never cleaned up.
Consumers assuming no duplicates.

Key points

Outbox solves dual writes
Polling or CDC relay
Stable event IDs + idempotent consumers

Further reading: microservices.io: Transactional outbox

Practice questions

Why not publish the event right after the DB commit in the same request?

Answer

The process can crash after commit but before publishing (event lost), or the broker can be down; the outbox makes publishing a durable, retryable step.

Can the outbox relay publish an event twice?

Answer

Yes — if it crashes after publishing but before marking the row published. That's why events need IDs and consumers must be idempotent.