Messaging, Events and Email Systems · Asynchronous Workflows

Transactional Outbox and Change Data Capture

Publish events reliably together with database changes.

Writing to the database and publishing to a broker are two systems — a crash between them loses or invents events (the dual-write problem). The transactional outbox writes the event into an outbox table in the same transaction; a relay publishes it afterwards (polling with FOR UPDATE SKIP LOCKED, or CDC with Debezium reading the database log). Consumers stay idempotent because relays may publish twice.

BEGIN;
  INSERT INTO orders (id, status, total) VALUES ('ord_1', 'paid', 2400);
  INSERT INTO outbox (id, aggregate_id, type, payload)
    VALUES (gen_random_uuid(), 'ord_1', 'order.paid', '{"order_id":"ord_1","total":2400}');
COMMIT;

-- relay (poller)
SELECT * FROM outbox WHERE published_at IS NULL ORDER BY created_at LIMIT 100 FOR UPDATE SKIP LOCKED;
-- publish each, then UPDATE outbox SET published_at = now() WHERE id = ...

Common pitfalls

  • Publishing inside the DB transaction (broker down → transaction blocked).
  • Outbox tables growing unbounded.
  • Assuming global ordering across aggregates.

Key points

  • Outbox solves dual writes
  • Polling relay or CDC
  • Consumers stay idempotent

Further reading: microservices.io: Transactional outbox

Practice questions

What is the dual-write problem?

Answer

Updating a database and publishing a message are separate operations; a failure between them leaves the two systems inconsistent (event lost or published for a rolled-back change).

What does CDC add compared to polling the outbox?

Answer

Lower latency and less database polling load, by streaming committed changes from the database log; it adds infrastructure (Debezium, Kafka Connect).

Open in the app to mark lessons complete, take notes and track your progress.