Idempotency & Retry-Safe Systems · Idempotency Keys in Practice

Expiry, Partial Failures and Recovery Points

Clean up keys and resume operations that failed halfway.

Expire keys after the documented window (Stripe uses 24 hours) with a cleanup job or TTL. The hardest case: an operation with several steps (create order → charge card → send email) crashed halfway. Model recovery points (atomic phases) and store which phase completed; a retry with the same key resumes from the last recovery point instead of starting over. Each external call uses its own derived idempotency key ({key}:charge) so repeating a phase is safe.

Recovery-point design

Phase type How to make it repeatable
Local DB transaction Atomic; record recovery point in same tx
External API call Derived idempotency key per call
Notification Outbox with dedupe id
Final response Stored with the key for replay
key = 01J9ZQ…
phase 1 (DB tx): create order (status=pending), record recovery_point='order_created'
phase 2: charge card with provider Idempotency-Key '01J9ZQ…:charge'
         → record recovery_point='charged', payment_id
phase 3 (DB tx): mark order paid, enqueue receipt email (outbox), recovery_point='finished', store response
Retry after crash in phase 2 → reads recovery_point='order_created' → repeats phase 2 with the same derived key
(provider dedupes the charge) → continues to phase 3.

Common pitfalls

  • Keys expiring before clients stop retrying.
  • Re-running all steps on retry (double side effects).
  • No alerting on keys stuck in 'processing'.

Key points

  • Documented expiry window
  • Recovery points for multi-step operations
  • Derived keys for downstream calls

Further reading: Brandur: idempotency keys and recovery points

Practice questions

Why derive a separate idempotency key for each external call?

Answer

So each external side effect is deduplicated by the provider individually; repeating a phase after a crash doesn't duplicate the charge or shipment.

What should happen to keys stuck in 'processing' long after their lease?

Answer

A sweeper should resume them from their recovery point (or mark them failed and alert), so they don't block retries indefinitely.

Open in the app to mark lessons complete, take notes and track your progress.