Retries and Dead-Letter Queues

Classify failures: transient (timeouts, 503, deadlocks) → retry with exponential backoff and jitter; permanent (validation errors, missing data, bugs) → don't retry endlessly. After max attempts, move the message to a dead-letter queue (DLQ) with error details, alert, and provide tooling to inspect, fix and redrive (replay) messages. DLQs are an operational queue, not a trash can.

DLQ mechanisms

{
  "deadLetterTargetArn": "arn:aws:sqs:eu-west-1:123456789012:orders-dlq",
  "maxReceiveCount": "5"
}

def handle(msg):
    try:
        process(msg)
    except (Timeout, ServiceUnavailable):
        raise                                    # transient → broker retries with backoff
    except (ValidationError, NotFound) as e:
        dead_letter(msg, reason=str(e))          # permanent → park immediately
        ack(msg)

Common pitfalls

• Infinite retries of permanent failures.
• DLQs without alerts or owners.
• Redriving before fixing the cause.

Key points

• Transient vs permanent
• Max attempts → DLQ
• Alert, inspect, redrive

Further reading: SQS: Dead-letter queues