Capstone: A Black-Friday Order Pipeline

• POST /orders (HTTP API + Lambda) validates input, requires an Idempotency-Key, and in one DynamoDB transaction writes the order (status = PENDING_PAYMENT) and conditionally decrements stock (stock >= qty). Returns 202 Accepted with the order ID in well under a second.
• A DynamoDB Stream or an explicit OrderPlaced event (outbox) puts the order on an SQS queue for payments.
• A payment worker (SQS event source, partial batch responses, DLQ) charges the card with the provider's idempotency key = order ID, then updates the order to PAID with a condition on PENDING_PAYMENT.
• OrderPaid events on EventBridge fan out to email (its own queue and worker) and fulfilment.
• A Step Functions standard workflow (or a scheduled sweeper) handles timeouts: orders stuck in PENDING_PAYMENT too long are cancelled, stock is released and the customer is notified.

Look for: asynchronous acceptance (202) with durable storage first, queues between the API and slow downstreams, events for fan-out, and an explicit path for stuck orders. Misses: charging the card inside the API request, synchronous chains, no handling of orders that never complete.

• Overselling: the stock decrement is a conditional update (stock >= :qty) inside the same transaction as the order insert; either both happen or neither. Cancellations increment stock with a condition on the order's status to avoid double releases.
• Double charges: Powertools Idempotency on the API (Idempotency-Key header) so client retries return the original order; the payment call uses the order ID as the provider's idempotency key, so SQS redeliveries and worker retries cannot charge twice; the status update to PAID is conditional.
• Crashes between charge and status update are safe: the redelivered message calls the provider again with the same key, which returns the original charge.

Strong answers use database-enforced atomicity and idempotency keys at each boundary (client->API, worker->provider). Weak answers rely on 'checking first' (read-then-write) or on SQS FIFO alone for exactly-once.

• Set the payment queue's event source maximum concurrency so that concurrency x per-invocation rate stays below 300 requests per second (for example 60 concurrent workers each making about 4 calls per second with batch size 1-5), and handle 429s with backoff by returning failed items.
• The queue absorbs the burst; the backlog drains at the provider's pace. Communicate expected confirmation delays.
• Separate queues and concurrency caps for email so a slow email service never delays payments.

Look for concrete numbers derived from the 300 rps limit and the use of queue-level concurrency caps rather than hoping retries will smooth things out.

• Request the account concurrency increase in advance; give the API function reserved concurrency (for example 1,000) and enough provisioned concurrency for the first minutes to avoid cold-start spikes.
• Cap the workers (payments 60, email 50) with event source maximum concurrency.
• DynamoDB on-demand (or provisioned with pre-warming) with keys that avoid hot partitions; stock items for popular products may need write sharding or a reservation approach.
• Cost: dominated by Lambda duration and DynamoDB writes; acceptable for one day, and provisioned concurrency is scheduled only for the event window.

Good answers allocate concurrency per function deliberately, think about DynamoDB hot keys for popular items, and time-box expensive settings.

• Load test at 1.5x forecast in a production-like account with the provider's sandbox, including failure injection (provider 429s and timeouts).
• Dashboards: accepted orders per second, queue depth and age per queue, payment success and failure rates, orders by status, DLQ depth, throttles, API p99.
• Alarms: DLQ not empty, oldest message age above SLO, payment failure rate, orders stuck in PENDING_PAYMENT.
• Structured logs with order ID and correlation ID across all functions so any order can be traced end to end.

Look for queue-age and stuck-state signals (not only error counts), and a load test that includes downstream failures.