Design a webhook delivery system.

Summary

Record events durably, enqueue one delivery per subscribed endpoint, send signed HTTP requests from workers with timeouts, retry failures with exponential backoff over hours/days, and isolate slow endpoints so they don't delay others.

Requirements

• Customers register endpoints for event types
• At-least-once delivery with retries
• Signature verification, replay protection
• Visibility: delivery logs, manual redelivery

API

POST /api/webhook-endpoints {"url": "https://customer/hook", "events": ["order.paid"]}
# Outbound
POST https://customer/hook  Webhook-Id: evt_1  Webhook-Timestamp: ...  Webhook-Signature: v1,...

Data Model

endpoints(id, customer_id, url, secret, events text[], status)
events(id, type, payload, created_at)
deliveries(id, event_id, endpoint_id, status, attempts, next_attempt_at, last_status_code)

Architecture

Domain services publish events via an outbox. A dispatcher fans out deliveries per matching endpoint into a queue partitioned by endpoint. Workers send requests (HMAC signature with timestamp), record results, and schedule retries. Endpoints failing continuously are disabled and the customer notified.

Deep Dives

• Isolation: per-endpoint concurrency limits so one slow customer doesn't block others
• Security: HMAC signatures, timestamps to prevent replay, SSRF protection for URLs
• Idempotency: event IDs let receivers dedupe

Scaling

• Partition by endpoint
• Autoscale workers
• Archive old delivery logs

Comparison / Tradeoffs

What interviewers look for

A structured approach: clarify requirements and scale, sketch the API and data model, draw the high-level architecture, then go deep on the hardest component and discuss trade-offs, bottlenecks and failure modes.

Follow-up questions

• How do you protect against SSRF via customer-provided URLs?