EventBridge: Patterns, Retries and Archives

Scenario

Example app: services publish domain events (OrderPlaced, OrderShipped) to a custom EventBridge bus. A rule routes high-value orders to a Lambda function that notifies the fraud team.

What you see in production

The fraud team notices that some high-value orders never triggered a notification. There are no errors in the function's logs, because the function was never invoked for those events.

The naive code

{
  "source": ["orders"],
  "detail-type": ["orderPlaced"],
  "detail": { "total": [{ "numeric": [">", 1000] }] }
}

The publisher sends "detail-type": "OrderPlaced", and some services send total as a string ("1250.00"). The rule's target has no dead-letter queue.

Reproduce it

A small matcher with EventBridge's semantics: values are case-sensitive, arrays mean "any of", and numeric matching only applies to JSON numbers:

function matches(pattern, event) {
  return Object.entries(pattern).every(([key, expected]) => {
    const value = event[key];
    if (Array.isArray(expected)) {
      return expected.some((e) => {
        if (e && typeof e === "object" && "numeric" in e) {
          const [op, n] = e.numeric;
          return typeof value === "number" && (op === ">" ? value > n : value < n);
        }
        return e === value;
      });
    }
    return value && typeof value === "object" && matches(expected, value);
  });
}

const naiveRule = { source: ["orders"], "detail-type": ["orderPlaced"], detail: { total: [{ numeric: [">", 1000] }] } };
const fixedRule = { source: ["orders"], "detail-type": ["OrderPlaced"], detail: { total: [{ numeric: [">", 1000] }] } };
const events = [
  { source: "orders", "detail-type": "OrderPlaced", detail: { id: "o1", total: 1250 } },
  { source: "orders", "detail-type": "OrderPlaced", detail: { id: "o2", total: "1250.00" } },
  { source: "orders", "detail-type": "OrderPlaced", detail: { id: "o3", total: 90 } },
];
for (const e of events) {
  console.log(e.detail.id, "naive:", matches(naiveRule, e), "| fixed:", matches(fixedRule, e));
}

o1 naive: false | fixed: true
o2 naive: false | fixed: false
o3 naive: false | fixed: false

Root cause

EventBridge matching is exact and type-sensitive: detail-type values are case-sensitive strings, and numeric filters only match JSON numbers. A non-matching event is not an error; it simply goes nowhere. When a target does fail (throttling, function errors), EventBridge retries with backoff for up to 24 hours by default, and then drops the event unless the target has a dead-letter queue.

The fix

• Treat events as a contract: publish a schema (the EventBridge schema registry, or JSON Schema in a shared package), validate events before publishing, and use consistent naming and types.
• Test rules against sample events in CI with the TestEventPattern API (or the console's sandbox).
• Configure every target with a DLQ and a deliberate retry policy (maximum event age and retry attempts), and alarm on the DLQ.
• Enable an archive on the bus so events can be replayed after a bug fix.

# AWS SAM
FraudNotifier:
  Type: AWS::Serverless::Function
  Properties:
    Events:
      HighValue:
        Type: EventBridgeRule
        Properties:
          EventBusName: !Ref OrdersBus
          Pattern: { source: [orders], detail-type: [OrderPlaced], detail: { total: [{ numeric: [">", 1000] }] } }
          DeadLetterConfig: { Arn: !GetAtt FraudDlq.Arn }
          RetryPolicy: { MaximumEventAgeInSeconds: 3600, MaximumRetryAttempts: 10 }

Trade-offs

• Strict schemas slow down change; version events (OrderPlaced.v2 or a version field) instead of changing meaning in place.
• Archives cost storage; set retention to what you need for recovery.
• EventBridge adds some latency compared with direct invocation; it buys decoupling and fan-out.

Detect it in production

• Per-rule metrics: MatchedEvents, Invocations, FailedInvocations, InvocationsSentToDlq.
• Compare published counts (from the publisher) with MatchedEvents for important rules.
• Alarms on DLQ depth and on rules that suddenly match zero events.

Key points

• Pattern matching is exact and type-sensitive; non-matches are silent.
• Validate events against a schema and test patterns in CI.
• DLQs and retry policies on every target; archive and replay for recovery.

Further reading: Amazon EventBridge event patterns · EventBridge retry policy and DLQs