Node.js Serverless on AWS Lambda · Event-Driven Lambda
Orchestration with Step Functions
Why chaining Lambdas that call each other ends in timeouts, double charges and lost state, and how Step Functions fixes it.
Scenario
Example app: order fulfilment: reserve stock, charge the card, create the shipment, email the customer. The first version has one function per step, each synchronously invoking the next.
What you see in production
When the shipping provider is slow, the whole chain times out. Customers are charged but no shipment exists. Nobody can tell which step failed for a given order without searching four log groups. Each waiting function is billed while it waits for the next.
The naive code
// reserve-stock function
export const handler = async (order) => {
await stock.reserve(order);
return lambda.invoke({ FunctionName: "charge-card", Payload: JSON.stringify(order) }); // waits for the whole chain
};
Reproduce it
The simulation runs the steps with a failure in shipping, first chained, then as a workflow with retries and compensation (a saga):
let shippingAttempts = 0;
const steps = {
reserve: async (s) => { s.log.push("stock reserved"); },
charge: async (s) => { s.log.push("card charged"); },
ship: async (s) => { shippingAttempts++; if (shippingAttempts < 4) throw new Error("shipping API timeout"); s.log.push("shipment created"); },
email: async (s) => { s.log.push("email sent"); },
};
const compensations = { charge: async (s) => s.log.push("refund issued"), reserve: async (s) => s.log.push("stock released") };
async function chained() {
const s = { log: [] };
try { for (const step of ["reserve", "charge", "ship", "email"]) await steps[step](s); }
catch (e) { s.log.push(`FAILED: ${e.message} (no record of which steps ran)`); }
return s.log;
}
async function workflow(maxAttempts) {
const s = { log: [] }, done = [];
for (const step of ["reserve", "charge", "ship", "email"]) {
let attempt = 0;
while (true) {
try { await steps[step](s); done.push(step); break; }
catch (e) {
if (++attempt >= maxAttempts) {
s.log.push(`${step} failed after ${attempt} attempts, compensating`);
for (const d of done.reverse()) if (compensations[d]) await compensations[d](s);
return s.log;
}
}
}
}
return s.log;
}
shippingAttempts = 0; console.log("chained: ", (await chained()).join(" -> "));
shippingAttempts = 0; console.log("workflow, 2 tries:", (await workflow(2)).join(" -> "));
shippingAttempts = 0; console.log("workflow, 5 tries:", (await workflow(5)).join(" -> "));
Output
chained: stock reserved -> card charged -> FAILED: shipping API timeout (no record of which steps ran) workflow, 2 tries: stock reserved -> card charged -> ship failed after 2 attempts, compensating -> refund issued -> stock released workflow, 5 tries: stock reserved -> card charged -> shipment created -> email sent
Root cause
Synchronous chains couple every step's availability and latency together, accumulate timeouts, pay for idle waiting, and keep the process state only in memory. When a middle step fails, nothing knows which earlier steps succeeded, so nothing undoes them: the classic partial-failure problem of distributed transactions.
The fix
Model the process as a Step Functions state machine: each step is a task with its own retry policy (exponential backoff for transient errors) and catch that routes to compensating steps (refund, release stock), implementing the saga pattern. Execution history shows exactly where each order is. Use Standard workflows for long-running, exactly-once business processes (up to a year, auditable history) and Express workflows for high-volume, short event processing.
"Ship": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": { "FunctionName": "${ShipFunction}", "Payload.$": "$" },
"Retry": [{ "ErrorEquals": ["ShippingTimeout"], "IntervalSeconds": 2, "BackoffRate": 2, "MaxAttempts": 4 }],
"Catch": [{ "ErrorEquals": ["States.ALL"], "Next": "RefundPayment" }],
"Next": "SendEmail"
}
Steps that only call AWS services (DynamoDB, SQS, SNS) can use direct service integrations without any Lambda code.
Trade-offs
- Step Functions adds per-transition cost (Standard) and some latency; for two trivial steps, a single function may be simpler.
- Compensations must themselves be idempotent and may fail; design for manual intervention paths.
- Workflow definitions are code: test them (Step Functions Local or test state API) and version them.
Detect it in production
- Failed and timed-out executions per state machine, with alarms.
- Executions stuck in a step for longer than expected.
- Orders with a charge but no shipment (reconciliation query) indicate missing compensation.
Key points
- Do not chain Lambdas synchronously for business processes.
- Step Functions: per-step retries, catches, compensations (sagas) and visible state.
- Standard for long or exactly-once processes; Express for high-volume short ones.
Further reading: What is Step Functions? · Choosing Standard or Express workflows
Practice questions
What problems come from Lambda functions synchronously invoking each other in a chain?
Answer
Timeouts accumulate and failures cascade, every waiting function is billed while idle, and when a middle step fails there is no record of which earlier steps succeeded, so they are not undone.
How does a saga handle a failure in the middle of a multi-step process?
Answer
Each completed step has a compensating action (refund, release stock); when a later step fails permanently, the workflow runs the compensations for the completed steps in reverse order. Step Functions catch blocks route failures to those compensations.
When would you choose an Express workflow over a Standard workflow?
Answer
For high-volume, short-lived event processing (up to five minutes) where at-least-once execution is acceptable and lower cost per execution matters. Standard suits long-running, auditable business processes needing exactly-once execution.
Open in the app to mark lessons complete, take notes and track your progress.