Async Handlers and Unfinished Work

Scenario

Example app: an order-confirmation function receives a batch of orders, writes an audit record for each to a database and returns { ok: true }. It runs a few hundred times a minute behind an API.

What you see in production

Auditors find missing audit records for roughly 1 in 20 orders. The function never reports an error. Some audit writes appear with timestamps minutes after the order, attached to a completely different request in the logs. Locally, with node and a test script, every record is written.

The naive code

The handler starts the writes but does not wait for them:

export const handler = async (event) => {
  event.orders.forEach(async (order) => {
    await db.put({ table: "audit", item: { orderId: order.id, at: Date.now() } });
  });
  return { ok: true };          // returns while the writes are still pending
};

Reproduce it

This simulation models Lambda's behaviour: when the handler's promise resolves, the environment is frozen. Pending work either never runs or resumes during a later invocation:

const written = [];
const pending = [];
const db = { put: (item) => new Promise((resolve) => pending.push(() => { written.push(item); resolve(); })) };

async function naiveHandler(event) {
  event.orders.forEach(async (order) => { await db.put({ orderId: order.id }); });
  return { ok: true };
}

async function invoke(handler, event) {
  const result = await handler(event);
  const unfinished = pending.length;   // Lambda freezes here: pending work stops
  return { result, unfinished };
}

const r1 = await invoke(naiveHandler, { orders: [{ id: "o-1" }, { id: "o-2" }, { id: "o-3" }] });
console.log("invocation 1:", r1.result, "writes left unfinished:", r1.unfinished);
console.log("written after invocation 1:", written.length);

// The environment thaws for the next request; old work may run now, attributed to the wrong request.
pending.splice(0).forEach((run) => run());
console.log("written during invocation 2:", written.map((w) => w.orderId));

invocation 1: { ok: true } writes left unfinished: 3
written after invocation 1: 0
written during invocation 2: [ 'o-1', 'o-2', 'o-3' ]

Root cause

Lambda considers an invocation finished when the async handler's promise settles (or the callback is called). It then freezes the execution environment. forEach ignores the promises returned by an async callback, so the handler resolves immediately. Any timers, sockets and promises still pending are suspended: they may never complete (the environment is recycled), or they continue during the next invocation, where their errors and logs belong to the wrong request. Locally, Node keeps the process alive until the event loop is empty, which hides the bug.

The fix

Await everything the response depends on, and decide deliberately how to handle partial failure:

export const handler = async (event) => {
  const results = await Promise.allSettled(
    event.orders.map((order) => db.put({ table: "audit", item: { orderId: order.id, at: Date.now() } })),
  );
  const failed = results.filter((r) => r.status === "rejected").length;
  if (failed) throw new Error(`${failed} audit writes failed`);   // let the caller or the retry policy handle it
  return { ok: true, written: results.length };
};

const written = [];
const db = { put: async (item) => { await Promise.resolve(); written.push(item); } };

async function handler(event) {
  const results = await Promise.allSettled(event.orders.map((o) => db.put({ orderId: o.id })));
  return { ok: true, written: results.filter((r) => r.status === "fulfilled").length };
}

console.log(await handler({ orders: [{ id: "o-1" }, { id: "o-2" }, { id: "o-3" }] }));
console.log("written:", written.map((w) => w.orderId));

{ ok: true, written: 3 }
written: [ 'o-1', 'o-2', 'o-3' ]

Use an async handler (callback-style handlers are legacy, and newer Node.js runtimes drop them), and never mix async with the callback argument.

Trade-offs

• Promise.all fails fast on the first error; Promise.allSettled lets you report exactly which items failed. For large batches, limit concurrency so you do not open hundreds of connections at once.
• If work really can happen later, hand it off durably (SQS, EventBridge) instead of "fire and forget" inside the function.
• Waiting for everything adds to the response time, which is the honest cost of the work.

Detect it in production

• Log lines whose request ID does not match the invocation that started them, and errors logged after END RequestId.
• Lint rules: @typescript-eslint/no-floating-promises and no-misused-promises catch most un-awaited promises at build time.
• Reconciliation jobs that compare expected and written records.

Key points

• Lambda freezes the environment when the handler's promise settles.
• forEach with an async callback does not wait; use Promise.all/allSettled or for...of with await.
• Use async handlers; never rely on background work inside the function.
• Lint for floating promises.

Further reading: AWS Lambda: define a Node.js handler · Lambda execution environment lifecycle