Durable Workflows with Temporal — Queues & Background Processing in Depth

Chains of queues and state flags become hard to reason about for long processes (onboarding, order fulfillment, subscriptions). Durable execution engines (Temporal, Restate, AWS Step Functions, Azure Durable Functions) persist workflow progress so code can sleep for days, retry activities with policies, and resume after crashes. In Temporal, workflow code must be deterministic (it is replayed from history); side effects go in activities, which are retried and must be idempotent.

Choreographed queues vs durable workflows

Concern	Queues + state flags	Durable workflow engine
Visibility of process state	Scattered	Central history per workflow
Long waits / timers	Cron + polling	Durable sleep
Retries & compensation	Hand-rolled	Policies + try/catch
Operational cost	Low	Engine to run (or managed)

@workflow.defn
class OrderFulfillment:
    @workflow.run
    async def run(self, order_id: str) -> str:
        opts = dict(start_to_close_timeout=timedelta(seconds=30),
                    retry_policy=RetryPolicy(initial_interval=timedelta(seconds=2), maximum_attempts=10))
        await workflow.execute_activity(reserve_stock, order_id, **opts)
        try:
            await workflow.execute_activity(charge_payment, order_id, **opts)
        except ActivityError:
            await workflow.execute_activity(release_stock, order_id, **opts)   # compensation
            raise
        await workflow.execute_activity(create_shipment, order_id, **opts)
        await workflow.sleep(timedelta(days=7))                                 # durable timer
        await workflow.execute_activity(request_review, order_id, **opts)
        return "done"

const { reserveStock, chargePayment, releaseStock, createShipment } =
  proxyActivities<typeof activities>({ startToCloseTimeout: "30s", retry: { maximumAttempts: 10 } });

export async function orderFulfillment(orderId: string) {
  await reserveStock(orderId);
  try { await chargePayment(orderId); }
  catch (e) { await releaseStock(orderId); throw e; }
  await createShipment(orderId);
  await sleep("7 days");                                   // durable timer
}

Common pitfalls

Non-deterministic code in workflows (random, time, I/O).
Non-idempotent activities.
Using workflows for simple fire-and-forget jobs.

Key points

Durable execution for long processes
Deterministic workflows, idempotent activities
Timers and compensation built in

Choreographed queues vs durable workflows

Common pitfalls

Key points

Practice questions