Background Job Frameworks by Stack — Queues & Background Processing in Depth

Most stacks have a mature job framework that handles serialization, retries, scheduling and monitoring on top of a broker: Celery/RQ/ARQ/Dramatiq (Python), BullMQ (Node), Sidekiq/Solid Queue (Ruby), Laravel Queues/Symfony Messenger (PHP), JobRunr/Spring Batch/Quartz (Java), Hangfire (.NET), Oban (Elixir, Postgres-backed). Postgres-backed queues (SKIP LOCKED) are a solid choice at moderate scale because enqueueing can be transactional with your data.

Job frameworks

Stack	Popular options	Broker
Python	Celery, RQ, ARQ, Dramatiq	RabbitMQ / Redis
Node.js	BullMQ	Redis
PHP	Laravel Queues, Symfony Messenger	Redis / DB / SQS / AMQP
Java	JobRunr, Spring Batch, Quartz	DB / various
.NET	Hangfire, MassTransit	DB / RabbitMQ / Azure SB
Postgres-native	Oban, Graphile Worker, pg-boss, river	PostgreSQL

@app.task(bind=True, autoretry_for=(TemporaryError,), retry_backoff=True, retry_jitter=True, max_retries=6, acks_late=True)
def send_receipt(self, order_id: str):
    order = Order.get(order_id)
    if order.receipt_sent_at:
        return                            # idempotent
    mailer.send_receipt(order)
    order.mark_receipt_sent()

send_receipt.delay("ord_1")               # after the DB transaction commits

const queue = new Queue("receipts", { connection });
await queue.add("send", { orderId: "ord_1" }, { attempts: 6, backoff: { type: "exponential", delay: 5000 }, jobId: "receipt:ord_1" });

new Worker("receipts", async (job) => sendReceiptOnce(job.data.orderId), { connection, concurrency: 20 });

class SendReceipt implements ShouldQueue {
    use Dispatchable, InteractsWithQueue, Queueable;
    public int $tries = 6;
    public function backoff(): array { return [5, 30, 120, 600]; }
    public function __construct(public string $orderId) {}
    public function handle(Mailer $mailer): void { /* idempotent send */ }
}
SendReceipt::dispatch('ord_1')->afterCommit();

@Job(name = "Send receipt %0", retries = 6)
public void sendReceipt(String orderId) { receipts.sendOnce(orderId); }

BackgroundJob.enqueue(() -> receiptService.sendReceipt("ord_1"));

Common pitfalls

Enqueuing before the transaction commits (afterCommit / outbox).
Passing large objects instead of IDs.
Running workers inside web processes.

Key points

Use your stack's framework
Enqueue after commit
Pass IDs; handlers idempotent

Job frameworks

Common pitfalls

Key points

Practice questions