Offload CPU-heavy work with worker threads

Approach

CPU-bound work blocks the event loop for every request. Move it to a worker thread pool (piscina) so the main thread keeps handling I/O. Size the pool to the number of cores minus one, and queue or reject when it's saturated.

Solution

// workers/thumbnail.ts
import sharp from "sharp";
export default async function makeThumbnail({ input, width }: { input: Uint8Array; width: number }) {
  return sharp(input).resize({ width }).webp({ quality: 80 }).toBuffer();
}

// server.ts
import os from "node:os";
import { Piscina } from "piscina";

const pool = new Piscina({
  filename: new URL("./workers/thumbnail.js", import.meta.url).href,
  maxThreads: Math.max(1, os.availableParallelism() - 1),
  maxQueue: 100,
});

app.post("/api/thumbnails", express.raw({ type: "image/*", limit: "10mb" }), async (req, res) => {
  try {
    const output = await pool.run({ input: req.body, width: 320 });
    res.type("image/webp").send(output);
  } catch (err) {
    if ((err as Error).message.includes("queue is full")) return res.status(503).set("Retry-After", "5").end();
    throw err;
  }
});

Pitfalls

• Spawning a new worker per request (expensive startup).
• Copying huge buffers between threads (transfer ArrayBuffers instead).
• Using worker threads for I/O-bound work (no benefit).

Variations

• Background job queues for work that doesn't need to finish within the request.
• Cluster mode/multiple processes for scaling I/O-heavy apps across cores.

Further reading: Node.js: worker_threads