Bulkheads: Isolating Resources — Resilience Patterns

Ships use bulkheads so one flooded compartment doesn't sink the vessel. In software, give each dependency or workload its own bounded pool: separate thread pools/semaphores per downstream service, separate connection pools, separate worker queues for batch vs interactive work, and separate deployments (or cells) for critical paths. A slow recommendations service then can't consume the threads checkout needs.

Bulkhead levels

Bulkhead level	Isolates	Example
Per-dependency pool	Slow downstream	20 concurrent calls to recommendations
Per-workload queues	Batch vs interactive	Separate worker pools
Per-tenant limits	Noisy neighbors	Concurrency per tenant
Separate deployments/cells	Whole failure domains	Checkout runs on its own fleet

BulkheadConfig cfg = BulkheadConfig.custom().maxConcurrentCalls(20).maxWaitDuration(Duration.ZERO).build();
Bulkhead recsBulkhead = Bulkhead.of("recommendations", cfg);
Supplier<List<Product>> call = Bulkhead.decorateSupplier(recsBulkhead, () -> recs.forUser(userId));
// BulkheadFullException → fallback immediately instead of queueing

var recsSlots = make(chan struct{}, 20)   // at most 20 concurrent calls to recommendations

func recommendations(ctx context.Context, userID string) ([]Product, error) {
    select {
    case recsSlots <- struct{}{}:
        defer func() { <-recsSlots }()
        return recsClient.ForUser(ctx, userID)
    default:
        return popularCached(), nil      // bulkhead full: degrade, don't wait
    }
}

recs_slots = asyncio.Semaphore(20)

async def recommendations(user_id: str):
    if recs_slots.locked():
        return await popular_cached()      # full: degrade immediately
    async with recs_slots:
        return await recs_client.for_user(user_id)

Common pitfalls

One shared HTTP client pool for all dependencies.
Bulkheads that queue indefinitely instead of rejecting.
Pools sized without load testing.

Key points

Bounded pools per dependency/workload
Reject or degrade when full
Cells for big blast radius

Further reading: Azure Architecture: Bulkhead pattern

Practice questions

How does a bulkhead differ from a circuit breaker?

Answer

A bulkhead limits how many resources a dependency can consume at once (isolation); a circuit breaker stops calls entirely when the dependency is failing (fail fast). They're complementary.

Why should a full bulkhead usually reject rather than queue?

Answer

Queueing adds latency and holds resources; failing fast with a fallback keeps the caller responsive.