Bulk and Batch Operations — REST APIs in Depth

Clients importing thousands of records shouldn't make thousands of requests. Options: a bulk endpoint on the collection (POST /products:batchCreate with an array), an async import job for very large inputs (upload a file → 202 → job status), or HTTP batch (multiple sub-requests in one call — rarely worth the complexity). Decide and document atomicity: all-or-nothing (one transaction, simpler semantics) or partial success with per-item results (status per item, 207-like semantics in the body). Cap batch sizes and make batches idempotent.

Bulk operation strategies

Criterion	All-or-nothing	Partial success	Async import job
Semantics	Simple: one result	Per-item statuses	Job status + report
Max size	Small–medium	Medium	Very large (files)
Client retries	Retry whole batch	Retry failed items only	Re-submit file / resume
Server cost	One transaction (locks)	Many small writes	Background workers

POST /products:batchCreate
Idempotency-Key: 9b1c…
Content-Type: application/json

{"items": [{"sku": "A1", "name": "Mug"}, {"sku": "A1", "name": "Dup"}, {"sku": "B2", "name": "Bowl"}]}

HTTP/1.1 200 OK
{
  "results": [
    {"index": 0, "status": 201, "id": "prd_1"},
    {"index": 1, "status": 409, "error": {"title": "Duplicate SKU", "detail": "SKU A1 already exists"}},
    {"index": 2, "status": 201, "id": "prd_2"}
  ]
}

Common pitfalls

Undefined behavior when some items fail.
Huge synchronous batches that time out.
No idempotency, so retries duplicate half the batch.

Key points

Choose atomic vs partial explicitly
Per-item results with indexes
Async jobs for large imports

Further reading: Google AIP-233: Batch methods

Practice questions

Why return per-item results with the item index?

Answer

Clients can match each result to their input (even without IDs) and retry only the failed items.

When should a bulk import become an asynchronous job?

Answer

When processing time could exceed request timeouts or input is large (files, many thousands of rows); return 202 with a job resource that reports progress and per-row errors.