Batch APIs & Background Processing

Classification of a million reviews, nightly summarization, backfilling embeddings — none of it needs real-time responses. Batch APIs accept many requests at once, process them asynchronously (results usually within hours) at a discount (about 50% on several providers). Give each request a custom ID and map results by ID — they may return in any order. For workloads that need minutes, not hours, use your own job queue with concurrency tuned to rate limits.

Execution modes

Python (Message Batches)

from anthropic.types.message_create_params import MessageCreateParamsNonStreaming
from anthropic.types.messages.batch_create_params import Request

batch = client.messages.batches.create(requests=[
    Request(custom_id=f"review-{r.id}",
            params=MessageCreateParamsNonStreaming(
                model=MODEL, max_tokens=256,
                messages=[{"role": "user", "content": f"Classify sentiment: {r.text}"}]))
    for r in reviews
])
# later (scheduled job): poll until processing_status == "ended", then:
for result in client.messages.batches.results(batch.id):
    if result.result.type == "succeeded":
        save(result.custom_id, result.result.message)
    else:
        retry_later(result.custom_id, result.result.type)      # errored / expired / canceled

Common pitfalls

• Mapping results by position instead of custom_id.
• Using batch for user-facing requests.
• Not handling expired/errored items.

Key points

• Batch non-interactive work for discounts
• custom_id for mapping
• Handle partial failures

Further reading: Batch processing