Cost-Based Limits: GraphQL, Search and LLM Tokens — Rate Limiting, Quotas & Abuse Protection

Requests differ wildly in cost. Assign costs: GraphQL query complexity (GitHub's node limits and point-based rate limits; Shopify's calculated query cost leaky bucket), search/export weights, or LLM tokens (input + output tokens per minute and per day). Deduct estimated cost up front, then adjust to actual cost after the request completes, and return the remaining budget to clients.

Cost units

API type	Cost unit	Example
GraphQL	Calculated query cost / points	GitHub: points per hour; Shopify: cost-based bucket
Search	Weighted per query type	Facets ×5, export ×50
LLM proxy	Tokens per minute/day	TPM + RPM limits per key
Data export	Rows or bytes	1M rows/day per tenant

export async function chat(apiKey: string, req: ChatRequest) {
  const estimate = estimateTokens(req.messages) + req.max_tokens;
  const reserve = await limit(`tpm:${apiKey}`, 200_000, 200_000 / 60, estimate);   // token bucket in tokens
  if (!reserve.ok) throw new RateLimited(reserve.retryAfter);
  const result = await llm.complete(req);
  const actual = result.usage.input_tokens + result.usage.output_tokens;
  if (actual < estimate) await refund(`tpm:${apiKey}`, estimate - actual);   // return unused budget
  return result;
}

# Shopify-style: responses report cost and remaining budget
{
  "data": { ... },
  "extensions": { "cost": { "requestedQueryCost": 252, "actualQueryCost": 140,
    "throttleStatus": { "maximumAvailable": 2000, "currentlyAvailable": 1860, "restoreRate": 100 } } }
}

Common pitfalls

Counting a 1-field query and a 10,000-node query the same.
No refund of over-estimated cost.
Not exposing remaining budget to clients.

Key points

Limit by cost units
Estimate, then adjust to actual
Expose budget to clients

Further reading: Shopify: API rate limits

Practice questions

Why do GraphQL APIs often use cost-based rate limits?

Answer

A single GraphQL request can be trivial or enormously expensive; counting requests doesn't reflect load, so they compute a cost per query and limit total cost.

How do you rate-limit LLM usage fairly?

Answer

By tokens (input + output) per minute and per day per key/tenant, reserving an estimate before the call and correcting with actual usage afterwards, alongside request-per-minute limits.