What's the difference between latency and throughput?

Summary

Latency is how long one operation takes (e.g., p99 request time); throughput is how many operations complete per unit of time (requests/second). Improving one doesn't automatically improve the other; batching raises throughput but can increase latency.

Longer Answer

Little's Law links them: concurrency = throughput × latency. If latency doubles at the same throughput, you need twice the concurrent capacity (threads, connections).

Key Points

• Latency: time per operation
• Throughput: operations per second
• Little's Law: L = λ × W
• Batching trades latency for throughput