How LLM APIs Work — Integrating LLM APIs

Modern LLM APIs (Anthropic Messages, OpenAI Responses/Chat, Google Gemini, open-weight servers like vLLM) share one model: you send a system prompt plus a list of messages (user/assistant turns, possibly with images, documents and tool results) and receive generated content blocks plus usage (input/output tokens) and a stop reason. The API is stateless — you resend the conversation each time (or use a provider-side conversation feature). Everything is billed and limited in tokens, and every model has a context window (max input + output tokens per request).

Examples in this course use Anthropic's SDK (Python and TypeScript) because it's documented and current; the patterns — streaming, structured outputs, tool calling, caching, batching, retries, evaluation — apply to every provider.

Core concepts

Concept	What it is	Why it matters for engineers
Token	Sub-word unit the model reads/writes	Unit of cost, latency and limits
Context window	Max tokens per request	Bounds history, documents and tool output you can send
max_tokens	Cap on output tokens per response	Too low truncates (stop_reason = max_tokens)
Stop reason	Why generation ended	Branch logic: end_turn, max_tokens, tool_use, refusal…
Usage	Input/output (and cache) token counts	Cost accounting and budgets
Statelessness	No memory between calls	You own conversation storage

import anthropic

client = anthropic.Anthropic()          # reads ANTHROPIC_API_KEY from the environment
response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=16000,
    system="You are a concise assistant for an e-commerce support team.",
    messages=[{"role": "user", "content": "Summarize this ticket: …"}],
)
text = "".join(b.text for b in response.content if b.type == "text")
print(response.stop_reason, response.usage.input_tokens, response.usage.output_tokens)

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();         // reads ANTHROPIC_API_KEY
const response = await client.messages.create({
  model: "claude-opus-5-5",
  max_tokens: 16000,
  system: "You are a concise assistant for an e-commerce support team.",
  messages: [{ role: "user", content: "Summarize this ticket: …" }],
});
const text = response.content.flatMap((b) => (b.type === "text" ? [b.text] : [])).join("");
console.log(response.stop_reason, response.usage);

Common pitfalls

Calling the provider directly from browsers or mobile apps (exposes your API key).
Ignoring stop_reason and treating truncated output as complete.
Assuming the model remembers previous requests.

Key points

Messages in, content blocks + usage + stop reason out
Tokens drive cost, latency and limits
Your backend owns keys and conversation state

Core concepts

Common pitfalls

Key points

Practice questions