Sign in Get started

Free course · 20 lessons · Intermediate

Integrating LLM APIs

Ship LLM features like any other production dependency: models and tokens, prompts as code, streaming, structured outputs, tool calling, RAG, conversation state, rate limits and retries, prompt caching and cost, batch processing, fallbacks, evaluation, prompt injection, privacy, safe output handling and observability.

Start the course

1. LLM API Foundations

How LLM APIs work, choosing models and managing tokens.

  1. How LLM APIs Work — Stateless message APIs, tokens, context windows and the request/response shape.
  2. System Prompts, Messages & Prompt Templates — Structuring instructions, context and user input reliably.
  3. Choosing Models, Effort & Trade-offs — Capability vs latency vs cost; routing by task.
  4. Tokens, Context Windows & Long Inputs — Counting tokens, budgeting context and handling long documents and conversations.

2. Building Blocks

Streaming, structured outputs, tools, retrieval and conversation state.

  1. Streaming Responses to Users — Token streaming from the provider through your backend to browsers via SSE.
  2. Structured Outputs & Validation — Getting reliable JSON that matches a schema.
  3. Tool Calling & Agent Loops — Letting the model call your functions safely.
  4. Retrieval-Augmented Generation & Grounding — Answer from your data with citations instead of model memory.
  5. Conversation State, Memory & Storage — Persisting chats, editing history safely and long-term memory.

3. Reliability & Cost in Production

Rate limits, retries, caching, batching and graceful degradation.

  1. Rate Limits, Retries & Timeouts — Requests-per-minute and tokens-per-minute limits; retrying correctly.
  2. Cost Control: Prompt Caching, Routing & Budgets — Pay for fewer tokens and cheaper tokens.
  3. Batch APIs & Background Processing — Asynchronous, cheaper processing for non-interactive workloads.
  4. Stop Reasons, Refusals, Fallbacks & Graceful Degradation — What to do when the model or provider can't give you an answer.

4. Quality, Safety & Observability

Evals, prompt injection, privacy, safe actions and monitoring.

  1. Evaluating LLM Features — Golden datasets, graders and regression tests for prompts and models.
  2. Prompt Injection & Agent Security — Direct and indirect injection, and designing systems that stay safe anyway.
  3. Privacy, Data Handling & Compliance — What you send to providers, retention, PII and user rights.
  4. Safe Use of Model Output — Rendering, executing and acting on generated content.
  5. Observability for LLM Features — Tracing calls, tokens, cost, latency and quality signals.

5. Capstone

Build a production support assistant and an extraction pipeline.

  1. Capstone: AI Support Assistant — Design a grounded, tool-using assistant for an e-commerce help center.
  2. Capstone: Document Extraction Pipeline — Extract structured data from 200k invoices per day reliably and cheaply.