Structured Outputs & Validation — Integrating LLM APIs

When code consumes model output, ask for structured output constrained to a JSON Schema (most providers now support schema-constrained decoding) and still validate with your own types. Design schemas for the model: clear field names and descriptions, enums for categories, null allowed when information is missing (so the model doesn't invent values). Handle the cases where you get no usable object: refusal, truncation (max_tokens) or business-rule validation failures.

Getting structured data

Approach	Reliability	Notes
'Respond in JSON' in prompt	Low–medium	Needs repair/retries
Tool/function call with strict schema	High	Good when the output is an action
Schema-constrained output (structured outputs)	High	Best for extraction/classification
Post-validation with Pydantic/Zod	Required anyway	Business rules beyond the schema

from pydantic import BaseModel, Field
from typing import Literal

class TicketTriage(BaseModel):
    category: Literal["billing", "shipping", "returns", "technical", "other"]
    urgency: Literal["low", "medium", "high"]
    order_id: str | None = Field(None, description="Order number if present, else null")
    summary: str

response = client.messages.parse(
    model=MODEL, max_tokens=2048,
    messages=[{"role": "user", "content": f"<ticket>{ticket_text}</ticket>\nTriage this ticket."}],
    output_format=TicketTriage,
)
triage = response.parsed_output            # validated TicketTriage instance

import { z } from "zod";
import { zodOutputFormat } from "@anthropic-ai/sdk/helpers/zod";

const TicketTriage = z.object({
  category: z.enum(["billing", "shipping", "returns", "technical", "other"]),
  urgency: z.enum(["low", "medium", "high"]),
  order_id: z.string().nullable(),
  summary: z.string(),
});

const response = await client.messages.parse({
  model: MODEL, max_tokens: 2048,
  messages: [{ role: "user", content: `<ticket>${ticketText}</ticket>\nTriage this ticket.` }],
  output_config: { format: zodOutputFormat(TicketTriage) },
});
if (!response.parsed_output) throw new Error(`no structured output: ${response.stop_reason}`);

Common pitfalls

Required fields the input can't support (forces hallucination).
Regex-parsing free text.
Skipping validation because the API 'guarantees' JSON.

Key points

Schema-constrained output for machine consumers
Nullable fields for missing info
Validate and handle refusal/truncation

Getting structured data

Common pitfalls

Key points

Practice questions