Safe Use of Model Output — Integrating LLM APIs

Model output is untrusted input to the rest of your system. Escape/sanitize before rendering HTML/Markdown (XSS, malicious links), never eval generated code outside a sandbox, use parameterized queries if the model proposes filters (or better: structured filters you translate), validate URLs before fetching, and keep humans in the loop for consequential decisions (credit, hiring, medical, legal). Label AI-generated content where users might rely on it.

Handling generated content

Output used as	Risk	Control
HTML/Markdown in UI	XSS, phishing links	Sanitize (e.g. DOMPurify), restrict link/image domains
SQL / queries	Injection, data leakage	Structured filters → parameterized queries; read-only role
Code to execute	RCE	Sandboxed containers, no secrets, resource limits
Shell commands	System compromise	Allowlisted commands, approvals
Decisions about people	Unfair or wrong outcomes	Human review, appeal, audit trail

import DOMPurify from "dompurify";
import { marked } from "marked";

const html = DOMPurify.sanitize(await marked.parse(modelMarkdown), {
  ALLOWED_URI_REGEXP: /^https:\/\/(docs\.example\.com|help\.example\.com)\//,   // only trusted links/images
});
container.innerHTML = html;

class OrderFilter(BaseModel):
    status: Literal["open", "shipped", "cancelled"] | None = None
    placed_after: date | None = None

f = client.messages.parse(model=MODEL, max_tokens=512, output_format=OrderFilter,
                          messages=[{"role": "user", "content": f"Convert to a filter: {user_query}"}]).parsed_output
stmt = select(Order).where(Order.customer_id == user.id)
if f.status: stmt = stmt.where(Order.status == f.status)
if f.placed_after: stmt = stmt.where(Order.placed_at >= f.placed_after)

Common pitfalls

Rendering raw model HTML.
Letting the model write SQL against a privileged connection.
Fully automated high-stakes decisions.

Key points

Treat output as untrusted
Structured intents over generated code/SQL
Humans decide high-stakes outcomes

Further reading: OWASP LLM05: Improper Output Handling

Practice questions

Why translate model output into structured filters instead of letting it write SQL?

Answer

Structured fields can be validated and mapped to parameterized queries with the user's scope, preventing injection and access to other data.

How can rendering model-generated Markdown leak data?

Answer

Injected instructions can make the model output an image or link whose URL contains sensitive data, which the browser fetches automatically; sanitize and restrict URLs.