Privacy, Data Handling & Compliance — Integrating LLM APIs

Every prompt is data processing. Know your provider's data retention and training policies and contractual terms (DPA, zero-data-retention options, regional processing), minimize what you send (redact or pseudonymize PII not needed for the task), apply retention to stored conversations, honor deletion requests across logs, memories and vector stores, and document the processing for privacy reviews (GDPR records, DPIAs for high-risk uses). Be transparent with users that AI is involved.

Privacy checklist

Topic	Question to answer	Typical control
Provider retention	How long are prompts kept?	Contract terms; ZDR where available
Training use	Is API data used for training?	Provider policy / contract
Data location	Where is it processed?	Regional endpoints / inference geography options
Minimization	Do we send unneeded PII?	Redaction, pseudonymization
Retention (yours)	How long do we keep chats/logs?	TTL jobs, user controls
Deletion	Can we erase a user's data everywhere?	Delete across DB, vectors, logs, memories

Python (pseudonymize before sending)

def pseudonymize(text: str, vault: dict[str, str]) -> str:
    def repl(m: re.Match) -> str:
        token = f"<EMAIL_{len(vault) + 1}>"
        vault[token] = m.group(0)
        return token
    return EMAIL_RE.sub(repl, text)

vault: dict[str, str] = {}
safe = pseudonymize(ticket.body, vault)
reply = generate_reply(safe)
final = reduce(lambda t, kv: t.replace(*kv), vault.items(), reply)   # restore for the agent UI

Common pitfalls

Logging full prompts with PII indefinitely.
Forgetting vector stores and caches in deletion flows.
No user disclosure of AI processing.

Key points

Know provider retention/training terms
Minimize and redact PII
Deletion covers every store

Further reading: GDPR Art. 25: Data protection by design

Practice questions

Why must right-to-erasure flows include vector stores?

Answer

Embeddings and chunks of user content are personal data derived from the user and can be retrieved into future prompts.

What's data minimization in an LLM context?

Answer

Sending only the data the task needs — e.g. redacting emails, phone numbers or account IDs that don't affect the answer.