Conversation State, Memory & Storage — Integrating LLM APIs

Because the API is stateless, you store conversations: messages with content blocks (including tool calls/results), model, prompt version, usage and timestamps. Append the full response content to history (not just extracted text) so tool calls and other block types stay consistent. Prefer append-only histories — rewriting earlier turns breaks caching and can invalidate provider features that depend on the exact prior content. For long-term memory, extract durable facts into a store the user can view and delete, and retrieve them as context.

What to persist

Data	Store	Retention
Messages (full content blocks)	Relational/document DB	Per policy; user-deletable
Usage + cost per turn	Analytics table	Long
Prompt/model version per turn	With the turn	Long
Long-term memories (facts)	Table + embeddings	Until user deletes
Uploaded files	Object storage	Per policy

Schema

CREATE TABLE conversations (id uuid PRIMARY KEY, user_id bigint NOT NULL, title text, created_at timestamptz DEFAULT now());
CREATE TABLE turns (
  id uuid PRIMARY KEY, conversation_id uuid REFERENCES conversations(id) ON DELETE CASCADE,
  seq int NOT NULL, role text NOT NULL, content jsonb NOT NULL,     -- full content blocks
  model text, prompt_id text, input_tokens int, output_tokens int, cache_read_tokens int,
  stop_reason text, created_at timestamptz DEFAULT now(), UNIQUE (conversation_id, seq)
);

Common pitfalls

Storing only the final text and losing tool calls.
Rewriting history mid-conversation.
Memories users can't see or delete (privacy).

Key points

Persist full content blocks + metadata
Append-only histories
User-controllable memory

What to persist

Common pitfalls

Key points

Practice questions