AI Agent Memory: Production Architecture and Evaluation

An AI agent does not gain memory merely because a chat log or embedding survives a process restart. Production memory is a governed mechanism for deciding what may influence future behavior, for whom, during which period, with what evidence, and how it can be corrected or deleted.

That distinction matters. Remembering a user’s explicit formatting preference can improve every future coding answer. Persisting a one-time medical disclosure, an adversarial instruction hidden in retrieved text, or an obsolete project decision can cause privacy harm and repeated errors.

The architecture should therefore optimize neither “maximum recall” nor “store everything.” It should optimize useful, authorized, current, attributable recall while making stale and unsafe memory cheap to reject. Technical claims were reviewed against primary research and current framework documentation on July 16, 2026.

Key Takeaways

• Separate thread state, durable memory, canonical business data, and immutable audit logs.
• Long-term memory can represent semantic facts, episodic examples, and procedural rules; these categories have different write and retrieval policies.
• A vector database is an optional index, not the source of truth or the memory lifecycle.
• Write less: require purpose, consent or another valid basis, provenance, confidence, time validity, sensitivity, and deletion policy.
• Never overwrite silently. Preserve events and represent corrections with versions or supersedes links.
• Retrieve by scope, time, authorization, type, and relevance; similarity alone is insufficient.
• Treat recalled memory as untrusted data, not system instruction.
• Evaluate extraction, retrieval, temporal reasoning, updates, abstention, privacy, latency, and deletion separately.

Four State Boundaries

The word “memory” often hides four systems with different ownership:

Do not copy canonical business state into a memory sentence and later trust it. If an agent needs an order status, query the order service. Memory may retain that the user often asks about a particular order only if that behavior is useful and permitted, but the current status must come from the source of truth.

Framework terminology follows the same distinction. Current LangGraph documentation describes checkpointers as thread-scoped persistence and stores as application-defined, cross-thread memory. A durable checkpoint is still not automatically a long-term user fact.

Semantic, Episodic, and Procedural Memory

The cognitive analogy is useful only when mapped to explicit application behavior.

Semantic Memory: Facts and Preferences

Examples:

• “The user explicitly prefers concise responses.”
• “Project Atlas uses PostgreSQL 17.”
• “The team’s fiscal year starts in April.”

Semantic memories need entity identity, provenance, time validity, and conflict handling. “The user likes Python” is not timeless truth if it came from one task or was inferred indirectly.

Episodic Memory: Past Experiences

An episode is a structured account of a relevant event:

• task and environment;
• actions and tool calls;
• observations;
• outcome and verifier results;
• failure cause;
• source trace and timestamp.

A raw transcript is evidence from which an episode may be built, not automatically a useful episodic memory. Storing every token creates noise, privacy exposure, and retrieval cost. For a coding agent, a successful migration with passing tests may be reusable; the surrounding small talk is not.

Procedural Memory: How to Behave

Procedural memory captures reusable rules, workflows, or strategies:

• “Before altering a payment schema, run backward-compatibility checks.”
• “For this tenant, route refunds above $500 to human approval.”
• “When deployment health checks fail, stop rather than retrying indefinitely.”

These rules can overlap with prompts, policies, playbooks, or code. High-impact procedures should be versioned, reviewed, and preferably enforced deterministically rather than learned silently from conversation.

The Memory Lifecycle

A robust system has six stages:

observe -> propose -> validate/write -> retrieve -> use/verify -> revise/forget

1. Observe Without Promoting Everything

Capture only events allowed by the retention policy. Keep transient model context separate from durable storage. Data minimization starts before extraction, not after the database fills up.

2. Propose Candidate Memories

An LLM can propose a structured candidate, but it should not have unilateral write authority. The candidate should identify:

• subject and predicate;
• value;
• memory type;
• exact source evidence;
• whether the user stated it or the model inferred it;
• confidence;
• valid time and observed time;
• sensitivity and purpose.

Inference must be labeled as inference. “User asked for a vegan recipe” does not prove “user is vegan.”

3. Apply a Write Policy

Reject or require confirmation when:

• future utility is unclear;
• the fact is transient;
• the source is untrusted or ambiguous;
• the information is sensitive;
• consent or another valid basis is absent;
• a canonical source should be queried instead;
• the candidate duplicates an existing active memory;
• a contradiction cannot be resolved safely.

Some products should expose a “Remember this” action and a memory dashboard. Explicit control is preferable to invisible profiling.

4. Persist Source and Index Separately

Store the authoritative record in a transactional database and build lexical, vector, graph, or time indexes as derived views. PostgreSQL can support structured filters, full-text search, and vector search through extensions such as pgvector; “SQL cannot do semantic search” is false.

Choose storage by access pattern:

• exact entity and temporal updates -> relational/document store;
• semantic candidate generation -> vector index;
• connected entities and provenance -> graph representation where justified;
• immutable raw events -> append-only event or object storage.

The application will often combine them. The vector hit should resolve back to a governed record.

5. Retrieve, Filter, and Pack

Retrieval order should be:

1. authenticate the caller;
2. enforce tenant, subject, purpose, and sensitivity scope;
3. filter active validity intervals and deletion state;
4. search by exact keys, time, lexical terms, vectors, or graph relationships;
5. rerank for the current task;
6. deduplicate and fit a memory token budget;
7. attach provenance and uncertainty.

A global top_k=5 is not a safety or quality policy. A temporal question may need an older superseded fact plus the update event; a preference question may need only the latest confirmed value.

6. Verify Use, Then Revise or Forget

Before acting, distinguish confirmed memory from inference and verify high-impact facts against canonical systems. Record which memory IDs affected the answer.

Support:

• correction without losing audit history;
• expiry and time-bounded validity;
• user-visible deletion;
• retention-based pruning;
• deletion propagation to vectors, summaries, caches, replicas, and backups according to policy;
• re-indexing when models or schemas change.

“Memory decay” should not mean deleting old facts solely because they were not retrieved. Some rare facts remain important; some recent facts are already obsolete.

A Versioned Memory Record

A practical record needs more than text and embedding:

{
  "memory_id": "mem_01J...",
  "tenant_id": "tenant_42",
  "subject_id": "user_123",
  "type": "semantic",
  "predicate": "preferred_programming_language",
  "value": "Rust",
  "source_event_id": "evt_987",
  "assertion": "explicit_user_statement",
  "confidence": 1.0,
  "observed_at": "2026-07-16T09:30:00Z",
  "valid_from": "2026-07-16T09:30:00Z",
  "valid_to": null,
  "supersedes": "mem_older_python",
  "purpose": "coding_assistance",
  "sensitivity": "low",
  "consent_basis": "user_requested_memory",
  "status": "active",
  "schema_version": 1
}

The previous Python preference should become superseded, not be rewritten into “the user likes programming languages.” That lossy consolidation destroys the timing and meaning of the correction.

Runnable Temporal Resolution

The following standard-library example selects a valid memory without pretending that similarity ranking resolves time or authorization:

from dataclasses import dataclass
from datetime import datetime, timezone


@dataclass(frozen=True)
class Memory:
    memory_id: str
    tenant_id: str
    subject_id: str
    predicate: str
    value: str
    valid_from: datetime
    valid_to: datetime | None = None
    status: str = "active"


def resolve_memory(
    memories: list[Memory],
    *,
    tenant_id: str,
    subject_id: str,
    predicate: str,
    at: datetime,
) -> Memory | None:
    if at.tzinfo is None:
        raise ValueError("at must be timezone-aware")

    candidates = [
        memory
        for memory in memories
        if memory.tenant_id == tenant_id
        and memory.subject_id == subject_id
        and memory.predicate == predicate
        and memory.status == "active"
        and memory.valid_from <= at
        and (memory.valid_to is None or at < memory.valid_to)
    ]
    if not candidates:
        return None
    return max(candidates, key=lambda memory: memory.valid_from)


utc = timezone.utc
history = [
    Memory(
        "mem_python", "tenant_42", "user_123",
        "preferred_language", "Python",
        datetime(2025, 1, 1, tzinfo=utc),
        datetime(2026, 7, 16, tzinfo=utc),
    ),
    Memory(
        "mem_rust", "tenant_42", "user_123",
        "preferred_language", "Rust",
        datetime(2026, 7, 16, tzinfo=utc),
    ),
]
print(resolve_memory(
    history,
    tenant_id="tenant_42",
    subject_id="user_123",
    predicate="preferred_language",
    at=datetime(2026, 7, 17, tzinfo=utc),
))

The database should enforce non-overlapping validity for single-valued predicates or route ambiguous states to conflict resolution. Application filtering does not replace row-level security or separate encryption boundaries.

Conflict, Uncertainty, and Time

Use an explicit operation:

• ADD: a new independent fact;
• CONFIRM: new evidence supports an existing fact;
• SUPERSEDE: a later valid value replaces an earlier one;
• RETRACT: the source or user says the fact was wrong;
• EXPIRE: the valid period ended;
• DELETE: remove under user request or policy.

Do not let “more recent” automatically win. A recent model inference should not replace an older explicit user statement. Rank evidence authority first, then temporal applicability.

Time has at least two dimensions:

• observed time: when the system learned the information;
• valid time: when the information is true in the modeled world.

“I move to Berlin next month” is observed now but valid later. LongMemEval explicitly tests temporal reasoning and knowledge updates because naive semantic retrieval commonly loses this distinction.

Security and Privacy

Long-term memory compounds risk because one malicious or accidental write can influence many future sessions.

• Treat recalled text as untrusted data and delimit it from system instructions.
• Do not store instructions found in emails, web pages, or tool output as procedural memory without validation.
• Apply authorization before vector search where possible and verify every result afterward.
• Use tenant isolation, row-level policy, encryption keys, and audited service authorization; metadata filtering alone is insufficient.
• Classify sensitivity and avoid embedding secrets or highly sensitive data unless the design explicitly supports it.
• Defend against cross-user leakage in caches, logs, evaluation datasets, and traces.
• Provide view, correct, forget, export, and disable-memory controls where appropriate.
• Never infer sensitive traits for personalization without a justified purpose and governance.

Memory inserted into a prompt should be labeled as potentially stale context with source and timestamp. It must not outrank developer policy or current tool results.

Evaluation: Test the Whole Lifecycle

LongMemEval organizes conversational memory around information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention. A production suite should extend that model.

Write Quality

• precision and recall of useful candidates;
• inference-vs-explicit classification;
• duplicate, sensitive, and transient write rates;
• correct subject and tenant association;
• time and provenance extraction accuracy.

Retrieval Quality

• evidence Recall@k and nDCG;
• temporal and entity filters;
• stale-memory retrieval rate;
• cross-user or cross-tenant leakage rate;
• unique useful tokens under a fixed context budget.

Use Quality

• task accuracy with and without memory;
• citation/provenance correctness;
• contradiction handling and appropriate abstention;
• personalization benefit without irrelevant references;
• harm from incorrect or injected memory.

Lifecycle and Operations

• update, retraction, expiry, and deletion correctness;
• propagation time to every derived store;
• p50/p95 read and write latency;
• storage and embedding cost per active useful memory;
• performance after model, index, or schema migration.

Include negative tests: a user who never consented, two users with similar names, conflicting preferences, future-dated facts, deleted facts, malicious memory text, and a query for which no memory should be used.

Frameworks: What to Evaluate

Mem0, Zep, Letta/MemGPT, LangGraph stores, and custom architectures occupy different layers and evolve quickly. Evaluate capabilities instead of copying a short SDK snippet:

• schema and provenance control;
• synchronous versus background writes;
• conflict and temporal semantics;
• namespace and authorization enforcement;
• hybrid retrieval and reranking;
• user correction and deletion APIs;
• observability and evaluation hooks;
• exportability and migration;
• pricing, hosting, and data residency.

MemGPT’s virtual-context design is an influential example of paging between limited prompt context and external storage. It does not remove the need for governance: allowing a model to decide what to remember makes write validation more important.

Production Checklist

• [ ] Every memory type has an explicit future-use case.
• [ ] Thread state, business truth, audit logs, and durable memory are separate.
• [ ] Writes require provenance, time, purpose, sensitivity, and valid authority.
• [ ] Inference is never presented as an explicit user fact.
• [ ] Updates preserve history through versioning or supersedes.
• [ ] Retrieval enforces tenant, subject, purpose, time, and deletion state.
• [ ] Recalled memory is treated as untrusted context.
• [ ] Users can inspect and correct persistent personalization where appropriate.
• [ ] Deletion reaches indexes, summaries, caches, replicas, and retention workflows.
• [ ] Evaluation covers writes, recall, time, conflicts, abstention, privacy, and cost.

Conclusion

Useful agent memory is selective and reversible. Its quality comes from disciplined writes, authoritative provenance, temporal semantics, secure retrieval, explicit conflict handling, and measurable user benefit.

Start with thread persistence and canonical tools. Add one narrowly defined memory type only when a real cross-session task requires it. A smaller memory store that users can understand, correct, and delete is better than an impressive vector index that silently accumulates guesses.

Primary Sources

• LangChain/LangGraph Memory Overview
• LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
• MemGPT: Towards LLMs as Operating Systems
• Generative Agents: Interactive Simulacra of Human Behavior