Direct Answer

AI agent state persistence is the engineering discipline that lets an interrupted workflow resume without losing authorized progress or repeating unsafe side effects. It requires more than saving messages: define a recovery contract, persist state transitions atomically, isolate nondeterministic calls, make external effects idempotent or reconcilable, migrate old state safely, and prove recovery with crash tests.

A checkpoint preserves data at a boundary. It does not by itself restart workers, guarantee exactly-once execution, validate remembered facts, or make a business action correct.

State Persistence Is Not Long-Term Memory

Execution state and long-term memory cross different boundaries.

Data class Scope Example Authority
Workflow state One run current node, retry count, pending approval workflow runtime
Checkpoint Recoverable snapshot state after tool result was accepted persistence layer
Effect ledger External action status refund request accepted by payment API application protocol
Long-term memory Across runs confirmed preference or prior episode governed memory service
Business record Domain truth order status, account balance system of record
Artifact Large immutable output report, image, source archive object/artifact store
Audit evidence Forensics actor, policy version, state transition append-only audit store

Do not restore an obsolete order status from a checkpoint and treat it as current. The checkpoint may retain the order ID and the version previously observed; after recovery, the agent must revalidate mutable facts against their authoritative system.

The companion AI Agent memory guide covers what may become reusable memory. The memory privacy guide covers retention and deletion. This page covers interrupted execution and safe resumption.

Define the Recovery Contract First

A persistence design is testable only when the team states what recovery means.

Contract field Engineering question
Run identity Which tenant, workflow, run, and release own this state?
Recovery point objective How much accepted progress may be lost?
Recovery time objective How quickly must the run become actionable again?
Checkpoint boundary Before or after which transition is state committed?
Effect semantics Can a tool call be retried, deduplicated, compensated, or only reconciled?
Resume authority Which worker or operator may resume the run?
State compatibility Which code and schema versions can read this checkpoint?
Retention When are checkpoints, histories, and artifacts pruned?
Terminal states Which outcomes end automatic recovery?
flowchart LR A["Trigger with run ID"] --> B["Load authorized state"] B --> C["Validate workflow and schema version"] C --> D["Select next transition"] D --> E["Execute pure computation or managed effect"] E --> F["Commit state and evidence"] F --> G{"Terminal?"} G -- "No" --> D G -- "Yes" --> H["Close run and retain evidence"]

“Resume from where it stopped” is underspecified. A useful contract says whether the system resumes before an uncertain effect, after a confirmed effect, or in a reconciliation state that requires observation or human review.

Model the Agent as a Versioned State Machine

Persist explicit state and transitions rather than serializing an entire process heap.

json 
{
  "tenant_id": "tenant_42",
  "workflow_id": "refund-review",
  "workflow_version": "3.2.0",
  "run_id": "run_01J...",
  "state_version": 17,
  "status": "waiting_for_approval",
  "next_transition": "execute_refund",
  "input_refs": ["case:8472"],
  "completed_steps": ["load_case", "policy_check", "draft_decision"],
  "pending_effects": [],
  "approval": {
    "required_role": "refund_manager",
    "request_id": "approval_91"
  },
  "budgets": {
    "steps_remaining": 4,
    "cost_remaining": 0.72
  },
  "checkpoint_schema": 2,
  "created_at": "2026-08-09T10:20:00Z"
}

Store references to large outputs rather than duplicating blobs in every checkpoint. Exclude credentials and ephemeral client objects. Persist the policy, prompt, model, tool, and code release identifiers needed to explain or reproduce a decision, but do not assume old providers or models remain callable forever.

A Checkpoint Has a Crash Window

Every external effect creates an ambiguity boundary.

sequenceDiagram participant W as Worker participant S as State Store participant T as External Tool W->>S: Commit intent and idempotency key W->>T: Execute effect with same key T-->>W: Accepted, effect_id W->>S: Commit effect_id and next state

Consider failures:

  1. Crash before intent commit: recovery sees no authorized effect and may plan again.
  2. Crash after intent commit but before tool call: recovery may execute with the same key.
  3. Crash after the tool acts but before its response arrives: outcome is ambiguous; query by idempotency key or reconcile.
  4. Crash after response but before state commit: retry must return the same outcome or detect the existing effect.
  5. Crash after state commit: recovery advances without repeating the effect.

This is why a checkpoint cannot guarantee exactly-once behavior. Durable execution products coordinate event history and retries, but application-level side effects still need explicit semantics.

Make Side Effects Idempotent or Reconcilable

Each consequential tool call needs an effect protocol.

text 
effect_key = tenant_id + workflow_id + run_id + logical_step

The protocol should persist:

  • stable effect key;
  • normalized request hash;
  • target system and operation;
  • status: planned, executing, confirmed, rejected, ambiguous, compensated;
  • provider receipt or resource ID;
  • attempt count and last error;
  • reconciliation and compensation instructions.
Effect type Preferred strategy
Create payment/refund Provider idempotency key plus receipt lookup
Send email Outbox row plus delivery-provider message ID
Update owned database State and outbox in one local transaction
Third-party API without deduplication Read-after-write reconciliation or human gate
Irreversible physical action Precondition, approval, narrow command, post-action verification

Do not retry every exception. Authentication failures, policy denials, invalid requests, exhausted budgets, and ambiguous irreversible effects usually require a different transition than transient timeouts.

Choose Snapshot, Event History, or Both

Snapshots and event histories solve different recovery costs.

Model Strength Cost and risk
Latest snapshot Simple and fast restore Limited history; partial writes need transaction protection
Versioned snapshots Time travel and rollback Storage growth and schema migration
Append-only transition log Audit and deterministic reconstruction Replay cost, event evolution, side-effect isolation
Snapshot plus tail log Fast restore with evidence More moving parts and consistency checks

Event sourcing is not automatically superior. Replay only works when orchestration is deterministic relative to recorded events. Model calls, wall-clock time, random values, network responses, and tool effects must be recorded or isolated behind managed activities; otherwise replay can choose a different path.

Temporal documents durable workflow execution through event history and deterministic replay. LangGraph documents checkpoints as thread-scoped graph-state snapshots and stores as cross-thread application data. These are different contracts, not interchangeable product labels.

Select Storage by Contract, Not Diagram

A fixed Redis + relational database + vector database stack is not a maturity model.

Requirement Candidate capability
Atomic checkpoint and effect intent Transactional relational or document store
Conditional state transition Compare-and-swap, row version, or transactional lock
Large immutable artifacts Object storage with checksum
Low-latency hot reads Cache, if database latency misses the SLO
Semantic candidate retrieval Vector index, possibly inside the primary database
Deterministic event replay Durable workflow/event-history engine
Search and forensic analysis Audit/event index derived from authoritative records

A single database can be a sound starting point. Current database integrations can store JSON checkpoints, namespaced long-term records, and vector indexes together. Splitting stores introduces replication lag, partial failure, access-control duplication, backup coordination, and deletion propagation. Add a service only when measured requirements outweigh these costs.

WAL is a database durability mechanism, not an agent architecture. PostgreSQL uses write-ahead logging to recover committed database changes; it does not know whether a payment API accepted a request or whether the agent should resume a semantic step.

Handle Concurrency and Ownership

Only one owner should advance a sequential run state at a time unless the workflow explicitly supports parallel branches.

Use:

  • optimistic concurrency through state_version;
  • leases with fencing tokens for worker ownership;
  • branch IDs and deterministic reducers for parallel results;
  • unique constraints on effect keys;
  • authorization scoped by tenant and run;
  • monotonic terminal states that ordinary retries cannot reopen.
sql 
UPDATE agent_runs
SET state = :next_state,
    state_version = state_version + 1
WHERE tenant_id = :tenant_id
  AND run_id = :run_id
  AND state_version = :expected_version
  AND status NOT IN ('completed', 'cancelled', 'failed');

Zero updated rows means the worker lost the race or the run is closed. It must reload rather than overwrite newer state.

Multi-agent systems should not share one mutable memory object. Give each branch explicit input/output schemas and ownership; merge through reducers or a coordinator that can detect conflicting writes.

Migrate State Across Releases

Long-running runs can outlive the code version that created them. Every checkpoint therefore needs workflow and schema versions.

A release must define:

  1. readers for supported old schema versions;
  2. pure, tested migrations to the current schema;
  3. compatibility rules for renamed or removed transitions;
  4. behavior when a referenced tool or model no longer exists;
  5. a quarantine state for checkpoints that cannot migrate safely;
  6. rollback compatibility with checkpoints written by the new release.

Never deserialize arbitrary types from an untrusted checkpoint. Use a restricted schema, validate size and fields, authenticate integrity where needed, and treat stored model/tool text as untrusted data.

A Runnable Recovery Decision

The following standard-library function does not execute a tool. It determines the only safe recovery action from persisted evidence:

python 
from dataclasses import dataclass
from enum import Enum
from typing import Optional


class EffectStatus(str, Enum):
    NONE = "none"
    PLANNED = "planned"
    CONFIRMED = "confirmed"
    AMBIGUOUS = "ambiguous"


class RecoveryAction(str, Enum):
    EXECUTE = "execute"
    ADVANCE = "advance"
    RECONCILE = "reconcile"
    STOP = "stop"


@dataclass(frozen=True)
class Checkpoint:
    status: str
    effect_status: EffectStatus
    effect_key: Optional[str]
    attempts: int
    max_attempts: int


def recovery_action(checkpoint: Checkpoint) -> RecoveryAction:
    if checkpoint.status in {"completed", "cancelled", "failed"}:
        return RecoveryAction.STOP
    if checkpoint.effect_status is EffectStatus.CONFIRMED:
        return RecoveryAction.ADVANCE
    if checkpoint.effect_status is EffectStatus.AMBIGUOUS:
        return RecoveryAction.RECONCILE
    if checkpoint.attempts >= checkpoint.max_attempts:
        return RecoveryAction.STOP
    if not checkpoint.effect_key:
        return RecoveryAction.STOP
    return RecoveryAction.EXECUTE


assert recovery_action(
    Checkpoint("running", EffectStatus.PLANNED, "run-7:send", 0, 3)
) is RecoveryAction.EXECUTE
assert recovery_action(
    Checkpoint("running", EffectStatus.AMBIGUOUS, "run-7:send", 1, 3)
) is RecoveryAction.RECONCILE
assert recovery_action(
    Checkpoint("completed", EffectStatus.CONFIRMED, "run-7:send", 1, 3)
) is RecoveryAction.STOP

The key behavior is refusal: ambiguous effects are reconciled, not blindly retried.

Test Recovery as a Product Property

Happy-path tests do not prove durability. Build a failure matrix.

Injection point Required assertion
Before model call no progress claimed; retry budget intact
After model response, before commit response may be recomputed; no effect duplicated
Before tool request intent and key recoverable
After tool effect, before receipt run enters reconciliation
After receipt, before state commit deduplication recovers same effect ID
During checkpoint write prior committed version remains readable
During schema migration checkpoint is migrated or quarantined
During approval wait approval identity and state survive worker loss
Two workers resume together fencing/version check permits one transition
Cancellation races with retry terminal cancellation cannot be reopened

Track:

  • recovery success rate by failure point;
  • recovery time and progress loss;
  • duplicate and untracked effect count;
  • ambiguous effects requiring intervention;
  • stale or incompatible checkpoint count;
  • checkpoint write latency and size;
  • replay length and migration failures;
  • retention and deletion completion.

An agent that returns a plausible answer after restart but repeats a refund or ignores a cancellation has failed recovery.

Framework Mapping Without Lock-In

Concept LangGraph example Durable workflow example
Run identity thread_id and checkpoint ID workflow ID and run ID
State persistence checkpointer event history plus workflow state
Cross-run data store external application store
Resume invoke from persisted thread state replay history and continue
Side effect node/tool code needs application protocol managed activity still needs business idempotency

Framework documentation changes. Keep the recovery contract, state schema, effect keys, and failure tests in application-owned artifacts so a backend can be replaced without changing business semantics.

Common Failure Modes

  • Message-log persistence: chat history is saved, but pending effects and approvals are not.
  • Checkpoint-after-effect gap: an action succeeds externally and is repeated after a crash.
  • Whole-object serialization: secrets, clients, and incompatible runtime types enter storage.
  • Mutable truth in snapshots: restored business facts bypass the system of record.
  • One global session key: tenants, users, or parallel runs overwrite each other.
  • Last-write-wins recovery: stale workers overwrite newer checkpoints.
  • Unbounded history: checkpoints and event histories grow without retention or compaction.
  • Replay drift: new code interprets old transitions differently.
  • Cache-as-authority: cache eviction becomes data loss.
  • Vector-store conflation: semantic search is mistaken for workflow durability.

Production Checklist

  • [ ] Every run has tenant, workflow, release, run, and state-version identifiers.
  • [ ] Terminal states, budgets, approvals, and pending effects are explicit.
  • [ ] Checkpoint commits are atomic and conditionally versioned.
  • [ ] Every external effect is idempotent, compensatable, or reconcilable.
  • [ ] Mutable business facts are revalidated after recovery.
  • [ ] State schemas have compatibility and quarantine paths.
  • [ ] Stored content is validated and deserialized safely.
  • [ ] Retention covers checkpoints, logs, artifacts, indexes, and backups.
  • [ ] Crash tests cover every side of the effect and commit boundary.
  • [ ] Recovery dashboards expose ambiguous effects and stuck runs.

Primary Sources