What is Agent Runtime?
Agent Runtime is the execution environment that creates, schedules, isolates, advances, suspends, resumes, cancels, and records AI agent runs while mediating access to models, tools, state, memory, credentials, policies, and external effects.
Quick Facts
| Specification | Official Specification |
|---|
How It Works
Execution lifecycle
The Runtime loads the pinned release configuration, resolves the principal and tenant, assembles bounded context, invokes a model adapter, receives an action proposal, applies policy, dispatches approved work, records observations, updates authoritative state, and evaluates stop conditions.
Runtime, Harness, and Workflow Engine
The Runtime is the executing substrate for run and step lifecycle. The Agent Harness is the broader control design around the model, including policies, context construction, evaluation, observability, and governance. A Workflow Engine provides durable process primitives such as history, timers, signals, retries, and task queues; an Agent Runtime may use one, but must still implement agent-specific model, tool, context, and policy adapters.
Run state and concurrency
A run needs stable run_id, release identity, principal, tenant, status, state version, budgets, active work, cancellation intent, and terminal reason. Use compare-and-set or an equivalent concurrency contract so two workers cannot silently advance the same run from one state version.
Separate conversation transcript, model context, durable execution state, long-term memory, and external effect records; they have different retention, privacy, consistency, and replay requirements.
Scheduling and cancellation
The scheduler enforces leases, heartbeats, queue priority, concurrency, backpressure, deadlines, retry classes, and graceful drain. Cancellation should propagate to model streams, tool calls, child runs, sandboxes, and waiting tasks where supported, but a cancellation acknowledgement does not prove an external write was rolled back. Worker processes can remain stateless while run state lives in a durable store.
Model and tool adapters
Model and tool adapters normalize provider-specific identifiers, streaming events, structured output, usage, errors, and cancellation without pretending all providers have identical semantics. Every tool call is a proposal. The Runtime resolves trusted identity and resource state, applies policy and approval, invokes with minimum credentials, validates and limits the result, and prevents untrusted output from changing authority.
Recovery and effect reconciliation
Checkpoints support recovery, but writes need stable operation keys and an Effect Journal with prepared, dispatched, committed, failed, or outcome_unknown. After a dispatch timeout or worker crash, reconcile with the downstream system before retrying. Persist approval and callback signals independently of a live process. Route in-flight runs to compatible code or migrate state explicitly when model, prompt, tool schema, policy, or runtime versions change.
Observability and release validation
Runtime events should correlate run, step, attempt, parent, model, tool, policy, operation, approval, effect, latency, usage, cost, cancellation, and terminal status. Redact secrets and unnecessary private payloads, and do not store hidden chain-of-thought.
Before release, validate tenant isolation, stale writes, duplicate delivery, crash-after-effect, provider fallback, poisoned results, cancellation races, budget exhaustion, schema migration, queue overload, and replay.
Key Characteristics
- Owns run and step lifecycle across create, schedule, execute, wait, resume, cancel, and terminate
- Separates durable run state, model context, memory, transcript, and effect records
- Mediates provider, model, tool, sandbox, storage, policy, and credential adapters
- Enforces tenant isolation, optimistic concurrency, leases, budgets, backpressure, and cancellation
- Recovers through checkpoints plus idempotent effect accounting and unknown-outcome reconciliation
- Emits versioned, redacted events for operations, evaluation, audit, and incident analysis
Common Use Cases
- Hosting many isolated customer-support or enterprise agent sessions
- Scheduling long-running coding and research agents across replaceable workers
- Mediating MCP, API, browser, shell, and code tools through one policy boundary
- Pausing runs for durable human approval and resuming on another worker
- Applying provider fallback while preserving model-specific result and usage semantics
- Streaming normalized lifecycle events to observability and evaluation systems
Example
Loading code...Frequently Asked Questions
How is an Agent Runtime different from an Agent Harness?
The Runtime executes and schedules runs and steps. The Harness is the broader control design around the model, covering context, policy, tools, evidence, evaluation, observability, and governance. A product may package both together.
Is an Agent Runtime the same as a workflow engine?
No. A workflow engine supplies durable process primitives. An Agent Runtime may build on it, but also owns agent-specific model and tool adapters, context assembly, policy mediation, budgets, result handling, and normalized events.
Can Agent Runtime workers be stateless?
Yes. Replaceable workers can load authoritative run state from durable storage, acquire a lease, execute a bounded step, and commit a versioned transition. The run is stateful even when each worker process is not.
Does cancelling a run undo its tool calls?
No. Cancellation prevents or interrupts future work where possible, but committed external effects remain. The runtime must record effect status and use reconciliation or explicit compensation when business recovery is required.
What should be tested before releasing an Agent Runtime?
Test tenant isolation, stale concurrent writes, duplicate delivery, crash-after-effect, unknown outcomes, cancellation races, provider failures, poisoned tool results, budget exhaustion, queue overload, state migration, and replay compatibility.