An Agent Harness is the runtime control layer that turns model output into bounded, observable work. It does not make a model authoritative. It decides which context the model receives, which proposed actions are valid, whose credentials apply, how side effects execute, what state is committed, and whether a failed run may continue.

This page covers AI agent runtime infrastructure, not electrical wiring harnesses, test-harness hardware, or mechanical harness engineering.

This page owns the architecture and data flow. For adjacent intents:

Direct Answer

There is no single standardized Agent Harness product or mandatory component list. A team may implement the architecture with application code, an agent SDK, a state machine, queues, policy services, isolated workers, and an event pipeline.

The useful invariant is ownership:

The model proposes. The harness authorizes, executes, records, and recovers.

A prompt or tool-calling API alone is not a production harness. A harness must preserve the boundary between probabilistic proposals and authoritative effects.

Agent Harness vs. Loop, Runtime, and Workflow Engine

These terms overlap across vendors. Define the contract in your system instead of assuming the label carries a universal meaning.

Concept Primary responsibility Who chooses the next step? What it does not prove
Model Produces text, structured output, or tool proposals from supplied context Model inference Permission, execution, persistence, or correctness
Agent loop Repeats model decisions, observations, and actions until a terminal condition Usually the model within runtime limits Durable state, isolation, or safe side effects
Workflow engine Executes a predefined graph, state machine, or schedule Application code or declared graph That model-selected actions are authorized
Agent Runtime Hosts and schedules one or more agent sessions Runtime plus the configured loop or workflow A specific policy, evidence, or recovery design
Agent Harness Mediates context, capabilities, policy, state, effects, evidence, and recovery around a run Model proposes; deterministic controls decide what may execute Perfect safety or correct model reasoning

Anthropic's engineering guidance makes a related distinction: workflows follow predefined code paths, while agents dynamically direct their process and tool use. Both can run inside a harness. The more autonomy the model receives, the more explicit the harness contract must become.

The Four Architectural Planes

A production design is easier to reason about when responsibilities are grouped into four planes.

1. Control Plane

The control plane owns decisions that must not be delegated to model text:

  • Identity, tenant, purpose, and resource context
  • Capability allowlists and tool schema versions
  • Policy evaluation and approval requirements
  • Step, wall-clock, token, spend, retry, and byte budgets
  • Cancellation, termination, and escalation rules

The model may recommend an action. It must not grant itself a capability, widen a credential, or mark its own action approved.

2. Execution Plane

The execution plane turns an authorized proposal into a bounded operation:

  • Model and provider adapters
  • Tool registry and argument validation
  • Sandboxed or isolated workers where the workload requires them
  • Per-tool timeouts, concurrency limits, and output limits
  • Stable operation IDs and downstream idempotency keys

A sandbox reduces blast radius; it does not replace authorization. A process isolated from the host can still misuse an API credential or send an externally visible request.

3. State Plane

The state plane is the source of truth for what the run knows and what the system has committed:

  • Run, step, tool-call, and approval identifiers
  • Optimistically versioned run state
  • Message, artifact, and result references
  • Checkpoints and pending Human-in-the-Loop decisions
  • Budget counters and terminal status
  • Side-effect status: not_started, in_flight, committed, failed, or unknown

Thread checkpoints and long-term memory are different data products. LangGraph, for example, separates checkpointers for thread-scoped graph state from stores for cross-thread application data. Mixing them makes retention, deletion, authorization, and recovery harder to define.

4. Evidence Plane

The evidence plane explains the run without treating private reasoning as ground truth:

  • Structured lifecycle events and distributed traces
  • Policy and approval decisions
  • Tool request metadata and bounded results
  • State transitions, timings, errors, and budget use
  • Business outcomes and evaluation labels

OpenTelemetry's GenAI semantic conventions are evolving and can help with portable spans, metrics, and events. Keep an application-owned event contract as the stable source, then map it to the current telemetry convention.

Runtime Components and Ownership

The four planes become concrete through these components:

Component Owns Critical invariant
Request gateway Authentication, tenant, request ID, input limits Identity is resolved before capabilities are selected
Run coordinator State machine, leases, budgets, cancellation One versioned transition wins for each state update
Context builder Instructions, messages, retrieved data, artifact references Untrusted content stays data, not policy
Model gateway Provider request, structured response, model metadata Model output is a proposal, never an authorization
Tool registry Versioned schemas, risk class, executor binding Only registered capabilities are callable
Policy and approval service Allow, deny, redact, or suspend decisions Approval is bound to actor, arguments, resource, and expiry
Tool executor Isolation, timeout, operation key, result classification A retry cannot silently duplicate a committed effect
State and checkpoint store Versioned snapshots, decisions, effect journal Resume starts from committed evidence, not model memory
Event and evaluation sink Redacted events, traces, outcomes, test fixtures Telemetry supports diagnosis without becoming a secret dump

These can be modules in one process or independent services. Distribution does not create correctness by itself; it adds network failures and consistency decisions that the contract must cover.

End-to-End Data Flow

sequenceDiagram participant C as Client participant H as Harness Control Plane participant S as State Store participant M as Model Gateway participant P as Policy and Approval participant X as Tool Executor participant E as Event Sink C->>H: Request plus authenticated identity H->>S: Create run at state version 1 H->>M: Bounded context and allowed tool schemas M-->>H: Final response or tool proposal H->>H: Validate schema, budgets, and current state H->>P: Evaluate actor, resource, purpose, and effect alt Approval required P-->>H: Suspend with approval ID H->>S: Commit pending approval checkpoint else Allowed P-->>H: Allow with scoped decision H->>X: Execute call ID plus operation key X-->>H: Result plus effect status H->>S: Compare-and-commit next state version else Denied P-->>H: Deny with reason code H->>S: Commit denied or terminal state end H->>E: Emit redacted lifecycle events H-->>C: Stream progress, suspension, or final outcome

The key boundary occurs between proposal and execution. Schema validation answers "is this well formed?" Policy answers "may this actor perform this operation on this resource for this purpose?" Approval answers "did the designated human authorize this exact effect?" They are separate checks.

A Versioned Run Record

A run record should make concurrency, approvals, and side effects explicit. This is an illustrative application contract, not a universal standard:

json 
{
  "runId": "run_01J...",
  "stateVersion": 12,
  "status": "waiting_for_tool",
  "stepId": "step_07",
  "actor": {
    "tenantId": "tenant_acme",
    "principalId": "user_42",
    "purpose": "resolve_support_case"
  },
  "proposal": {
    "model": "provider/model-version",
    "tool": "create_support_ticket",
    "toolVersion": "3",
    "callId": "call_09"
  },
  "policy": {
    "decision": "allow",
    "approvalId": null
  },
  "effect": {
    "operationKey": "run_01J:call_09:create_support_ticket:3",
    "status": "in_flight"
  },
  "budgets": {
    "stepsUsed": 7,
    "stepsLimit": 12
  }
}

Do not use a mutable prompt, timestamp, or argument hash alone as the operation identity. Generate a stable semantic operation ID before dispatch and persist it. If the downstream API supports idempotency, send that key and retain the returned resource ID. If it does not, define a lookup or reconciliation path.

State Transitions, Not an Unbounded while Loop

A five-line while loop is useful for teaching tool calling, but it hides the production contract. Model the lifecycle explicitly:

text 
created
  -> running
  -> waiting_for_approval
  -> waiting_for_tool
  -> running
  -> completed | failed | cancelled | needs_reconciliation

Every transition should specify:

  1. Expected state version
  2. Triggering event
  3. Policy and budget preconditions
  4. State mutation
  5. Emitted event
  6. Whether an external effect may already exist

Use compare-and-set or an equivalent transaction when committing a transition. A lease can reduce concurrent work, but the state version remains necessary because leases expire and workers crash.

Failure and Recovery Semantics

Retries are safe only when the system knows whether an effect occurred.

Failure point Effect knowledge Default response
Model request fails before a proposal No external effect Retry within model and run budgets
Proposal fails schema or policy validation No authorized effect Reject, ask for correction, or terminate
Tool dispatch fails before acceptance Usually no effect, but verify transport contract Bounded retry with the same operation ID
Tool times out after dispatch Unknown Query operation status or enter reconciliation; do not blindly retry
Tool commits but state write conflicts Effect may be committed Read by operation ID, then commit the known result
Approval expires or is denied No approved effect Cancel the proposal or return to planning
Policy changes while a run is suspended Previous decision may be stale Re-evaluate before resuming
Worker is cancelled during execution Tool-specific Attempt cooperative cancellation and classify the final effect

unknown is a valid operational state. Collapsing it into failed creates duplicate side effects; collapsing it into completed hides missing work.

Trust and Authorization Boundaries

Treat every boundary crossing as untrusted until a deterministic control validates it:

  • User, retrieved, and tool content: may contain prompt injection. Keep it separate from system policy and tool definitions.
  • Model proposals: validate types, ranges, resource identifiers, and business rules. Structured output improves parsing, not truth or permission.
  • Tool results: cap size, validate schemas, label provenance, and prevent returned text from silently changing policy.
  • Credentials: bind them to the actor, audience, scope, and operation. The MCP security guidance explicitly warns against token passthrough and confused-deputy designs.
  • Approvals: show the concrete resource and effect. Bind the approval to immutable arguments and expire it.
  • Telemetry: redact secrets and personal data before export. Restrict trace access and deletion separately from production state.

OWASP describes excessive agency as excessive functionality, permissions, or autonomy. Capability reduction is therefore architectural: expose the smallest tool surface, grant the narrowest credential, and require human review when impact or reversibility demands it.

Budgets and Termination

maxSteps is necessary but insufficient. Define independent limits for:

  • Wall-clock duration
  • Model requests and tokens
  • Tool calls and retries by risk class
  • Concurrent work
  • Input, output, and artifact bytes
  • Spend where reliable usage data exists
  • Approval wait time

Termination should be deterministic. Stop on a verified business outcome, explicit refusal, policy denial, cancellation, budget exhaustion, unrecoverable error, or a state that requires reconciliation. "The model said it was done" is only one proposal to evaluate.

What to Trace and Evaluate

Capture observable evidence, not hidden chain-of-thought:

  • Request, run, step, call, approval, and operation IDs
  • Model, prompt template, tool schema, policy, and executor versions
  • State version before and after each transition
  • Proposal class, policy result, approval result, and effect status
  • Latency, retry, token, and cost fields when available
  • Redacted error codes and bounded result metadata
  • Final business outcome and deterministic checks

For design details, see Agent Observability. For release testing, the Agent Harness evaluation guide covers scenario fixtures, tool doubles, fault injection, trajectory checks, and judge-assisted scoring.

Production Readiness Checklist

  • [ ] Authenticated identity and tenant context are resolved before tool selection.
  • [ ] Tool schemas, risk classes, and executor versions are registered.
  • [ ] Model output is validated as an untrusted proposal.
  • [ ] Policy and approval decisions are enforced outside the model.
  • [ ] Credentials are audience- and scope-bound; tokens are not passed through blindly.
  • [ ] Run state is versioned and checkpoints can survive a worker restart.
  • [ ] Side-effecting operations use stable IDs, idempotency, or reconciliation.
  • [ ] Time, step, token, retry, byte, concurrency, and spend limits are explicit.
  • [ ] Cancellation and unknown outcomes have documented transitions.
  • [ ] Events are structured, redacted, access-controlled, and retention-limited.
  • [ ] Tests cover duplicate dispatch, timeout after commit, stale approval, policy change, restart, and cancellation.
  • [ ] A human can inspect evidence and recover or terminate a suspended run.

Sources and Further Reading

Conclusion

Agent Harness architecture is not a larger prompt or a branded framework. It is the contract that separates model proposals from authorized effects.

The durable design is explicit about identity, policy, state versions, operation IDs, approvals, budgets, observable events, and unknown outcomes. Once those boundaries exist, model providers and orchestration libraries can change without erasing the system's operational guarantees.