Summary

This page covers AI agent runtime infrastructure, not electrical or mechanical harness engineering. Harness Engineering designs the control layer around a model: identities, allowed tools and resources, state transitions, budgets, approvals, observable events, evaluation, and recovery. The shorthand agent = model + harness is a useful decomposition, not an industry standard or a safety proof.

This article defines the discipline and its boundaries. The Agent Harness glossary owns the concise entity definition; the anatomy guide owns architecture; the practical guide owns implementation; and the evaluation guide owns testing.

Introduction: From "Prompts" to "Scaffolding"

Teams may move from exploratory Vibe Coding toward explicit Spec Coding, but a specification cannot enforce runtime authority. A model can still request an unauthorized tool, repeat an effect, exceed a budget, consume stale state, or produce malformed output.

Harness Engineering makes those runtime concerns explicit and testable. It does not make model behavior inherently safe, and it does not move authorization into the prompt.

What is Harness Engineering?

Harness Engineering is the discipline of building the runtime control layer around an AI agent. The model proposes content or actions; the harness authenticates the principal, filters available capabilities, validates requests, executes policy, records state and evidence, enforces budgets, and classifies completion or failure.

Some applications are useful without tool execution, and adding a harness does not automatically produce autonomy, correctness, or safety. The value is separation of responsibilities: probabilistic model output remains an untrusted proposal until deterministic or independently authorized controls permit an effect.

Model and Runtime Responsibilities

Core Concept: Agent = Model + Harness

The decomposition separates:

  1. Model: produces probabilistic messages, structured proposals, or tool-call arguments from supplied context.
  2. Harness: owns identity, authorized context, tool mediation, durable state, limits, approval, event capture, cancellation, and recovery.

The application and each downstream service must still enforce their own authorization and data invariants. A harness-side allowlist cannot compensate for an overprivileged tool server.

The decomposition is useful only when ownership is explicit:

Concern Model may propose Harness must control Downstream service must enforce
task plan steps and alternatives step budget, cancellation, checkpoint not applicable
context requested evidence authorized retrieval, size, provenance data access policy
tool call tool name and arguments capability exposure, validation, approval object and tenant authorization
side effect intended outcome idempotency key, timeout, commit evidence transactional invariant
completion self-reported answer terminal-state rules and independent checks authoritative final state

A model saying "done" is not completion evidence. The harness should derive completion from tests, downstream state, a verifier, or an explicit human decision.

The Three Paradigm Shifts

graph LR A["Prompting: influence model output"] --> B["Specification: define intended result"] B --> C["Harness: enforce runtime boundaries"]

These layers solve different problems. A prompt expresses intent; a specification turns intent into acceptance conditions; a harness controls how a run obtains evidence and produces effects. None replaces service-level access control or product governance.

Core Components of Harness Engineering

A production harness usually separates at least these concerns:

1. Guardrails

Policy enforcement resolves an authenticated principal, tenant, resource, operation, and purpose before execution. Input filtering can reduce noise, but prompt-injection detection is not an authorization boundary. Tool arguments require schema and semantic validation.

2. Memory Management

The harness distinguishes transient context, durable workflow state, and long-term memory. Retrieval must apply access control before content reaches the model. Checkpoints need versioning, retention, deletion, concurrency, and migration rules.

3. Tool Mediation and Execution

The tool layer exposes narrowly scoped operations, validates arguments, enforces timeouts and idempotency, bounds results, redacts secrets, and records committed side effects. Sandboxing can reduce blast radius but does not replace authorization or host hardening.

4. Budgets, Approval, Evidence, and Recovery

Step, time, token, retry, byte, concurrency, and spend limits must be outside model control. High-impact actions can pause for a durable approval bound to the exact proposed effect. Observable events support evaluation and incident response; retries need idempotency and unknown-outcome handling.

5. Independent Evaluation

The runtime should not accept the generating model's confidence or self-review as sufficient proof. Deterministic tests, schema checks, policy engines, separate evaluators, or human review should decide whether a step passes. Anthropic's long-running Agent guidance separates planner, generator, and evaluator roles partly so evaluation can survive context resets and avoid inheriting every assumption from generation.

An independent evaluator can still be probabilistic and wrong. Record its model, rubric, evidence, and disagreement; deterministic safety and authorization invariants remain hard gates.

The Agent Harness Control Loop

A useful control loop has six phases:

  1. Admit: authenticate the principal, load policy, assign a run ID, and reject impossible or unauthorized goals.
  2. Assemble: retrieve only permitted context, attach provenance, reserve output headroom, and expose task-scoped tools.
  3. Propose: let the model produce a message, plan update, or typed action request.
  4. Decide and execute: validate the request, obtain bound approval if needed, call the tool with an idempotency key, and capture the authoritative outcome.
  5. Evaluate: compare the state with acceptance criteria and decide whether to continue, retry, escalate, or terminate.
  6. Checkpoint: persist a bounded state snapshot and recovery cursor without treating generated prose as authoritative state.

The loop needs explicit terminal states. succeeded, failed, cancelled, expired, escalated, and outcome_unknown are operationally different. Collapsing them into "completed" makes retries and incident response unsafe.

This standard-library Go fragment shows the policy boundary around a proposed step:

go 
package main

import (
	"errors"
	"fmt"
)

type Run struct {
	Tenant          string
	RemainingStep   int
	AllowedTools    map[string]bool
	IdempotentTools map[string]bool
}

type Proposal struct {
	Tenant       string
	Tool         string
	EffectDigest string
}

type Approval struct {
	EffectDigest string
	Granted      bool
}

func admit(run *Run, proposal Proposal, approval Approval) error {
	if run.RemainingStep <= 0 {
		return errors.New("step budget exhausted")
	}
	if run.Tenant == "" || run.Tenant != proposal.Tenant {
		return errors.New("tenant denied")
	}
	if !run.AllowedTools[proposal.Tool] {
		return errors.New("tool denied")
	}
	if !run.IdempotentTools[proposal.Tool] {
		return errors.New("tool is not retry-safe")
	}
	if !approval.Granted || approval.EffectDigest != proposal.EffectDigest {
		return errors.New("approval does not match effect")
	}
	run.RemainingStep--
	return nil
}

func main() {
	run := Run{
		Tenant:          "tenant-a",
		RemainingStep:   2,
		AllowedTools:    map[string]bool{"invoice.read": true},
		IdempotentTools: map[string]bool{"invoice.read": true},
	}
	proposal := Proposal{
		Tenant:       "tenant-a",
		Tool:         "invoice.refund",
		EffectDigest: "sha256:refund-request",
	}
	approval := Approval{EffectDigest: "sha256:refund-request", Granted: true}
	if err := admit(&run, proposal, approval); err != nil {
		fmt.Println("blocked:", err)
		return
	}
	fmt.Println("admitted")
}

The example intentionally does not call a model or tool. It demonstrates the contract: generated arguments stay inert until trusted code admits the exact effect.

Harness vs. Adjacent Disciplines

Discipline Primary question Does not replace
Prompt Engineering How should the model interpret and answer this request? authorization, state, recovery
Agent orchestration Which component or worker runs next? policy and business invariants
Harness Engineering Under what authority, limits, evidence, and recovery rules may this run proceed? downstream service enforcement
DevOps and SRE How is the service built, deployed, observed, and restored? per-step Agent decisions
Application security Which identities may access which resources and effects? model evaluation

Harness Engineering overlaps all of them but has a distinct object: a model-driven run whose next action is probabilistic and whose state may outlive one context window.

Why Tool-Using Agents Need Runtime Controls

With the rise of MCP (Model Context Protocol), AI has gained unified interfaces to local files, databases, and external APIs. This power brings massive risks.

MCP standardizes how clients and servers describe and invoke capabilities; it does not automatically establish end-user authorization, tenant isolation, least privilege, or safe side effects. A harness must bind protocol calls to application identity and policy, while each server independently enforces access.

The same rule applies to framework-level sandboxes, browser profiles, code containers, and cloud credentials. Isolation can reduce blast radius, but credentials inside the sandbox still determine what the process can reach. Start with no authority, then grant the smallest capability needed for one task and one lifetime.

Failure Modes to Design Before Production

Context drift

A long run may compact away a constraint, reuse stale policy, or inherit poisoned memory. Version context, retain source references, revalidate authority after reset, and fail when required evidence is missing.

Duplicate or unknown effects

A timeout can happen after a downstream service commits. Use idempotency keys, durable attempt records, and reconciliation before retry. "No response" is not the same as "no effect."

Approval drift

A user may approve one recipient or amount while the model later changes the arguments. Hash or otherwise bind approval to the complete canonical request and expire it when anything changes.

Evaluator coupling

A generator and evaluator sharing the same context and assumptions may agree on the same wrong answer. Use deterministic checks where possible, isolate evaluator evidence, and track human disagreement.

Unbounded autonomy

Without limits, a recoverable error becomes a loop that spends money, floods a service, or repeatedly modifies state. Enforce step, time, retry, token, byte, concurrency, and spend budgets outside model control.

How to Evaluate a Harness

Evaluate the complete run, not only final prose:

  • task success and acceptance-criteria coverage;
  • unauthorized reads, writes, destinations, and cross-tenant attempts;
  • duplicate effects and unknown outcomes;
  • recovery after tool, model, network, and state-store failures;
  • approval binding and cancellation behavior;
  • trace completeness and secret exposure;
  • p50, p95, and p99 latency;
  • cost per successful task and human correction time.

Use representative normal, boundary, adversarial, long-running, and failure-injection cases. Keep model capability evaluation separate from harness evaluation: a stronger model can hide a weak harness on ordinary tasks, while the control system still fails under adversarial or partial-failure conditions.

Conclusion

Harness Engineering turns implicit agent glue into an explicit runtime control system. It can reduce and expose risk, but it cannot make an agent perfectly safe or self-correcting. Start from the effects the system may produce, assign enforcement ownership, test failure and recovery paths, and grant authority incrementally.

Primary Sources