What is Human-in-the-Loop?
Human-in-the-Loop is a control pattern in which an AI system pauses at defined decision points so an authenticated, authorized person can review, approve, reject, edit, escalate, or halt proposed work before execution continues.
Quick Facts
| Specification | Official Specification |
|---|
How It Works
Separate Interrupt, Review, Approval, and Authorization
An interrupt pauses and persists a Run; a review presents one proposal to a person; an approval records a decision; authorization determines whether the actor, approver, Tenant, Resource, operation, and current Policy permit execution. Human-on-the-Loop monitoring, post-event audit, and data labeling remain useful but do not provide the same pre-effect gate. The model cannot exempt itself, and downstream services still enforce ownership and least privilege after approval.
Bind and Revalidate a Durable Pending Decision
Store the Run, Checkpoint, Call, actor, authenticated approver, Tenant, Resource, exact Tool and Schema Revision, canonical argument digest, Policy Revision, decision, issue time, expiry, and single-use state. Current [OpenAI Agents SDK guidance](https://openai.github.io/openai-agents-python/human_in_the_loop/) supports serializing paused state and warns that deserialization does not authenticate the snapshot or reviewer. Microsoft Agent Framework likewise persists pending requests with checkpoints. Resume only the same compatible Run, and revalidate authority, proposal identity, resource state, policy, schema, and expiry immediately before dispatch.
Handle Edits, Replay, Failure, and Reviewer Load
Approve, reject, edit, clarify, escalate, cancel, and emergency halt are distinct outcomes. An edit creates a new proposal and digest that must pass policy and any required approval again. Atomically consume an approval with creation of a stable operation or outbox record; after an ambiguous timeout, reconcile the effect instead of asking again and blindly retrying. Test bypass, cross-Tenant review, stale or changed arguments, expiry, concurrent decisions, cancellation, crashes, and reviewer outage. Measure coverage, false allow and block rates, queue age, decision latency, abandonment, escalation, duplicate effects, incidents, and reviewer fatigue.
Key Characteristics
- Enforced pause before a defined consequence, independent of model cooperation
- Authenticated reviewer and explicit separation between interrupt, approval, and authorization
- Proposal-bound decision covering actor, resource, arguments, schema, policy, expiry, and single use
- Durable pending state that can survive process loss and resume the same compatible run
- Approve, reject, edit, clarify, escalate, cancel, and emergency-halt paths
- Coverage, decision quality, latency, fatigue, bypass, and incident metrics
Common Use Cases
- Approving a refund, payment, account change, deployment, merge, deletion, or external message
- Correcting extracted contract or invoice fields before they enter a system of record
- Escalating novel, ambiguous, high-impact, or policy-sensitive cases to a domain expert
- Pausing a long-running agent and resuming it after an asynchronous review
- Separating the requester's authority from an independent reviewer for privileged operations
- Capturing reviewed examples for evaluation without treating approval as a training label by default
Example
Loading code...Frequently Asked Questions
Is Human-in-the-Loop the same as asking the model for confirmation?
No. A prompt can be ignored or manipulated. A real gate is enforced by trusted runtime or downstream code and prevents the sensitive operation from executing until a valid decision is supplied.
Does human approval replace authorization?
No. Approval records a review decision. Authorization must still verify the actor, approver authority, tenant, resource ownership, current policy, exact arguments, and least-privilege credential at execution time.
What happens when a reviewer edits a tool call?
The edit creates a new proposal and argument digest. Validate its schema and business rules, rerun policy, and obtain fresh approval when the changed operation still requires it. Never reuse approval for different arguments.
How should delayed approvals be resumed?
Persist the run and pending Approval Envelope, authenticate the reviewer, then resume the same compatible run. Before executing, recheck expiry, single-use status, policy and schema versions, resource state, and approver authority.
How is a Human-in-the-Loop system evaluated?
Test bypass, stale or replayed approvals, argument changes, unauthorized reviewers, timeouts, concurrent decisions, crashes, and cancellation. Measure coverage, false allows and blocks, latency, edits, escalations, abandonment, fatigue, and incidents.