What is Structured Output?
Structured Output is an LLM inference capability that makes a response conform to a declared machine-readable contract, usually a provider-supported subset of JSON Schema, so application code can parse it predictably.
How It Works
Four mechanisms that are often confused
Prompt-only JSON asks the model to follow an example but provides no structural guarantee. JSON mode targets syntactically valid JSON without ensuring a particular schema. Schema-constrained output restricts generation to a provider-supported schema subset. Strict function or tool calling applies a schema to tool names and arguments, while a response-format schema controls the assistant's final data. Choose tool calling for actions and response formatting for typed results; do not treat these mechanisms as interchangeable.
How schema-constrained generation works
Providers commonly compile the supplied schema into a grammar or equivalent constraint and mask tokens that would make the partial response structurally invalid. This can add first-use compilation latency and may interact with prompt caching. Support is not full JSON Schema portability: accepted keywords, nesting, recursion, optional fields, enum sizes, and strictness rules vary by provider and model. Validate schemas during deployment and pin the provider, model, API version, and schema version used by each request.
Validation does not end at decoding
After checking the transport status, parse with unknown-field rejection when appropriate and validate the object against the application's canonical schema. Then enforce semantic and business invariants such as date ranges, currency allowlists, identifier ownership, cross-field consistency, evidence provenance, and authorization. Generate wire schemas from application types when SDK support permits, or add CI checks that prevent schema and code types from drifting.
Refusal, truncation, and incompatible input
A safety refusal or token-limit stop can take precedence over the requested schema and produce no usable object. Streaming consumers may see an incomplete prefix until generation finishes. If the source lacks a required value, the model may invent one merely to satisfy a required field unless the schema and prompt define nullable or explicit unknown states. Handle refusal and finish reasons before parsing, cap retries, preserve the original input and response metadata for diagnosis, and route ambiguous or high-risk records to review.
Key Characteristics
- Constrains responses to a declared machine-readable contract
- Provides stronger guarantees than prompt-only formatting or JSON mode
- Usually implements only a provider-specific subset of JSON Schema
- Does not guarantee factual accuracy, evidence quality, or business validity
- Requires explicit handling for refusals, truncation, and incompatible input
- Supports reliable extraction and orchestration when paired with application validation
Common Use Cases
- Extracting invoice or contract fields into typed application records
- Classifying support requests against an enumerated label contract
- Returning validated arguments for a function or agent tool
- Producing data for typed UI components and workflow routing
- Creating reviewable candidate records before database writes or automation
Example
Loading code...Frequently Asked Questions
How is structured output different from JSON mode?
JSON mode is intended to produce syntactically valid JSON but does not guarantee required keys, types, enums, or nesting. Schema-constrained structured output restricts generation to a supported schema. Even then, the application must handle refusal or truncation and validate semantic facts and business rules.
Does schema compliance mean the returned data is correct?
No. It proves structural conformance within the provider's supported schema subset, not truth. A valid date can still be extracted from the wrong sentence, an identifier can belong to another customer, and a cited claim can lack evidence. Validate provenance, cross-field consistency, authorization, and domain invariants separately.
What happens when the model refuses or reaches its token limit?
The provider may return a refusal indicator or a finish reason such as a token-limit stop instead of a schema-compliant object. Inspect response metadata before parsing, never execute a partial streamed object, use bounded retries only for recoverable failures, and route persistent or safety-related cases to an explicit fallback.
Can the same JSON Schema be used with every LLM provider?
Not reliably. Providers and models support different subsets, limits, optional-field conventions, and API shapes. Keep one canonical application type where possible, generate or adapt provider schemas explicitly, validate them in CI, and pin the provider, model, API version, and schema version in evaluation records.
Should structured output or function calling be used?
Use a response-format schema when the assistant's final answer should be typed data. Use strict function or tool calling when the model is selecting an application capability and supplying arguments. Agent workflows often need both: validated tool arguments during execution and a separate structured final response.