What is JSON Mode?
JSON Mode is a provider-specific LLM output setting that makes a normally completed response syntactically valid JSON without guaranteeing a requested schema, factual accuracy, authorization, or business validity.
Quick Facts
| Specification | Official Specification |
|---|
How It Works
JSON Mode is an output contract exposed by some model APIs and serving runtimes, not a universal protocol or one standardized parameter. In OpenAI-style APIs, json_object requests a JSON response; Ollama exposes format: "json" and accepts a Schema object for stricter output. Google Gemini combines an application/json response MIME type with a response JSON Schema, while Anthropic exposes Schema-constrained output through output_config.format; neither contract inherits OpenAI parameter names. Support can vary by model, endpoint, API version, SDK, deployment platform, and streaming path, so an OpenAI-compatible URL does not by itself prove equivalent behavior.
The useful mental model is a guarantee ladder. A prompt that says "return JSON" is only an instruction. JSON Mode moves syntax validity into the provider or runtime contract, so a normally completed response should be parseable JSON, but {}, an unexpected array, renamed fields, wrong types, invented values, and unsafe strings may all remain valid JSON. Strict Structured Output adds conformance to a supported JSON Schema subset. JSON Schema defines the data contract, while Constrained Decoding is one possible enforcement mechanism; neither term is a synonym for JSON Mode. Function Calling has a different intent: it proposes a named operation and arguments, and those arguments may have their own strictness setting.
The guarantee is conditional on successful completion. A request rejected before generation, a safety refusal, content filtering, output-token exhaustion, context exhaustion, timeout, transport interruption, or provider fallback may return no JSON, an out-of-band result, or an incomplete document. OpenAI and Azure also require an explicit JSON instruction when using json_object; without it, the request can fail or generation can consume its budget as whitespace. That prompt rule is provider-specific, not part of JSON itself. During streaming, chunks are fragments rather than independent documents: buffer or use the SDK accumulator, wait for the terminal event, inspect its status or finish reason, and only then parse.
A production consumer should apply gates in order: verify the HTTP and provider status; select the documented content field; enforce byte, depth, and collection-size limits; parse with a standard JSON parser; validate against an application-owned, versioned Schema; check domain invariants, evidence, freshness, and authorization; then permit any side effect. RFC 8259 allows edge cases that parsers handle differently, including duplicate object names and numbers outside commonly interoperable precision, so security-sensitive ingestion may need stricter duplicate-key and numeric policies. A validator placed after JSON Mode catches shape errors, but it does not retroactively make generation schema-constrained.
Strict Structured Output is stronger but still not a truth or safety guarantee. Providers implement different JSON Schema subsets and may reject, simplify, or transform unsupported keywords. A schema-valid object can contain fabricated facts, unsafe URLs, stale identifiers, unauthorized actions, or strings carrying injection content. Keep the application schema as the source of truth, test it against the exact backend, represent missing evidence explicitly with nullable or status fields, and validate semantic and authorization rules in deterministic code.
Retries need an explicit policy. Classify transport, incomplete-generation, parse, schema, and semantic failures separately; bound attempts with backoff and jitter; avoid retrying permanent schema or policy errors; and preserve trace and idempotency keys. Do not execute a tool, charge a payment, write a database record, or send a message until the entire response passes acceptance. A repair prompt is another probabilistic model call, not validation, and can multiply cost or duplicate effects if the original request already committed work.
Treat every parsed value as untrusted input. Never pass fields directly to eval, a shell, SQL, templates, file paths, URLs, or tool dispatch. Apply allowlists, escaping, length limits, destination controls, least privilege, and human approval where impact requires it. Redact credentials, personal data, prompts, and raw payloads from logs. Observability should record provider, endpoint and API revision, immutable model identity, requested output mode, Prompt or Template revision, expected Schema revision, terminal reason, parse/schema/business-rule outcomes, retry count, latency, token use, and final effect status.
Use JSON Mode when the shape is intentionally flexible or when a target model lacks strict schema support and application-side validation is acceptable. Prefer Strict Structured Output when downstream code expects a known shape, Function Calling when the model is proposing an operation, and ordinary text when a human-readable answer is the product. Maintain a tested capability matrix and fail closed instead of silently dropping from schema enforcement to syntax-only JSON.
Key Characteristics
- Provider-scoped contract - parameter names, supported models, endpoints, schema behavior, and streaming semantics are not portable by name alone
- Syntax-level guarantee - a normally completed response is parseable JSON, but its root shape, fields, types, values, and meaning may still be wrong
- Completion-dependent result - refusals, filtering, truncation, timeouts, transport failures, and fallbacks remain separate outcomes
- Distinct from strict schemas - Structured Output can enforce a supported JSON Schema subset, while JSON Mode alone cannot
- Whole-response parsing - streamed fragments must be accumulated and terminal status checked before parsing and validation
- Application-owned acceptance - schema, domain, evidence, authorization, security, idempotency, and observability controls remain outside the mode
Common Use Cases
- Compatibility fallback - obtain parseable JSON from a model or endpoint that does not support the required strict Schema
- Exploratory extraction - stabilize a changing object shape while application validation and failure telemetry remain mandatory
- Open-ended JSON maps - return dynamic keys or heterogeneous metadata that cannot be represented by a fixed provider Schema subset
- Provider adapters - normalize different JSON-output settings behind an explicit capability and downgrade policy
- Reliability evaluation - measure completion, parse, Schema, semantic, retry, latency, and duplicate-effect outcomes separately
Example
Loading code...Frequently Asked Questions
Does JSON Mode always guarantee valid JSON?
Only within the documented provider contract and after a normal completion. A rejected request, refusal, content filter, token-limit truncation, timeout, interrupted stream, or unsupported fallback can produce no JSON, partial JSON, or a separate result. Check the terminal status or finish reason before parsing, and test the exact model, endpoint, API version, SDK, and streaming path.
How is JSON Mode different from Strict Structured Output?
JSON Mode guarantees syntax, so any allowed JSON object may pass even when keys or types are wrong. Strict Structured Output constrains a completed response to a provider-supported JSON Schema subset. Neither guarantees factual accuracy, authorization, safety, or business validity, and both still require application checks.
Is JSON Mode the same as Function Calling?
No. JSON Mode controls the representation of a model response. Function Calling or Tool Use represents a proposal to invoke a named capability with arguments and requires dispatch, authorization, execution, and result handling. Some providers apply JSON or strict Schema constraints to tool arguments, but that does not make the two features equivalent.
Is JSON Mode portable across LLM providers?
No common standard defines its request field or guarantee. OpenAI-style APIs use `json_object`, Ollama uses `format: "json"`, and other providers expose schema-oriented controls under different names. Even compatible endpoints can differ by model, API revision, prompt requirement, streaming behavior, and fallback, so maintain a tested capability matrix.
How should production code validate JSON Mode output?
First require a successful terminal status, then enforce size limits, parse once with a standard JSON parser, validate an application-owned versioned Schema, check domain invariants and evidence, authorize any action, and only then commit side effects. Bound retries, preserve idempotency, reject unsafe values, and record parse, Schema, semantic, and effect outcomes separately.