What is ChatModel?
Updated on
ChatModel is a framework-level adapter interface that accepts structured conversational requests and returns normalized model messages, streams, metadata, tool proposals, or errors.
How It Works
Request and response contract
The request should identify the provider and model revision, ordered role-tagged messages, multimodal parts, tool schemas, output schema, generation options, timeout, and trace context. The response should preserve content parts, tool-call IDs and arguments, refusal or safety state, finish reason, usage, provider request ID, and raw extension data when needed for diagnosis. Do not flatten all messages into one string or discard metadata that controls the next turn.
Capability negotiation
Tool calling, strict structured output, images, audio, reasoning blocks, log probabilities, server-side tools, and streaming are independent capabilities. A ChatModel should advertise or validate support before dispatch and return a typed unsupported-capability error. Generic options such as temperature or maximum output tokens still have provider-specific ranges and semantics; unsupported fields must not be silently ignored when correctness depends on them.
Streaming, cancellation, and retries
A stream is an ordered sequence of partial content, tool-call deltas, usage updates, errors, and one terminal state. Consumers must assemble chunks by documented identity rules, stop on cancellation, close readers, and never execute incomplete tool arguments. Retry only failures known to be transient and safe. A request that might have produced a billed result or side effect needs an idempotency or reconciliation policy rather than blind replay.
Testing and provider migration
Use deterministic fake implementations to test orchestration and recorded provider fixtures to test normalization, streaming assembly, errors, tool calls, refusals, and finish reasons. Then run behavioral evaluations against the exact candidate provider and model because a shared interface does not equal shared model behavior. Keep provider-specific extensions accessible behind an explicit escape hatch and version the adapter with the release contract.
Key Characteristics
- Accepts ordered typed messages rather than only a concatenated prompt string
- Normalizes model responses while preserving provider-specific diagnostic metadata
- Validates capabilities such as tools, schemas, multimodality, reasoning, and streaming
- Propagates context cancellation, deadlines, typed errors, finish states, and usage
- Supports deterministic fakes and recorded fixtures for orchestration tests
- Exposes provider extensions explicitly because the abstraction is intentionally leaky
Common Use Cases
- Injecting a model boundary into a chain, graph, workflow, or agent runtime
- Testing tool and structured-output orchestration with deterministic fakes
- Recording normalized usage, latency, request IDs, finish reasons, and errors
- Comparing cloud and local model adapters behind one application-owned interface
- Migrating providers through capability and behavioral regression tests
Example
Loading code...Frequently Asked Questions
Is ChatModel the same as an LLM?
No. An LLM is the learned model or deployed model service. ChatModel is a framework adapter that maps typed messages and options to that service and normalizes its response. The same adapter interface can wrap several providers, while the same underlying model can be exposed through different adapters.
Does a ChatModel make providers interchangeable?
Only at the interface level. Providers differ in role hierarchy, tools, schemas, multimodal parts, reasoning data, safety responses, token accounting, streaming, errors, and model behavior. Use capability checks and provider-specific escape hatches, then rerun behavioral evaluations before switching.
What should a production ChatModel response preserve?
Preserve typed content parts, tool-call identities and arguments, refusal or safety state, finish reason, usage, provider request ID, and relevant raw metadata. Dropping these fields can break the next tool turn, hide truncation, misstate cost, or make an incident impossible to reproduce.
How should ChatModel streaming be handled?
Treat the stream as ordered partial events with one terminal state. Assemble text and tool-call deltas according to stable IDs, propagate cancellation, close the stream, record mid-stream errors, and never execute partial tool arguments. A fallback after partial output also needs an explicit user-visible policy.
How should ChatModel adapters be tested?
Use deterministic fakes for application control flow and recorded fixtures for request translation, response normalization, streaming, refusals, usage, and errors. Add contract tests for every provider adapter and behavioral regression tests for the exact model release; interface compatibility cannot prove output equivalence.