What is Chat Template?
Chat Template is a model-specific serialization contract that converts structured messages, roles, tool interactions, and multimodal content into the text or token sequence a chat model actually receives.
Quick Facts
| Specification | Official Specification |
|---|
How It Works
A Chat Template does not decide the instruction, retrieved context, or output schema; those application-level concerns belong to a Prompt Template. Instead, it renders structured messages into the exact control-token format used when a model was trained. The contract can include role markers, turn boundaries, beginning- and end-of-sequence tokens, tool definitions and results, multimodal placeholders, and the marker that starts an assistant response.
Implementations commonly store a Jinja template on a Tokenizer or multimodal Processor, but the syntax alone does not make a template compatible with a model. Training normally renders complete conversations without an extra generation prompt; inference may append a model-specific assistant-start marker, while prefill continues an existing assistant message. Loss masking is a separate training policy.
Production identity therefore binds the model, Tokenizer or Processor, template hash, Special Token map, message content schema, rendering mode, stop configuration, Tool Parser, and Runtime renderer. Golden fixtures should compare both rendered text and Token IDs across training and serving paths. A valid template is not an authorization boundary: applications must still validate roles and content shapes, prevent untrusted control-token injection, enforce tool permissions, and test safety behavior.
Key Characteristics
- Serializes role and content records into the model-specific control-token sequence learned during training
- Binds the model revision to a Tokenizer or Processor, template revision, Special Token map, content schema, and stop behavior
- Uses distinct rendering modes for complete training conversations, assistant generation starts, and assistant-message prefill
- Can render tool definitions, tool calls, tool responses, and multimodal placeholders when the model format supports them
- Requires text and Token ID fixtures to detect drift across Transformers, vLLM, llama.cpp, or another serving Runtime
- Does not replace Loss Mask policy, message validation, role authorization, tool permissions, or output validation
Common Use Cases
- Preparing conversational SFT records with the same message format used by the target model
- Rendering Chat API requests into model inputs while preserving role boundaries and stopping behavior
- Configuring model-specific tool calling with a compatible Chat Template and Tool Parser
- Serializing image, video, or typed content parts through a multimodal Processor
- Blocking releases when training and serving produce different rendered text or Token IDs for golden conversations
Example
Loading code...Frequently Asked Questions
How is a Chat Template different from a Prompt Template?
A Prompt Template assembles application instructions, variables, examples, retrieved context, and output constraints. A Chat Template takes the resulting structured messages and serializes them into the model-specific text or Token sequence. They can be used together, but they solve different problems: application prompt composition versus model input formatting.
Why must training and inference use the same Chat Template?
A chat model learns particular role markers, turn boundaries, Special Tokens, and end-of-sequence behavior. If serving renders a different sequence, the model may continue the wrong role, stop incorrectly, ignore tools, or lose quality. Keep the same template identity, use the correct training or generation mode, and compare rendered-text and Token ID fixtures on the real serving path.
What do add_generation_prompt and continue_final_message do?
add_generation_prompt commonly appends the control tokens that begin a new assistant response. continue_final_message instead leaves the final message open so the model can continue it, which is useful for prefill. Their exact behavior depends on the template, and incompatible modes should not be combined. Training complete conversations normally needs neither an extra assistant start nor prefill continuation.
Does a valid Chat Template guarantee tool calling works?
No. The template must encode tool definitions, calls, and results in the format expected by the model, while the serving Runtime needs a compatible Tool Parser and stop configuration. Parseable JSON only proves syntax; it does not prove that the model selected the correct tool, supplied valid arguments, respected permissions, or handled tool errors safely.
Can a Chat Template prevent prompt injection or unsafe role changes?
No. A template controls serialization, not trust. The application must validate allowed roles and content shapes, reject or escape reserved control tokens where appropriate, keep system and tool authority outside user control, enforce tool permissions, validate outputs, and run adversarial evaluations. Security research shows format mismatches can affect tested models, but it does not establish one universal failure rate or fix.