← AI Terminology
Chat Template
A chat template is the model-specific string format that serialises roles (system/user/assistant/tool) and special tokens into the flat token sequence the model was trained on.
Wrong templates silently destroy open-model quality.
Wrong templates silently destroy open-model quality.
Why It Matters in AI
Open models expect exact control tokens and role markers from finetuning. Mismatched templates cause refusals, loops, or dumb answers. Hugging Face
apply_chat_template standardises this critical detail.Key Points
| Aspect | Description |
|---|---|
| Tools | HF apply_chat_template, vLLM/openai-compatible servers |
| Related | System prompts, special tokens |
| Includes | BOS/EOS, role headers, tool call syntax, generation prompt |
| Variants | ChatML, Llama-3 header format, Mistral instruct |
| Failure mode | Using Vicuna format on a ChatML model |
| Source of truth | Tokenizer chat_template (Jinja) in model card |
Simple Analogy
Each embassy expects forms in its own language and layout — same content, wrong form, application rejected.
Common Usage Examples
tokenizer.apply_chat_template(messages, add_generation_prompt=True)- Inspect
tokenizer.chat_template - vLLM serves the correct template per model
- Never hand-roll formats for instruct models
Summary
In short: A chat template is the model-specific packaging of roles and special tokens — get it wrong and even a strong open model looks broken.