← AI Terminology
Instruction Tuning
Instruction tuning (also called supervised fine-tuning for instruction following) is the process of fine-tuning a pre-trained language model on a dataset of (instruction, response) pairs — teaching it to follow natural language commands rather than just complete text.
It is the first step that converts a raw pre-trained LLM into a useful assistant.
It is the first step that converts a raw pre-trained LLM into a useful assistant.
Why It Matters in AI
A raw pre-trained LLM continues text; it does not follow instructions. GPT-3 would complete "Write a poem about the sea" with more instructions, not a poem. Instruction tuning flips this: the model learns that an instruction is a command, not text to continue. FLAN, Alpaca, InstructGPT, and all modern chat models are instruction-tuned. It is the lowest-cost, highest-impact fine-tuning step — often achievable with thousands of examples.
Key Points
| Aspect | Description |
|---|---|
| SFT | Supervised Fine-Tuning is the technical name — cross-entropy loss on response tokens only |
| FLAN | Fine-Tuned Language Net: instruction tuned across 1,000+ tasks — proved massive generalisation |
| Scale | 1,000–100,000 high-quality examples is typically enough; quality > quantity |
| After SFT | Often followed by RLHF/DPO alignment to improve safety and preference alignment |
| Data format | (instruction, [input,] response) triples — e.g. {"instruction": "Summarise:", "input": article, "output": summary} |
| Key datasets | FLAN (Google, 1800 tasks), Alpaca (52K GPT-4 generated), Dolly, ShareGPT, Open-Hermes |
Simple Analogy
A brilliant but socially oblivious genius (pre-trained LLM) who knows everything but answers any question by rambling about related topics. Instruction tuning is etiquette school: they learn that when someone asks a question, they should answer it — and when asked to write a poem, they should write a poem, not discuss the history of poetry.
Common Usage Examples
trl.SFTTrainer(model, train_dataset=alpaca_dataset, formatting_func=format_instruct_sample)- Alpaca format:
{"instruction": "...", "input": "...", "output": "..."}— standard instruction tuning schema --data_path data/alpaca_data.jsonin Stanford Alpaca fine-tuning script- Open-Hermes 2.5: high-quality instruction dataset used to fine-tune Mistral-7B to near-GPT-4 quality
chat_templatein HuggingFace tokenizer: formats conversation history into instruction-tuning prompt format
Summary
In short: Instruction tuning teaches a pre-trained model to follow commands rather than just complete text — the transformation that turns a language model into a useful assistant.