← AI Terminology
Fine-Tuning
Fine-tuning is the process of taking a pre-trained model and continuing to train it on a smaller, task-specific dataset — updating some or all of its weights to specialise it for a particular application while retaining the knowledge learned during pre-training.
It is the dominant paradigm for deploying LLMs in production applications.
It is the dominant paradigm for deploying LLMs in production applications.
Why It Matters in AI
Pre-training on vast data gives a model broad knowledge; fine-tuning focuses that knowledge on your specific use case — a medical chatbot, a legal document classifier, a coding assistant. Fine-tuning achieves much better task performance than prompting alone and requires far less compute than pre-training from scratch. The ecosystem of fine-tuning techniques (LoRA, QLoRA, SFT, RLHF) is one of the most commercially active areas of applied AI.
Key Points
| Aspect | Description |
|---|---|
| SFT | Supervised Fine-Tuning: train on (instruction, response) pairs — the standard first stage |
| PEFT | Parameter-Efficient Fine-Tuning (LoRA, adapters) — update small fraction of weights, much cheaper |
| RLHF | Second stage: align model with human preferences using reward model + PPO |
| Full fine-tuning | Update all model weights — best performance, highest compute, risk of catastrophic forgetting |
| Data requirements | SFT: as few as 100–1,000 high-quality examples can work well for instruction following |
| Catastrophic forget | Fine-tuning on narrow data causes the model to "forget" general knowledge — regularisation helps |
Simple Analogy
A generalist doctor who specialises in cardiology: their medical school (pre-training) gave them broad knowledge; their cardiology residency (fine-tuning) focused that knowledge on heart disease. They're now expert in their specialty while retaining general medical knowledge — unless the residency was so narrow they forgot everything else (catastrophic forgetting).
Common Usage Examples
trl.SFTTrainer(model, train_dataset=instruct_data, peft_config=lora_config)— LoRA SFT- HuggingFace PEFT:
get_peft_model(model, LoraConfig(r=16, target_modules=["q_proj","v_proj"])) - Instruction fine-tuning: fine-tune Llama-3 on 50K (instruction, response) pairs from FLAN or Alpaca
- Domain adaptation: fine-tune BERT on PubMed abstracts for biomedical NLP tasks
- OpenAI fine-tuning API:
openai.fine_tuning.jobs.create(model="gpt-4o-mini", training_file=file_id)
Summary
In short: Fine-tuning specialises a pre-trained model for your task by continuing training on task-specific data — the fastest and most cost-effective path from a general model to a production application.