← AI Terminology
Prompt Tuning
Prompt tuning learns a small set of continuous embeddings (soft prompts) prepended to inputs, keeping the entire pretrained model frozen.
Simpler than deep prefix tuning; effective at scale (Lester et al.).
Simpler than deep prefix tuning; effective at scale (Lester et al.).
Why It Matters in AI
As models grow, soft prompts alone can match full fine-tuning on many tasks. Prompt tuning enables huge multi-task serving: one backbone, many tiny prompt vectors.
Key Points
| Aspect | Description |
|---|---|
| Limit | May lag LoRA on hard domain shifts |
| Paper | Lester et al., 2021 — better with larger models |
| Params | Only soft prompt embeddings (thousands to millions) |
| Related | Prefix tuning, P-Tuning v2, PEFT |
| Serving | Swap prompt embeddings per request/task |
| Vs discrete | Learned vectors not human-readable tokens |
Simple Analogy
Sticking a tiny learned magnet on the fridge note that changes how the household AI interprets every shopping list — fridge (model) stays put.
Common Usage Examples
- Train 20–100 soft tokens for a classifier task
- HF PEFT prompt tuning
- Multi-tenant: prompt ID → embedding table
- Scale: works better on larger backbones
Summary
In short: Prompt tuning trains only soft input embeddings on a frozen model — ultra-light multi-task adaptation at scale.