← AI Terminology
Foundation Model
A foundation model is a large AI model trained on broad data at scale that can be adapted (fine-tuned, prompted, or used as a backbone) for a wide range of downstream tasks — a single model as the foundation for many applications.
The term was coined by Stanford HAI in 2021 to describe GPT-3-class models and their successors.
The term was coined by Stanford HAI in 2021 to describe GPT-3-class models and their successors.
Why It Matters in AI
Foundation models represent a paradigm shift: instead of training a separate model for each task, one enormous model trained on diverse data serves as the starting point for thousands of applications. GPT-4, Claude, Llama, CLIP, Stable Diffusion, and Whisper are all foundation models. They concentrate capability and investment — and raise new questions about who controls the infrastructure of AI.
Key Points
| Aspect | Description |
|---|---|
| - | Concentration risk: if the foundation model has bias or errors, all downstream models inherit them |
| Scale | Trained on internet-scale data (trillions of tokens, billions of images) using massive compute |
| Ecosystem | Foundation model → many fine-tuned variants → many applications — the upstream determines the downstream |
| Generality | Single model serves translation, summarisation, coding, Q&A, image generation, etc. |
| Modalities | Language (GPT, Llama), Vision (ViT, SAM), Multimodal (Gemini, Claude 3), Audio (Whisper) |
| Adaptation methods | Fine-tuning, LoRA, RLHF, few-shot prompting, RAG — all ways to specialise for a task |
Simple Analogy
A foundation model is like a university education: broad training across many disciplines, enabling graduates to specialise quickly into law, medicine, engineering, or teaching — rather than training each graduate from scratch for their specific profession. The university (foundation model) is expensive; the specialisation (fine-tuning) is cheap.
Common Usage Examples
- GPT-4 as foundation: powers ChatGPT, GitHub Copilot, Bing Chat, and thousands of API applications
- Llama 3 as open foundation: 10,000+ fine-tuned variants on HuggingFace for every domain
- CLIP as visual foundation: backbone of Stable Diffusion, DALL-E guidance, semantic image search
- SAM (Segment Anything): visual foundation model — zero-shot segmentation of any object
- Whisper: audio foundation model — fine-tuned for 99 languages, medical transcription, live captions
Summary
In short: A foundation model is trained once at enormous scale and adapted for thousands of applications — the infrastructure layer of modern AI that determines what every downstream product can and cannot do.