← AI Terminology
Diffusion Model
A diffusion model is a generative model that learns to reverse a gradual noise-addition process — it is trained to incrementally denoise data starting from pure Gaussian noise, ultimately generating realistic samples (images, audio, video) from nothing.
DALL-E 2, Stable Diffusion, Sora, and most modern image generators are diffusion models.
DALL-E 2, Stable Diffusion, Sora, and most modern image generators are diffusion models.
Why It Matters in AI
Diffusion models produce higher-quality, more diverse samples than GANs and are more stable to train — there's no adversarial game. They powered the text-to-image generation wave (2022–present), producing photorealistic and artistic images from text prompts at commercial scale. They've since expanded to video (Sora), audio (AudioDiff), protein design, and drug discovery.
Key Points
| Aspect | Description |
|---|---|
| LDM | Latent Diffusion Models (Stable Diffusion) — diffusion in compressed latent space for speed |
| DDPM | Denoising Diffusion Probabilistic Models (Ho et al., 2020) — foundational paper |
| vs GAN | More stable training, better diversity; slower sampling (requires many denoising steps) |
| Conditioning | Text, image, or other signals guide the denoising — enables text-to-image generation |
| Forward process | Gradually adds Gaussian noise to data over T steps until it's pure noise — a fixed Markov chain |
| Reverse process | Learned neural network (U-Net or Transformer) predicts and removes noise step by step |
Simple Analogy
Start with a photograph, gradually bury it under random static until it's pure noise — then learn to reverse that process. A diffusion model learns to remove snow from a snowy image step by step. Text conditioning is the instruction that guides which image to unbury from the noise.
Common Usage Examples
- Stable Diffusion:
pipe = StableDiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1") - DALL-E 3 via API:
openai.images.generate(model="dall-e-3", prompt="...") - Sora (OpenAI): video diffusion model generating high-quality 60-second clips from text
- AudioDiff / MusicGen: diffusion-based audio generation from text descriptions
- ControlNet: adds spatial conditioning (edge maps, depth maps) to Stable Diffusion generation
Summary
In short: Diffusion models learn to generate by learning to denoise — starting from pure noise and iteratively sculpting a realistic image, video, or audio output.