← AI Terminology
Latent Diffusion
Latent diffusion runs the diffusion process in a compressed latent space from a pretrained autoencoder instead of raw pixels — dramatically cutting training and sampling cost.
Stable Diffusion popularised the approach.
Stable Diffusion popularised the approach.
Why It Matters in AI
Pixel-space diffusion is expensive. Latent diffusion made high-res open image generation practical on consumer GPUs and defined the Stable Diffusion architecture era.
Key Points
| Aspect | Description |
|---|---|
| Stack | VAE + U-Net/DiT + text conditioner |
| Origin | Rombach et al., High-Resolution Image Synthesis with LDMs |
| Benefit | Faster/cheaper than pixel diffusion |
| Related | VAE, diffusion models, Stable Diffusion |
| Pipeline | Encode image → diffuse latents → decode to pixels |
| Tradeoff | Autoencoder limits reconstruction fidelity |
Simple Analogy
Composing music as compact MIDI and only rendering full audio at the end — work in a smaller space, expand when needed.
Common Usage Examples
- Stable Diffusion LDM pipeline
- Train LoRAs in latent space
- Swap VAEs for different fidelity
- DiT latents in newer generators
Summary
In short: Latent diffusion denoises in autoencoder space — the efficiency breakthrough behind Stable Diffusion and modern open T2I.