← AI Terminology
FP8 - 8-bit Floating Point
FP8 is an 8-bit floating-point format (variants E4M3/E5M2) used for low-precision training and inference on modern accelerators (e.g. NVIDIA Hopper/Blackwell Transformer Engine).
It sits between FP16/BF16 and INT8 in the precision stack.
It sits between FP16/BF16 and INT8 in the precision stack.
Why It Matters in AI
Lower precision multiplies throughput and cuts memory. FP8 enables faster LLM training/inference with hardware support while retaining dynamic range advantages over pure ints for some tensors.
Key Points
| Aspect | Description |
|---|---|
| Use | Training and inference mixed-precision recipes |
| Care | Scaling factors and delayed scaling strategies |
| Related | BF16, mixed precision, Tensor Cores |
| Vs INT8 | Different tradeoffs; not a full replacement |
| Hardware | Transformer Engine on recent NVIDIA GPUs |
| Variants | E4M3 (more precision), E5M2 (more range) |
Simple Analogy
Writing numbers with fewer digits but scientific notation still allowed — compact, yet able to represent both tiny and huge values.
Common Usage Examples
- Transformer Engine FP8 training examples
- Inference serving with FP8 weights/activations
- Compare speed vs BF16
- Monitor loss spikes from unstable scales
Summary
In short: FP8 is 8-bit floating point for faster train/serve on modern GPUs — a key low-precision format beyond FP16/BF16.