← AI Terminology
N:M Sparsity
N:M sparsity is a structured pattern where every group of M weights keeps at most N non-zeros (e.g. 2:4), matching sparse tensor core constraints.
It is the practical sparse format on modern NVIDIA GPUs.
It is the practical sparse format on modern NVIDIA GPUs.
Why It Matters in AI
2:4 sparsity can deliver inference speedups on compatible hardware with moderate quality loss after finetuning. It is the sparsity flavour engineers mean when discussing GPU-accelerated sparse LLMs/CNNs.
Key Points
| Aspect | Description |
|---|---|
| Use | Efficient CNN/LLM inference research and products |
| Train | ASP (automatic sparse pruning) style recipes |
| Example | 2:4 → two non-zeros in every four weights |
| Related | Structured sparsity, pruning |
| Hardware | Sparse Tensor Cores accelerate 2:4 |
| Tradeoff | Accuracy vs dense; needs recovery finetune |
Simple Analogy
In every four lockers, only two may hold items — a tidy rule that lets robots skip empty pairs quickly.
Common Usage Examples
- PyTorch/NVIDIA ASP 2:4 training examples
- Export sparse inference engines
- Compare dense vs 2:4 latency
- Finetune after mask is fixed
Summary
In short: N:M sparsity keeps N non-zeros per M weights — the structured pattern GPUs can actually accelerate.