← AI Terminology
Structured Sparsity
Structured sparsity zeroes parameters in hardware-friendly patterns (blocks, channels, N:M) so accelerators can skip compute, unlike irregular unstructured pruning.
It aims for real wall-clock speedups, not only parameter counts.
It aims for real wall-clock speedups, not only parameter counts.
Why It Matters in AI
Unstructured 90% sparse models often do not run faster without special kernels. Structured patterns (e.g. 2:4) map to Sparse Tensor Cores and deliver measured speedups — key for efficient deployment research.
Key Points
| Aspect | Description |
|---|---|
| Metric | Achieved speedup, not only % zeros |
| Related | Pruning, MoE (conditional compute) |
| Hardware | NVIDIA sparse tensor cores for 2:4 |
| Patterns | Block, filter, N:M (2:4), head pruning |
| Training | Sparse aware training or prune+finetune |
| Vs unstructured | Easier speedups; sometimes more accuracy loss |
Simple Analogy
Removing whole shelves from a warehouse in neat rows so forklifts take shorter paths — not randomly deleting single boxes everywhere.
Common Usage Examples
- 2:4 sparsity on supported GPUs
- Channel prune then finetune CNNs
- Measure real tokens/s not only sparsity %
- Combine with quantisation
Summary
In short: Structured sparsity removes weights in accelerator-friendly patterns so sparsity becomes real speed, not only smaller files.