← AI Terminology
Capsule Network
A Capsule Network (CapsNet) is a neural network architecture proposed by Geoffrey Hinton (2017) that replaces scalar neuron activations with vectors ("capsules") representing both the presence and the spatial properties (pose, orientation, scale) of features.
It was designed to address CNNs' inability to encode spatial relationships between features.
It was designed to address CNNs' inability to encode spatial relationships between features.
Why It Matters in AI
CNNs achieve position invariance through pooling, which discards spatial information — a face detector doesn't care where the eyes are relative to the nose. Capsule Networks preserve this relational structure using dynamic routing between capsules. While they haven't displaced CNNs at scale, they represent an influential architectural idea and motivation for thinking about compositional representations in AI.
Key Points
| Aspect | Description |
|---|---|
| Status | Influential but not mainstream — has not yet displaced CNNs or Vision Transformers in practice |
| Capsule | A group of neurons whose activity vector represents presence (magnitude) and pose (direction) |
| Advantage | Equivariant to spatial transformations (rotation, scale) rather than just invariant |
| EM routing | Later variant (Hinton, 2018) uses expectation-maximisation instead of dynamic routing |
| Limitation | Much slower than CNNs; difficult to scale to large datasets like ImageNet competitively |
| Dynamic routing | Lower-level capsules "vote" for which higher-level capsule they belong to — iterative clustering |
Simple Analogy
A CNN looking at a face might recognise "there are eyes, a nose, and a mouth" — but doesn't track their relative positions. A Capsule Network represents not just "I see an eye" but "I see an eye at this angle and location" — and checks that all parts are in the right spatial relationship to call it a face.
Common Usage Examples
- Original CapsNet paper achieved 99.75% on MNIST with fewer parameters than CNNs
pip install torch-capsuleor custom implementations in PyTorch- Medical image segmentation research uses CapsNets for spatial equivariance
- Hinton's EM Routing CapsNet (2018): "Matrix Capsules with EM Routing" — improved version
- CapsNet used in NLP for text classification (semantic composition with part-whole relationships)
Summary
In short: Capsule Networks encode both the presence and the pose of features — a theoretically elegant alternative to CNNs that hasn't yet scaled to match them in practice.