← AI Terminology
ResNet - Residual Network
ResNet (Residual Network) is a CNN architecture introduced by He et al. (Microsoft Research, 2015) that uses residual connections (skip connections) to enable training of very deep networks — winning ILSVRC 2015 with 152 layers and a 3.57% top-5 error, surpassing human-level performance on ImageNet.
It is the most cited paper in computer vision history and remains a foundational architecture.
It is the most cited paper in computer vision history and remains a foundational architecture.
Why It Matters in AI
Before ResNet, networks deeper than ~20 layers degraded in accuracy — not from overfitting but from optimisation failure. ResNet's residual blocks allowed 50, 101, and 152-layer networks to train effectively, demonstrating that depth alone dramatically improves accuracy. It established skip connections as a universal principle adopted by every subsequent architecture: EfficientNet, ViT, BERT, GPT. ResNet-50 and ResNet-101 remain standard transfer learning backbones in production computer vision systems.
Key Points
| Aspect | Description |
|---|---|
| Variants | ResNet-18, -34 (basic blocks), -50, -101, -152 (bottleneck blocks), ResNeXt, Wide ResNet |
| Batch Norm | ResNet popularised Batch Normalisation after each conv — enabled stable training at depth |
| Pretraining | torchvision.models.resnet50(pretrained=True) — pretrained on ImageNet, fine-tune anywhere |
| Residual block | y = F(x, W) + x — two conv layers + skip connection from input to output |
| Bottleneck block | 1×1 → 3×3 → 1×1 convolutions — reduces compute for deeper variants (ResNet-50+) |
| ImageNet results | ResNet-152: 3.57% top-5 error — first model to surpass estimated human performance (5%) |
Simple Analogy
A skyscraper with emergency stairwells parallel to the elevators: if an elevator (layer) breaks, people can still climb the stairs (skip connection) to reach upper floors. Without stairwells, a broken elevator strands everyone below — with them, the building (network) keeps functioning at full height.
Common Usage Examples
torchvision.models.resnet50(pretrained=True)— standard pretrained ResNet-50- Transfer learning: replace final FC layer →
model.fc = nn.Linear(2048, num_classes)→ fine-tune from timm import create_model; model = create_model("resnet50", pretrained=True)— timm library- ResNeXt:
torchvision.models.resnext50_32x4d()— grouped convolutions variant, higher accuracy - Backbone in detection: ResNet-50 + FPN in Faster R-CNN, Mask R-CNN — standard detection backbone
Summary
In short: ResNet introduced residual connections to enable training networks over 100 layers deep — winning ImageNet with superhuman accuracy, establishing skip connections as a universal deep learning principle adopted by every major architecture since.