← AI Terminology
FPN - Feature Pyramid Network
FPN stands for Feature Pyramid Network: a multi-scale feature extraction architecture for computer vision that builds a top-down pathway with lateral connections, producing rich feature maps at multiple resolutions for detecting objects at different scales.
Introduced by Lin et al. (Facebook AI, 2017) and now ubiquitous in object detection backbones.
Introduced by Lin et al. (Facebook AI, 2017) and now ubiquitous in object detection backbones.
Why It Matters in AI
Object detection must handle objects at wildly different scales — a pedestrian 5 metres away is small in the image; the same pedestrian 1 metre away is large. CNNs naturally produce features at multiple scales (deep = abstract + small, shallow = detailed + large), but earlier features lack semantic richness. FPN adds a top-down pathway that enriches shallow features with deep semantic information — enabling accurate detection at every scale.
Key Points
| Aspect | Description |
|---|---|
| BiFPN | Bi-directional FPN (EfficientDet): weighted bidirectional connections — better feature fusion |
| Output | P2–P5 (or P6/P7) feature maps at multiple resolutions — each level used for different object scales |
| Used in | Faster R-CNN + FPN, RetinaNet, Mask R-CNN, FCOS, DETR variants, Segment Anything Model (SAM) |
| Top-down | Upsample deep semantic features and merge with shallower spatial features via 1×1 convs |
| Bottom-up | Standard CNN forward pass — produces features at decreasing spatial resolutions |
| Lateral conn. | 1×1 conv on bottom-up feature + top-down upsampled feature — fused at each scale level |
Simple Analogy
An art conservator examining a painting: they look at it from across the room (low resolution, global structure) and up close with a magnifying glass (high resolution, fine details). FPN is like simultaneously providing both views to the object detector — every scale of inspection available at once, each enriched by context from the other scales.
Common Usage Examples
- Faster R-CNN + FPN:
torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True) - Mask R-CNN: FPN backbone + mask head — instance segmentation in Detectron2
detectron2.model_zoo.get("COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml")- RetinaNet: FPN backbone + focal loss — one-stage detector competitive with two-stage
- SAM (Segment Anything): uses a ViT image encoder + FPN-style multi-scale features for mask prediction
Summary
In short: FPN combines features from multiple CNN scales by passing semantic information downward — making detectors accurate for both tiny and large objects in the same image.