← AI Terminology
NAS - Neural Architecture Search
Neural Architecture Search is an automated ML technique that searches the space of possible neural network architectures — layer types, connections, widths, depths — to find the optimal design for a given task and hardware constraint, replacing manual architecture engineering.
EfficientNet, MobileNetV3, and NASNet were all discovered via NAS.
EfficientNet, MobileNetV3, and NASNet were all discovered via NAS.
Why It Matters in AI
Manually designing neural architectures (ResNet, VGG, Inception) requires deep expertise and months of experimentation. NAS automates this: a search algorithm explores the architecture space and evaluates candidates, finding designs that outperform hand-crafted architectures on specific tasks and hardware targets. EfficientNet, discovered via NAS, achieved state-of-the-art ImageNet accuracy with 8× fewer parameters than ResNet. Hardware-aware NAS finds architectures optimal for specific chips (iPhone NPU, Edge TPU).
Key Points
| Aspect | Description |
|---|---|
| DARTS | Differentiable Architecture Search: relaxes discrete choices to continuous — gradient-friendly |
| Evaluation | Train and evaluate each candidate — naïve approach requires thousands of GPU-days |
| One-shot NAS | Train a single "supernet" containing all architectures; subnet accuracy predicted without training |
| Search space | Defines valid architectures: possible layer types, skip connections, widths, kernel sizes |
| Hardware-aware | Proxy metrics (FLOPs, latency on specific hardware) guide search toward deployable architectures |
| Search strategy | Reinforcement learning, evolutionary algorithms, gradient-based (DARTS), random search |
Simple Analogy
An AI hiring committee for architects: instead of one expert designing a building manually, the committee reviews thousands of candidate floor plans against a rubric (task accuracy + construction cost), selects the best, and iterates. NAS is the committee — it reviews millions of architectures instead of one human reviewing dozens.
Common Usage Examples
- EfficientNet:
torchvision.models.efficientnet_b0(pretrained=True)— NAS-discovered architecture torch.hub.load("rwightman/pytorch-image-models", "tf_efficientnet_b4_ns")— timm library NAS models- DARTS:
from darts import Network; arch = architect.step(...)— gradient-based NAS training - AutoML platforms: Google AutoML, AWS AutoGluon — NAS-backed automated model selection
- MNASNet:
torchvision.models.mnasnet1_0()— mobile-targeted NAS for ImageNet + Pixel phone latency
Summary
In short: Neural Architecture Search automates the design of neural network architectures — discovering models like EfficientNet and MobileNetV3 that outperform hand-crafted designs at given accuracy-efficiency tradeoffs.