← AI Terminology
MLP - Multilayer Perceptron
A multilayer perceptron (MLP) is a fully connected feedforward neural network with at least one hidden layer — each neuron connected to every neuron in the next layer via learned weights, with nonlinear activation functions between layers.
It is the simplest and most foundational deep learning architecture.
It is the simplest and most foundational deep learning architecture.
Why It Matters in AI
MLPs are the building block that everything else extends: convolutional layers are weight-tied MLPs over local patches; transformers alternate attention with MLP "FFN" sublayers; the final classifier head of most neural networks is an MLP. Universal approximation theorems guarantee that a sufficiently wide single-hidden-layer MLP can approximate any continuous function — providing the theoretical foundation for why neural networks work. For structured tabular data, MLPs remain competitive with gradient boosting.
Key Points
| Aspect | Description |
|---|---|
| Layers | Input → [hidden layers with activation] → output — minimum one hidden layer to be "deep" |
| vs CNN | MLP has no spatial inductive bias — CNNs outperform on images; MLPs competitive on tabular data |
| Activation | ReLU (standard), GELU (transformers), Swish/SiLU (LLMs) — introduces nonlinearity |
| Fully connected | Every input neuron connects to every hidden neuron — O(n×m) parameters per layer |
| Parameter count | MLP parameters grow with layer width squared — large MLPs can be expensive |
| FFN in transformers | Each transformer block contains a 2-layer MLP (FFN): Linear → activation → Linear → residual |
Simple Analogy
An assembly line where each station transforms the product: raw material (input) passes through stations (hidden layers) where workers (neurons) combine inputs with learned weights and apply a shape filter (activation). The final station outputs the finished product (prediction). Each station specialises based on what it learned during training.
Common Usage Examples
nn.Sequential(nn.Linear(784, 256), nn.ReLU(), nn.Linear(256, 10))— basic MNIST classifier MLPnn.Linear(d_model, 4*d_model), nn.GELU(), nn.Linear(4*d_model, d_model)— transformer FFN sublayer- Tabular data:
MLPClassifier(hidden_layer_sizes=(256, 128), activation='relu')in scikit-learn keras.Sequential([Dense(256, activation='relu'), Dense(128, activation='relu'), Dense(10, activation='softmax')])- PyTorch Lightning: MLP module with
forward,training_step,configure_optimizers— clean MLP template
Summary
In short: The MLP is the foundational neural network — fully connected layers with nonlinear activations — and the building block underlying every modern architecture, from convolutional networks to transformer FFN layers.