← AI Terminology
Early Stopping
Early stopping is a regularisation technique that halts training when the model's performance on a validation set stops improving — preventing the model from overfitting by knowing when to stop, not just how to train.
It requires no modification to the model architecture — just a patience counter watching the validation metric.
It requires no modification to the model architecture — just a patience counter watching the validation metric.
Why It Matters in AI
Neural networks can always continue to decrease training loss by memorising data — but validation loss eventually starts rising, signalling overfitting. Early stopping detects this inflection point and stops training there, saving compute and producing a better-generalising model. It is often the first regularisation technique applied alongside or before L2 regularisation and dropout.
Key Points
| Aspect | Description |
|---|---|
| Patience | Number of epochs without improvement before stopping — typical: 5–20 epochs |
| Min delta | Minimum change to count as improvement — prevents stopping on trivial fluctuations |
| Limitation | Validation set must be representative; if too small, stopping criterion is noisy |
| Best model save | Save the model checkpoint from the best epoch — not the one at the stopping point |
| LLM pre-training | Not commonly used — Chinchilla laws define compute-optimal stopping points analytically |
| Monitored metric | Usually validation loss; can be any metric (AUC, F1, BLEU) depending on task |
Simple Analogy
A sprinter practising for a race who improves their time every session — until they plateau and their times start getting slightly worse (fatigue/overfitting). A smart coach stops the athlete at peak form rather than running them into the ground. Early stopping is that coach watching the clock.
Common Usage Examples
keras.callbacks.EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)torch_early_stopping.EarlyStopping(patience=10, min_delta=0.001)— PyTorch equivalent- XGBoost:
xgb.train(params, dtrain, evals=[(dval,'val')], early_stopping_rounds=50) sklearn.neural_network.MLPClassifier(early_stopping=True, n_iter_no_change=10)- Combined with learning rate reduction:
ReduceLROnPlateau → EarlyStopping— standard Keras pipeline
Summary
In short: Early stopping watches validation performance and stops training at the best model checkpoint — simple, effective regularisation that saves compute and prevents overfitting.