← AI Terminology
Underfitting
Underfitting occurs when a model is too simple to capture the underlying patterns in the training data — producing high bias, low variance predictions that perform poorly on both training and test sets because the model hasn't learned the task's complexity.
It is the opposite problem from overfitting.
It is the opposite problem from overfitting.
Why It Matters in AI
While overfitting gets more attention, underfitting is equally common and similarly damaging: a linear model fit to non-linear data, a shallow tree on a complex classification task, or an LLM trained for too few steps — all underfit. The symptom is consistent: poor training accuracy, and equally poor test accuracy (no generalisation gap). Underfitting requires more model capacity, more training, or better features — not more regularisation.
Key Points
| Aspect | Description |
|---|---|
| Causes | Model too small, too few features, too few training steps, excessive regularisation, bad LR |
| Symptoms | High training loss AND high validation loss — no gap between them (unlike overfitting) |
| High bias | Model systematically wrong in the same direction — has strong prior beliefs that override data |
| Solutions | Larger model, more training epochs, less regularisation, better feature engineering, higher LR |
| Low variance | Model doesn't change much with different training sets — because it hasn't learned data specifics |
| vs Overfitting | Overfitting: low train loss, high val loss; Underfitting: both losses high — diagnosis differs |
Simple Analogy
A student who studied too superficially: they wrote "plants need water" for every biology question because they only learned the most basic fact. On both practice tests (training set) and the real exam (test set), they score poorly — not because they memorised specific wrong answers, but because they never learned enough to answer any question well.
Common Usage Examples
- Training curves: train_loss ≈ val_loss ≈ high → underfitting — increase model capacity or train longer
LinearRegressionon clearly non-linear data — systematically underfits; switch toPolynomialFeaturesor tree model- LLM: training for 1,000 steps when 100,000 are needed — perplexity remains high, model hasn't converged
- Too much dropout:
nn.Dropout(0.9)removes 90% of neurons — excessive regularisation → underfitting max_depth=1in DecisionTreeClassifier — stump; can only learn one binary split — extreme underfitting
Summary
In short: Underfitting means a model is too simple to learn the training data's patterns — producing poor performance on both training and test sets, requiring more model capacity, more training, or less regularisation to resolve.