← AI Terminology
Cross-Validation
Cross-validation is a model evaluation technique that splits data into multiple folds, trains the model on different combinations of training/validation splits, and averages the results — giving a more reliable performance estimate than a single train-test split.
K-fold cross-validation is the most common form: split into k folds, train on k−1, validate on the held-out fold, rotate k times.
K-fold cross-validation is the most common form: split into k folds, train on k−1, validate on the held-out fold, rotate k times.
Why It Matters in AI
A single train-test split can be lucky or unlucky depending on which data ends up in the test set. Cross-validation reduces this variance by evaluating on every partition of the data, giving a lower-variance estimate of real-world performance. It is particularly valuable for small datasets where holding out 20% for testing wastes too much data.
Key Points
| Aspect | Description |
|---|---|
| K-fold | Split into k equal folds; train on k−1, test on 1; rotate k times; average k scores |
| Nested CV | Outer CV for model evaluation, inner CV for hyperparameter tuning — avoids optimistic bias |
| Limitation | k× more training cost; models are discarded after evaluation — not the final production model |
| Time series | Walk-forward validation: always train on past, validate on future — no random shuffling |
| Leave-One-Out | k = n; each sample is its own test set once — unbiased but computationally expensive |
| Stratified K-fold | Preserves class distribution in each fold — essential for imbalanced datasets |
Simple Analogy
Instead of taking one exam to determine your final grade, you sit five exams on five different days. Your grade is the average of all five — a more reliable measure of your knowledge than any single test, because one bad day or one easy exam doesn't dominate.
Common Usage Examples
cross_val_score(model, X, y, cv=5, scoring='roc_auc')in scikit-learnStratifiedKFold(n_splits=10)for classification with imbalanced classesTimeSeriesSplit(n_splits=5)for proper sequential validation on time-ordered data- Nested CV:
cross_val_score(GridSearchCV(model, param_grid, cv=5), X, y, cv=10) - Kaggle: local cross-validation is the standard way to trust leaderboard position before submitting
Summary
In short: Cross-validation gives you a reliable performance estimate by testing on every slice of the data — essential when you can't afford to waste data on a single held-out test set.