← AI Terminology
Log-Loss
Log-loss (logarithmic loss, also called binary cross-entropy for two-class problems) is a loss function that measures the quality of a classifier's predicted probabilities — penalising confident wrong predictions far more than uncertain wrong predictions.
It is the standard loss function for training and evaluating probabilistic classifiers.
It is the standard loss function for training and evaluating probabilistic classifiers.
Why It Matters in AI
Accuracy only cares about the final class prediction (right or wrong); log-loss cares about calibration — how confident the model was. A model that predicts 51% probability for every correct answer has good accuracy but terrible log-loss; a well-calibrated model that says "95% certain" and is usually right scores well on both. In production AI systems, probability scores drive downstream decisions (risk thresholds, bet sizing), making log-loss more actionable than accuracy.
Key Points
| Aspect | Description |
|---|---|
| Range | [0, ∞) — perfect predictions → 0; predicting 0 probability for the true class → ∞ |
| In Sklearn | sklearn.metrics.log_loss(y_true, y_pred_proba) — takes probability array, not class labels |
| Multiclass | −Σ yᵢ × log(pᵢ) — sum over all classes; reduces to binary formula for two classes |
| Calibration link | Low log-loss requires both accuracy and calibration — overconfident wrong predictions blow up the score |
| Formula (binary) | −(y × log(p) + (1−y) × log(1−p)) — where y ∈ {0,1}, p = predicted probability of class 1 |
| Penalty structure | Log function makes wrong confident predictions catastrophically costly (asymptotic penalty) |
Simple Analogy
A weather forecaster graded not on "right or wrong" but on confidence: saying "90% chance of rain" and it doesn't rain is catastrophic; saying "55% chance of rain" when it doesn't rain is barely penalised. Log-loss is the scoring system that makes forecasters honest about uncertainty.
Common Usage Examples
sklearn.metrics.log_loss(y_true, y_pred_proba)— evaluate classifier probability outputloss = nn.BCELoss()(sigmoid(logits), targets)— binary log-loss in PyTorchloss = nn.CrossEntropyLoss()(logits, targets)— multiclass log-loss (includes softmax internally)- Kaggle competitions: log-loss is the evaluation metric for probabilistic classification challenges
- XGBoost:
objective='binary:logistic'— optimises log-loss for binary classification
Summary
In short: Log-loss measures the quality of predicted probabilities, not just class predictions — rewarding confident correct answers and severely penalising confident wrong answers, making it the go-to metric for probabilistic classifiers.