← AI Terminology
Semi-Supervised Learning
Semi-supervised learning is a machine learning paradigm that leverages a small labelled dataset together with a large unlabelled dataset — using the structure in unlabelled data to improve a model trained primarily on the scarce labelled examples.
It bridges fully supervised and self-supervised learning.
It bridges fully supervised and self-supervised learning.
Why It Matters in AI
Labelling data is expensive: a medical imaging project might have 500 expert-labelled scans and 50,000 unlabelled ones. Supervised learning uses only the 500; semi-supervised learning uses all 50,500. Techniques like pseudo-labelling (label unlabelled data with the current model), consistency regularisation (augmented views of unlabelled data should produce the same prediction), and co-training unlock the structure in unlabelled data — often achieving the accuracy of 10× more labelled data.
Key Points
| Aspect | Description |
|---|---|
| FixMatch | Semi-supervised SOTA on CIFAR-10 — uses confidence threshold + strong augmentation on unlabelled data |
| Co-training | Two models trained on different views; each labels data for the other — disagreement is informative |
| Self-training | Iterative pseudo-labelling: label → add confident pseudo-labels → retrain — repeat |
| Consistency reg. | FixMatch, MixMatch: augmented versions of an unlabelled image should give the same prediction |
| Pseudo-labelling | Run model on unlabelled data; add high-confidence predictions as labels; re-train |
| Foundation models | LLM few-shot prompting is effectively semi-supervised — model pre-trained unsupervised, fine-tuned with few labels |
Simple Analogy
A student who has 20 textbook problems with solutions (labelled) and 2,000 unsolved practice problems (unlabelled): they use the pattern from the 20 solved problems to attempt the 2,000, mark their best guesses as provisional solutions, and re-study based on those — learning far more than from the 20 alone.
Common Usage Examples
- FixMatch:
unlabeled_loss = consistency_loss(weak_aug_pred, strong_aug_pred) if max(pred) > threshold sklearn.semi_supervised.LabelPropagation— graph-based semi-supervised for tabular data- Pseudo-labelling:
pseudo_labels = model.predict(X_unlabeled); confident = (probs.max(1) > 0.95) - BERT fine-tuning on 100 labelled examples + continued MLM on domain corpus — semi-supervised NLP
torch_semi_supervised.FixMatch(teacher_model, student_model, threshold=0.95)— implementation example
Summary
In short: Semi-supervised learning combines scarce labelled examples with abundant unlabelled data — using unlabelled structure to achieve labelling-efficiency gains of 10× or more, critical for domains where expert annotation is expensive.