← AI Terminology
SVM - Support Vector Machine
A Support Vector Machine is a supervised learning algorithm that finds the hyperplane in a high-dimensional feature space that maximally separates classes — the "maximum margin" classifier — with support vectors being the training examples closest to the decision boundary.
SVMs were the dominant ML algorithm from the late 1990s until deep learning (2012).
SVMs were the dominant ML algorithm from the late 1990s until deep learning (2012).
Why It Matters in AI
SVMs introduced the kernel trick — mapping data to high-dimensional spaces where non-linear problems become linearly separable, without explicitly computing the transformation. This gave SVMs theoretical guarantees (maximum margin = good generalisation) and practical power on moderate-sized datasets. For small-to-medium labelled datasets where deep learning is impractical, SVMs remain highly competitive. Text classification, bioinformatics, and anomaly detection are domains where SVMs are still used in production.
Key Points
| Aspect | Description |
|---|---|
| SVR | Support Vector Regression — same maximum margin principle applied to regression |
| γ (gamma) | RBF kernel width — high γ: narrow Gaussians, complex boundary; low γ: smooth boundary |
| C parameter | Regularisation: low C = wide margin, more misclassification allowed; high C = narrow, few errors |
| Kernel trick | Compute similarity in high-dim space without explicit mapping — RBF, polynomial, linear kernels |
| Support vectors | Training examples on the margin boundary — the model depends only on these points |
| Decision boundary | Hyperplane maximising the margin (distance to nearest data points from each class) |
Simple Analogy
Drawing the widest possible road between two neighbourhoods on a map: the road (decision boundary) must separate all houses (data points) of each neighbourhood, and should be as wide as possible (maximum margin). The houses right on the road's edge (support vectors) are the critical points — moving any other house doesn't change the road.
Common Usage Examples
sklearn.svm.SVC(kernel='rbf', C=1.0, gamma='scale')— standard SVM classifiersklearn.svm.SVC(kernel='linear')— linear SVM for high-dim sparse features (text)GridSearchCV(SVC(), {'C': [0.1, 1, 10], 'gamma': ['scale', 'auto']}, cv=5)— hyperparameter tuning- Text:
Pipeline([("tfidf", TfidfVectorizer()), ("svc", LinearSVC())])— text classification pipeline sklearn.svm.OneClassSVM(nu=0.05)— anomaly detection: learn the boundary of normal data
Summary
In short: SVMs find the maximum-margin hyperplane separating classes using the kernel trick to handle non-linear boundaries — the dominant pre-deep-learning algorithm, still competitive for small-to-medium datasets and valued for its theoretical guarantees.