← AI Terminology
Algorithm
An algorithm is a finite, ordered set of instructions that transforms input into output — a precise recipe for solving a problem.
In ML, "algorithm" usually refers to the learning procedure (gradient descent, k-means, ID3) that produces a trained model.
In ML, "algorithm" usually refers to the learning procedure (gradient descent, k-means, ID3) that produces a trained model.
Why It Matters in AI
Every AI system runs on algorithms at multiple levels: the training algorithm that adjusts weights, the inference algorithm that generates predictions, and the search algorithm that finds the best output. Understanding algorithms lets practitioners choose the right tool for a problem, diagnose failures, and reason about computational cost and correctness.
Key Points
| Aspect | Description |
|---|---|
| Complexity | Described in Big-O notation — O(n²) means cost grows quadratically with input size |
| Determinism | Algorithms are deterministic given the same input and random seed — reproducibility depends on this |
| Distinction | Algorithm ≠ model — the algorithm is the process; the model is the result |
| Examples in AI | PageRank (ranking), A* (pathfinding), MCTS (game playing), attention (Transformers) |
| Learning algorithms | Gradient descent, backpropagation, expectation-maximisation, MCMC |
| Inference algorithms | Beam search (NLP), Viterbi (HMM), non-maximum suppression (object detection) |
Simple Analogy
A recipe is an algorithm for cooking: exact ingredients (input), ordered steps (instructions), and a dish (output). If you follow the recipe perfectly twice, you get the same dish. The recipe is not the meal — the algorithm is not the model.
Common Usage Examples
- Gradient descent: iteratively adjusts weights in the direction that reduces loss
- k-means: assigns data points to clusters by iterating centroid assignment and update steps
- Beam search: explores k candidate sequences in parallel during text generation
- Sorting algorithms embedded in data preprocessing pipelines (mergesort, quicksort)
- MCTS (Monte Carlo Tree Search): algorithm behind AlphaGo's game-tree search
Summary
In short: An algorithm is the step-by-step procedure an AI uses to learn or to predict — the instructions, not the learned knowledge.