← AI Terminology
Training a Model
Training a model is the process of adjusting a model's internal parameters so that its predictions match the target values in a labelled dataset as closely as possible.
The model repeatedly sees examples, measures how wrong its predictions are (the loss), and nudges its weights in the direction that reduces that error.
The model repeatedly sees examples, measures how wrong its predictions are (the loss), and nudges its weights in the direction that reduces that error.
Why It Matters in AI
Training is where a model acquires all of its knowledge — before training, weights are random noise; after training, they encode patterns from potentially billions of examples. Compute cost, data quality, and training stability determine whether a model becomes useful or fails to converge. The dramatic progress in AI since 2017 is largely a story of scaling training: more data, more parameters, more GPU-hours.
Key Points
| Aspect | Description |
|---|---|
| Epoch | One full pass through the entire training dataset |
| Hardware | GPUs/TPUs parallelise the matrix operations; large models require multi-node clusters |
| Core loop | Forward pass → compute loss → backward pass (gradients) → update weights via optimiser |
| Optimiser | Algorithm that updates weights (SGD, Adam, AdamW are most common) |
| Overfitting | Model memorises training data but fails on new data — countered by regularisation and more data |
| Loss function | Measures prediction error (e.g. cross-entropy for classification, MSE for regression) |
Simple Analogy
Training a model is like a student doing thousands of practice exam questions. Each wrong answer is scored, the student reviews the mistake, and adjusts their understanding slightly. After enough practice rounds (epochs), the student generalises — they can answer questions they've never seen before.
Common Usage Examples
model.fit(X_train, y_train, epochs=10)in Keras- PyTorch training loop:
loss.backward(); optimizer.step(); optimizer.zero_grad() - Pre-training GPT on trillions of tokens of internet text, then fine-tuning on curated instruction data
--train_batch_size,--learning_rate,--num_train_epochsflags in HuggingFaceTrainer- Monitoring training loss vs validation loss curves in TensorBoard or Weights & Biases
Summary
In short: Training is the process of turning a blank set of weights into a model that knows something — it's where all the compute budget goes and where all the intelligence comes from.