← AI Terminology
Mini-Batch
A mini-batch is a small, random subset of the training dataset used for one gradient update — a compromise between computing gradients over a single example (stochastic) and the entire dataset (batch), offering a practical balance between training speed, memory use, and gradient quality.
Modern deep learning uses mini-batch gradient descent almost exclusively.
Modern deep learning uses mini-batch gradient descent almost exclusively.
Why It Matters in AI
Batch gradient descent requires loading the entire dataset into memory — impossible for ImageNet-scale data. Stochastic gradient descent (one example at a time) is noisy and can't leverage GPU parallelism. Mini-batches solve both: they fit in GPU memory, parallelise efficiently, provide reasonably accurate gradient estimates, and the noise in the estimate helps escape local minima. Batch size is a key hyperparameter: 32–512 for vision models, 1–32 for LLM training on large sequences.
Key Points
| Aspect | Description |
|---|---|
| Typical sizes | 32, 64, 128, 256 for vision; 1–32 for LLM pre-training (limited by sequence length × GPU memory) |
| Gradient noise | Smaller batches → noisier gradients → acts as regularisation; larger batches → sharper minima |
| GPU utilisation | Larger batches → better GPU utilisation up to a point — diminishing returns above ~512 |
| Epoch definition | One epoch = dataset_size / batch_size mini-batches processed |
| Learning rate link | Larger batch → can use larger LR (linear scaling rule) — but may harm generalisation |
| Gradient accumulation | Simulates large batches by averaging gradients over K mini-batches before updating weights |
Simple Analogy
A chef who tastes a spoonful of soup to judge the seasoning (mini-batch), rather than the whole pot (full batch) or a single drop (stochastic). The spoonful is big enough to be representative but small enough to sample quickly — and slightly imperfect estimates are fine when you can taste again in a moment.
Common Usage Examples
DataLoader(dataset, batch_size=64, shuffle=True)— standard PyTorch mini-batch loader- LLM training:
batch_size=4, gradient_accumulation_steps=32— effective batch of 128 for batch in dataloader: loss = model(batch); loss.backward(); optimizer.step()— training loop- Mixed precision: larger effective batch size as FP16 halves memory per sample
TrainingArguments(per_device_train_batch_size=8, gradient_accumulation_steps=4)— HuggingFace config
Summary
In short: A mini-batch is a random subset of training data used for one gradient update — the universal compromise that enables efficient GPU training on datasets too large to fit in memory.