← AI Terminology
Experiment Tracking
Experiment tracking logs parameters, metrics, code versions, and artifacts for each training or eval run so results are comparable and reproducible.
It is day-one infrastructure for serious ML teams.
It is day-one infrastructure for serious ML teams.
Why It Matters in AI
Without logs, optimisation is folklore. Trackers make hyperparameter search and ablations scientific. For LLMs, log prompts, judges, and token costs too.
Key Points
| Aspect | Description |
|---|---|
| LLM | Track prompt versions and eval harness scores |
| Tools | W&B, MLflow, Neptune, TensorBoard, Comet |
| Logged | Hparams, metrics, git commit, data hash, charts |
| Benefit | Compare runs; resume; collaborate |
| Hygiene | One run = one intentional change when possible |
| Related | Model registry, MLOps |
Simple Analogy
A lab notebook that automatically records every knobs setting and every measurement so you can prove which change helped.
Common Usage Examples
wandb.log({'loss': loss})- MLflow nested runs for sweeps
- Attach confusion matrices as artifacts
- Compare learning-rate sweeps in UI
Summary
In short: Experiment tracking records params and metrics per run — the memory that makes model development reproducible.