← AI Terminology
LiveCodeBench
LiveCodeBench continuously harvests new coding problems from contests to evaluate LLMs with reduced contamination from training on leaked benchmarks.
It aims for a live, contamination-aware coding leaderboard.
It aims for a live, contamination-aware coding leaderboard.
Why It Matters in AI
Static code benches leak into training data over time. Live, time-stamped problems better estimate real coding ability for agents and copilots — increasingly important as older benches saturate and contaminate.
Key Points
| Aspect | Description |
|---|---|
| Use | Contamination-aware comparisons |
| Idea | Problems released after model training cutoffs |
| Task | Competitive programming style with tests |
| Caveat | Still not full software engineering |
| Metric | pass@k over time windows |
| Related | HumanEval, SWE-bench, Codeforces-style evals |
Simple Analogy
Pop quizzes written after the semester so students cannot have memorised last year’s exact exam PDF.
Common Usage Examples
- Track model scores by problem release date
- Compare to HumanEval for contamination suspicion
- Use in coding agent papers
- Report time-split results
Summary
In short: LiveCodeBench evaluates coding on newly released problems — a live benchmark designed to fight training-data contamination.