LiveCodeBench history

Task and data. LiveCodeBench continuously collects coding-contest problems after model release dates and evaluates code generation, self-repair, code execution, and output prediction.

Scoring. The paper uses scenario-specific pass@1. The anchors below are total code-generation pass@1 on the paper's post-September-2023 window; higher is better.

What it measures. coding performance on recently released contest problems and several adjacent code-reasoning scenarios with time-window analysis for possible contamination.

Comparison rules. Pin benchmark release, scenario, date window, difficulty aggregation, prompt, and model snapshot. The paper-v2 anchors are not current leaderboard standings.

Access and licensing. The official code is MIT-licensed, but contest problem statements originate with LeetCode, AtCoder, and Codeforces and retain platform-specific terms. No problems are reproduced here.

Task and data. LiveCodeBench continuously collects coding-contest problems after model release dates and evaluates code generation, self-repair, code execution, and output prediction.

Scoring. The paper uses scenario-specific pass@1. The anchors below are total code-generation pass@1 on the paper's post-September-2023 window; higher is better.

What it measures. coding performance on recently released contest problems and several adjacent code-reasoning scenarios with time-window analysis for possible contamination.

Comparison rules. Pin benchmark release, scenario, date window, difficulty aggregation, prompt, and model snapshot. The paper-v2 anchors are not current leaderboard standings.

Access and licensing. The official code is MIT-licensed, but contest problem statements originate with LeetCode, AtCoder, and Codeforces and retain platform-specific terms. No problems are reproduced here.

Task and data. LiveCodeBench continuously collects coding-contest problems after model release dates and evaluates code generation, self-repair, code execution, and output prediction.

Scoring. The paper uses scenario-specific pass@1. The anchors below are total code-generation pass@1 on the paper's post-September-2023 window; higher is better.

What it measures. coding performance on recently released contest problems and several adjacent code-reasoning scenarios with time-window analysis for possible contamination.

What it does not measure. repository engineering, product-code quality, security, maintainability, cost, latency, or immunity from all training-data contamination.

Limitations. It is rolling: problem counts, date windows, releases, and leaderboards change. Contest platform mix and difficulty distribution also affect results.

Comparison rules. Pin benchmark release, scenario, date window, difficulty aggregation, prompt, and model snapshot. The paper-v2 anchors are not current leaderboard standings.

Access and licensing. The official code is MIT-licensed, but contest problem statements originate with LeetCode, AtCoder, and Codeforces and retain platform-specific terms. No problems are reproduced here.

Result-reading note. Every result below was published by the named third party and reviewed by the Compendium on 2026-07-13. Flintglade did not run, reproduce, certify, or independently validate these evaluations. A score only belongs to its recorded benchmark version, split, prompt or scaffold, metric, and subject version; omitted setup details are called out instead of guessed.