MATH history

Task and data. MATH contains 12,500 competition-mathematics problems across seven subjects and five difficulty levels, each paired with a worked solution in the released dataset.

Scoring. The paper primarily reports exact-answer accuracy after normalization of equivalent final-answer formats; higher is better within the same split and generation rules.

What it measures. competition-style mathematical problem solving across algebra, geometry, number theory, counting and probability, and related school-level areas.

Comparison rules. Separate few-shot GPT-3 from AMPS-pretrained and fine-tuned GPT-2 results. Modern MATH variants and scoring pipelines are different lineages.

Access and licensing. The Compendium does not reproduce problems, figures, or solutions. Code licensing does not automatically grant rights to every source problem.

Task and data. MATH contains 12,500 competition-mathematics problems across seven subjects and five difficulty levels, each paired with a worked solution in the released dataset.

Scoring. The paper primarily reports exact-answer accuracy after normalization of equivalent final-answer formats; higher is better within the same split and generation rules.

What it measures. competition-style mathematical problem solving across algebra, geometry, number theory, counting and probability, and related school-level areas.

Comparison rules. Separate few-shot GPT-3 from AMPS-pretrained and fine-tuned GPT-2 results. Modern MATH variants and scoring pipelines are different lineages.

Access and licensing. The Compendium does not reproduce problems, figures, or solutions. Code licensing does not automatically grant rights to every source problem.

Task and data. MATH contains 12,500 competition-mathematics problems across seven subjects and five difficulty levels, each paired with a worked solution in the released dataset.

Scoring. The paper primarily reports exact-answer accuracy after normalization of equivalent final-answer formats; higher is better within the same split and generation rules.

What it measures. competition-style mathematical problem solving across algebra, geometry, number theory, counting and probability, and related school-level areas.

What it does not measure. formal proof correctness, open-ended research mathematics, tool-assisted mathematics, calibration, or the pedagogical quality of generated derivations.

Limitations. Exact-answer grading can accept flawed derivations, human comparison used only 20 sampled problems, and static public items can contaminate training.

Comparison rules. Separate few-shot GPT-3 from AMPS-pretrained and fine-tuned GPT-2 results. Modern MATH variants and scoring pipelines are different lineages.

Access and licensing. The Compendium does not reproduce problems, figures, or solutions. Code licensing does not automatically grant rights to every source problem.

Result-reading note. Every result below was published by the named third party and reviewed by the Compendium on 2026-07-13. Flintglade did not run, reproduce, certify, or independently validate these evaluations. A score only belongs to its recorded benchmark version, split, prompt or scaffold, metric, and subject version; omitted setup details are called out instead of guessed.