GSM8K
GSM8K evaluates multi-step grade-school mathematical word-problem solving and, in the original work, the value of ranking generated solutions with a trained verifier.
GSM8K OpenAI October 2021 release
Permanent entry- Task and data
- GSM8K is an 8,500-problem collection of linguistically varied grade-school arithmetic word problems whose natural-language solutions typically require several calculation steps.
- Primary metric
- The release reports final-answer solve rate. Higher is better when the same sample, candidate count, and answer extraction rules are retained.
- What it measures
- multi-step grade-school mathematical word-problem solving and, in the original work, the value of ranking generated solutions with a trained verifier.
- Comparison rules
- Keep the 100-candidate verifier setup separate from single-generation results and modern provider-reported GSM8K variants.
- Access and licensing
- The Compendium links to the official release and does not reproduce word problems or solutions. Check the dataset repository before reuse.
| Published subject | Score | Metric | Split and evaluation setting | Source record |
|---|---|---|---|---|
| OpenAI verification system (exact model variant not named on release page)gsm8k-2021:sample:100-candidate-verifier | 55%higher, percent | solve ratehistorical provider result | sample shared with child comparisonThe system generates 100 candidate solutions and selects the verifier's highest-ranked candidate; the page does not name the exact model variant for this 55% comparison.The subject's exact internal model snapshot is missing from the release page and is deliberately not guessed. | OpenAIPublished 2021-10-29, reviewed 2026-07-13model-provider, published-third-party |
| Small sample of children aged 9–12gsm8k-2021:sample:children | 60%higher, percent | solve ratehistorical human comparison | same sampled problems as the 55% system comparisonVolunteer child comparison reported by OpenAI; sample size and participant-level details are not stated on the release page.This small-sample comparison is not a population-level estimate of child performance. | OpenAIPublished 2021-10-29, reviewed 2026-07-13human-baseline, published-third-party |
Claims and evidence
Claim ledger, 2GSM8K is an 8,500-problem collection of linguistically varied grade-school arithmetic word problems whose natural-language solutions typically require several calculation steps.
The listed result anchors are third-party publications tied to explicit evaluation settings and are not evaluations run by Draconitic.