ARC-AGI-2 history

Task and data. ARC-AGI-2 evaluates programmatic solutions on novel abstract grid-transformation tasks; the 2025 competition used a hidden private evaluation under efficiency and cost constraints.

Scoring. A task scores as solved when one of two submitted outputs exactly matches the hidden output; the final accuracy is averaged over private tasks and higher is better.

What it measures. few-shot abstraction, rule induction, and exact generalization on unfamiliar visual grid transformations under the competition's resource constraints.

Comparison rules. Compare only the same ARC-AGI-2 year, split, submission allowance, and budget. These are 2025 competition system results, not model-only scores.

Access and licensing. Use ARC data and competition assets only under their official terms. The Compendium reproduces no grids, tasks, or solution artifacts.

Task and data. ARC-AGI-2 evaluates programmatic solutions on novel abstract grid-transformation tasks; the 2025 competition used a hidden private evaluation under efficiency and cost constraints.

Scoring. A task scores as solved when one of two submitted outputs exactly matches the hidden output; the final accuracy is averaged over private tasks and higher is better.

What it measures. few-shot abstraction, rule induction, and exact generalization on unfamiliar visual grid transformations under the competition's resource constraints.

Comparison rules. Compare only the same ARC-AGI-2 year, split, submission allowance, and budget. These are 2025 competition system results, not model-only scores.

Access and licensing. Use ARC data and competition assets only under their official terms. The Compendium reproduces no grids, tasks, or solution artifacts.

Task and data. ARC-AGI-2 evaluates programmatic solutions on novel abstract grid-transformation tasks; the 2025 competition used a hidden private evaluation under efficiency and cost constraints.

Scoring. A task scores as solved when one of two submitted outputs exactly matches the hidden output; the final accuracy is averaged over private tasks and higher is better.

What it measures. few-shot abstraction, rule induction, and exact generalization on unfamiliar visual grid transformations under the competition's resource constraints.

What it does not measure. language knowledge, broad vision, real-world agency, safety, production reliability, or general intelligence as a complete construct.

Limitations. The private set is hidden, systems are highly engineered, and score depends on compute/cost rules. Public, semi-private, private, and later-year tracks differ.

Comparison rules. Compare only the same ARC-AGI-2 year, split, submission allowance, and budget. These are 2025 competition system results, not model-only scores.

Access and licensing. Use ARC data and competition assets only under their official terms. The Compendium reproduces no grids, tasks, or solution artifacts.

Result-reading note. Every result below was published by the named third party and reviewed by the Compendium on 2026-07-13. Flintglade did not run, reproduce, certify, or independently validate these evaluations. A score only belongs to its recorded benchmark version, split, prompt or scaffold, metric, and subject version; omitted setup details are called out instead of guessed.