GAIA history

Task and data. GAIA contains 466 real-world assistant questions divided into three difficulty levels and requiring combinations of reasoning, web browsing, files, multimodal inputs, and tools.

Scoring. Accuracy is the share of questions receiving the unambiguous final answer; higher is better for a fixed split and tool-access policy.

What it measures. general-assistant task completion when answers require coordinated reasoning, retrieval, multimodal handling, and external tool use.

Comparison rules. A GAIA score is a system score. Match the 2023 release, split, tool set, browser state, and answer normalization before comparing results.

Access and licensing. The GAIA dataset card forbids resharing validation or test data in a crawlable form and requires gated access. The Compendium reproduces no questions, files, or answers.

Task and data. GAIA contains 466 real-world assistant questions divided into three difficulty levels and requiring combinations of reasoning, web browsing, files, multimodal inputs, and tools.

Scoring. Accuracy is the share of questions receiving the unambiguous final answer; higher is better for a fixed split and tool-access policy.

What it measures. general-assistant task completion when answers require coordinated reasoning, retrieval, multimodal handling, and external tool use.

Comparison rules. A GAIA score is a system score. Match the 2023 release, split, tool set, browser state, and answer normalization before comparing results.

Access and licensing. The GAIA dataset card forbids resharing validation or test data in a crawlable form and requires gated access. The Compendium reproduces no questions, files, or answers.

Task and data. GAIA contains 466 real-world assistant questions divided into three difficulty levels and requiring combinations of reasoning, web browsing, files, multimodal inputs, and tools.

Scoring. Accuracy is the share of questions receiving the unambiguous final answer; higher is better for a fixed split and tool-access policy.

What it measures. general-assistant task completion when answers require coordinated reasoning, retrieval, multimodal handling, and external tool use.

What it does not measure. open-ended work quality, safety, long-horizon reliability, cost, latency, enterprise permissions, or autonomous performance on arbitrary computer tasks.

Limitations. System architecture and tool access dominate results, test answers are private, web content changes, and the dataset is deliberately gated to reduce leakage.

Comparison rules. A GAIA score is a system score. Match the 2023 release, split, tool set, browser state, and answer normalization before comparing results.

Access and licensing. The GAIA dataset card forbids resharing validation or test data in a crawlable form and requires gated access. The Compendium reproduces no questions, files, or answers.

Result-reading note. Every result below was published by the named third party and reviewed by the Compendium on 2026-07-13. Flintglade did not run, reproduce, certify, or independently validate these evaluations. A score only belongs to its recorded benchmark version, split, prompt or scaffold, metric, and subject version; omitted setup details are called out instead of guessed.