GAIA
GAIA evaluates general-assistant task completion when answers require coordinated reasoning, retrieval, multimodal handling, and external tool use.
GAIA GAIA 2023 paper release
Permanent entry- Task and data
- GAIA contains 466 real-world assistant questions divided into three difficulty levels and requiring combinations of reasoning, web browsing, files, multimodal inputs, and tools.
- Primary metric
- Accuracy is the share of questions receiving the unambiguous final answer; higher is better for a fixed split and tool-access policy.
- What it measures
- general-assistant task completion when answers require coordinated reasoning, retrieval, multimodal handling, and external tool use.
- Comparison rules
- A GAIA score is a system score. Match the 2023 release, split, tool set, browser state, and answer normalization before comparing results.
- Access and licensing
- The GAIA dataset card forbids resharing validation or test data in a crawlable form and requires gated access. The Compendium reproduces no questions, files, or answers.
| Published subject | Score | Metric | Split and evaluation setting | Source record |
|---|---|---|---|---|
| Human respondentsgaia-2023:paper:human | 92%higher, percent | accuracyhistorical human baseline | GAIA paper evaluationHuman evaluation described in the paper; per-participant and tool-condition detail is not restated in the abstract.This is a paper-level human anchor, not an unrestricted population estimate. | GAIA authorsPublished 2023-11-21, reviewed 2026-07-13human-baseline, published-third-party |
| GPT-4 equipped with pluginsgaia-2023:paper:gpt4-plugins | 15%higher, percent | accuracyhistorical system result | GAIA paper evaluationGPT-4 system with plugins; exact model snapshot, plugin inventory, and all execution parameters are not stated in the abstract.The missing exact plugin and model setup is explicit; this is not a base GPT-4 score. | GAIA authorsPublished 2023-11-21, reviewed 2026-07-13original-paper, published-third-party |
Claims and evidence
Claim ledger, 2GAIA contains 466 real-world assistant questions divided into three difficulty levels and requiring combinations of reasoning, web browsing, files, multimodal inputs, and tools.
The listed result anchors are third-party publications tied to explicit evaluation settings and are not evaluations run by Draconitic.