Benchmarks
19 entries
| Entry | Summary | As of |
|---|---|---|
| ARC-AGI-2benchmark | ARC-AGI-2 evaluates few-shot abstraction, rule induction, and exact generalization on unfamiliar visual grid transformations under the competition's resource constraints. | 2026-07-13 |
| GAIAbenchmark | GAIA evaluates general-assistant task completion when answers require coordinated reasoning, retrieval, multimodal handling, and external tool use. | 2026-07-13 |
| GPQAbenchmark | GPQA evaluates graduate-level scientific question answering and the gap between domain experts, skilled outsiders, and model baselines under specified resource conditions. | 2026-10-26 |
| GSM8Kbenchmark | GSM8K evaluates multi-step grade-school mathematical word-problem solving and, in the original work, the value of ranking generated solutions with a trained verifier. | 2026-07-13 |
| HarmBenchbenchmark | HarmBench evaluates comparative robustness of model defenses and effectiveness of red-team methods on the benchmark's specified harmful behavior classes. | 2026-07-13 |
| HumanEvalbenchmark | HumanEval evaluates small, self-contained Python code generation from natural-language function specifications under functional tests. | 2026-10-26 |
| Humanity's Last Exam — static releasebenchmark | Humanity's Last Exam — static release evaluates performance on a deliberately difficult, broad set of expert-level closed-ended academic questions, including multimodal items. | 2026-07-13 |
| IFEvalbenchmark | IFEval evaluates whether model responses satisfy automatically checkable natural-language constraints under the benchmark's instruction inventory. | 2026-07-13 |
| LiveCodeBenchbenchmark | LiveCodeBench evaluates coding performance on recently released contest problems and several adjacent code-reasoning scenarios with time-window analysis for possible contamination. | 2026-07-13 |
| LongBench v2benchmark | LongBench v2 evaluates deep understanding and reasoning over long single documents, multiple documents, code repositories, dialogues, structured data, and in-context learning material. | 2026-07-13 |
| MATHbenchmark | MATH evaluates competition-style mathematical problem solving across algebra, geometry, number theory, counting and probability, and related school-level areas. | 2026-07-13 |
| MMLUbenchmark | MMLU evaluates broad text-only knowledge recall and problem solving across the sampled school, university, and professional domains. | 2026-07-13 |
| MMLU-Probenchmark | MMLU-Pro evaluates more discriminative knowledge-and-reasoning performance under the paper's ten-option, curated multiple-choice construction. | 2026-07-13 |
| MMMUbenchmark | MMMU evaluates college-level multimodal perception plus domain knowledge and reasoning across art, business, science, medicine, humanities, and engineering. | 2026-07-13 |
| MTEB v1benchmark | MTEB v1 evaluates how one text-embedding model transfers across several retrieval, similarity, classification, clustering, reranking, and related English tasks. | 2026-07-13 |
| OSWorldbenchmark | OSWorld evaluates multimodal computer-agent ability to perceive interfaces and execute mouse, keyboard, and code actions across real desktop applications. | 2026-07-13 |
| SWE-Bench Probenchmark | SWE-Bench Pro evaluates agentic repository work on larger, more diverse, and less easily memorized software tasks than the original SWE-bench family. | 2026-07-13 |
| SWE-bench Verified — historical releasebenchmark | SWE-bench Verified — historical release evaluates agent-plus-model ability to navigate Python repositories and produce test-passing patches for selected real-world issues. | 2026-07-13 |
| TruthfulQAbenchmark | TruthfulQA evaluates resistance to producing selected common falsehoods when answering short questions, with a deliberate distinction between truth and informativeness. | 2026-07-13 |