MTEB v1
MTEB v1 evaluates how one text-embedding model transfers across several retrieval, similarity, classification, clustering, reranking, and related English tasks.
MTEB v1 MTEB paper v3; 56 English datasets
Permanent entry- Task and data
- MTEB v1 spans eight embedding task types; the paper's English aggregate covers 56 datasets across classification, clustering, pair classification, reranking, retrieval, semantic similarity, and summarization.
- Primary metric
- Each task type has its own principal metric; the paper's overall score averages task-level main metrics on a 0–100 scale. Higher is better within the fixed dataset suite.
- What it measures
- how one text-embedding model transfers across several retrieval, similarity, classification, clustering, reranking, and related English tasks.
- Comparison rules
- These are MTEB v1 paper aggregates over 56 English datasets. Do not merge them with current MTEB leaderboard variants, multilingual suites, or different task weighting.
- Access and licensing
- The benchmark framework and each underlying dataset can have different licenses. The Compendium reproduces no examples or dataset content.
| Published subject | Score | Metric | Split and evaluation setting | Source record |
|---|---|---|---|---|
| ST5-XXLmteb-v1:english-56:paper-average | 59.51higher, benchmark points | average of task main metricshistorical paper result | 56 English datasetsPaper Table 1 aggregate; task-specific main metrics averaged across the English subsets.The aggregate mixes task-specific metrics and is not a percentage of examples correct. | MTEB authorsPublished 2023-03-19, reviewed 2026-07-13original-paper, published-third-party |
| GTR-XXLmteb-v1:english-56:paper-average | 58.97higher, benchmark points | average of task main metricshistorical paper result | 56 English datasetsPaper Table 1 aggregate; task-specific main metrics averaged across the English subsets.Task-level tradeoffs are hidden by the overall average. | MTEB authorsPublished 2023-03-19, reviewed 2026-07-13original-paper, published-third-party |
| SGPT-5.8B-msmarcomteb-v1:english-56:paper-average | 58.81higher, benchmark points | average of task main metricshistorical paper result | 56 English datasetsPaper Table 1 aggregate; task-specific main metrics averaged across the English subsets.Training on MS MARCO can affect retrieval-oriented performance and comparability. | MTEB authorsPublished 2023-03-19, reviewed 2026-07-13original-paper, published-third-party |
| OpenAI Ada Similaritymteb-v1:english-56:paper-average | 49.52higher, benchmark points | average of task main metricshistorical paper result | 56 English datasetsPaper Table 1 aggregate; exact provider snapshot beyond the published model label is not stated.This historical provider label is not interchangeable with later OpenAI embedding models. | MTEB authorsPublished 2023-03-19, reviewed 2026-07-13original-paper, published-third-party |
Claims and evidence
Claim ledger, 2MTEB v1 spans eight embedding task types; the paper's English aggregate covers 56 datasets across classification, clustering, pair classification, reranking, retrieval, semantic similarity, and summarization.
The listed result anchors are third-party publications tied to explicit evaluation settings and are not evaluations run by Draconitic.