MTEB v1 history

Task and data. MTEB v1 spans eight embedding task types; the paper's English aggregate covers 56 datasets across classification, clustering, pair classification, reranking, retrieval, semantic similarity, and summarization.

Scoring. Each task type has its own principal metric; the paper's overall score averages task-level main metrics on a 0–100 scale. Higher is better within the fixed dataset suite.

What it measures. how one text-embedding model transfers across several retrieval, similarity, classification, clustering, reranking, and related English tasks.

Comparison rules. These are MTEB v1 paper aggregates over 56 English datasets. Do not merge them with current MTEB leaderboard variants, multilingual suites, or different task weighting.

Access and licensing. The benchmark framework and each underlying dataset can have different licenses. The Compendium reproduces no examples or dataset content.

Task and data. MTEB v1 spans eight embedding task types; the paper's English aggregate covers 56 datasets across classification, clustering, pair classification, reranking, retrieval, semantic similarity, and summarization.

Scoring. Each task type has its own principal metric; the paper's overall score averages task-level main metrics on a 0–100 scale. Higher is better within the fixed dataset suite.

What it measures. how one text-embedding model transfers across several retrieval, similarity, classification, clustering, reranking, and related English tasks.

Comparison rules. These are MTEB v1 paper aggregates over 56 English datasets. Do not merge them with current MTEB leaderboard variants, multilingual suites, or different task weighting.

Access and licensing. The benchmark framework and each underlying dataset can have different licenses. The Compendium reproduces no examples or dataset content.

Task and data. MTEB v1 spans eight embedding task types; the paper's English aggregate covers 56 datasets across classification, clustering, pair classification, reranking, retrieval, semantic similarity, and summarization.

Scoring. Each task type has its own principal metric; the paper's overall score averages task-level main metrics on a 0–100 scale. Higher is better within the fixed dataset suite.

What it measures. how one text-embedding model transfers across several retrieval, similarity, classification, clustering, reranking, and related English tasks.

What it does not measure. generation, factuality, multilingual performance from the English aggregate, production latency, memory, price, domain-specific retrieval, or fairness and safety.

Limitations. Averaging unlike task metrics hides tradeoffs, training-data overlap can affect results, dataset availability changes, and current MTEB has expanded beyond v1.

Comparison rules. These are MTEB v1 paper aggregates over 56 English datasets. Do not merge them with current MTEB leaderboard variants, multilingual suites, or different task weighting.

Access and licensing. The benchmark framework and each underlying dataset can have different licenses. The Compendium reproduces no examples or dataset content.

Result-reading note. Every result below was published by the named third party and reviewed by the Compendium on 2026-07-13. Flintglade did not run, reproduce, certify, or independently validate these evaluations. A score only belongs to its recorded benchmark version, split, prompt or scaffold, metric, and subject version; omitted setup details are called out instead of guessed.