Model Evaluation
MMLU
The original benchmark and code repository for Measuring Massive Multitask Language Understanding, using multidisciplinary tests to analyze model knowledge and understanding. Its paper appeared at ICLR 2021.
Official identity and purpose checked · 2026-10-24
- Plans
- 0
- Official sources
- 1
- Hands-on tests
- 0 not tested here
- Data date
- 2026-10-24
Purpose
The original benchmark and code repository for Measuring Massive Multitask Language Understanding, using multidisciplinary tests to analyze model knowledge and understanding. Its paper appeared at ICLR 2021.
Features described on the official site
- Test areas include humanities, social sciences, STEM and other subjects.
- Links to the paper, test data and model-evaluation code.
- The README includes a results table and submission guidance.
Suggested starting workflow
- Confirm that you need original MMLU rather than MMLU-Pro or another variant.
- Fix data splits, model versions and few-shot settings.
- Retain subject results and errors; add task evaluation and contamination analysis.
These are editorial starting suggestions based on product purpose, not workflows tested by this site.
Before you use it
- Not tested hands-on here; feature descriptions come from official material.
- Pricing, regions, languages, commercial use, watermarks and export conditions have not been verified for a single plan.
- The discovered leaderboard URL redirected and failed to load; this entry relies on the original project. The README retains historical model results, not a current 2026 ranking. MMLU-Pro is not mixed in, and API evaluation code was not run.
Pricing & plans
Plan pricing has not been checked. Verify it with the provider; a starting price may not cover your task.
Official sources
Sources with the same saved document, review date and validity period are shown together. Expand to see the plans that cite each source. Plan conditions are checked separately; public sources do not establish hands-on task results.
More tools in this category
Explore categories ↗
ArenaOfficial source checked
Arena, formerly LMArena / Chatbot Arena, is a model-output comparison and evaluation platform with preference voting, agent-category performance and cost views, and a Code Arena development mode.
Category: Model Evaluation
C-EvalOfficial source checked
A comprehensive Chinese benchmark for foundation models. The official README describes 13,948 multiple-choice questions across 52 subjects and four difficulty levels.
Category: Model Evaluation
CMMLUOfficial source checked
A Chinese multitask language-understanding benchmark covering 67 subjects for analysis of knowledge, reasoning and China-specific contexts.
Category: Model Evaluation
DataifyOfficial source checked
Dataify is an AI data-services platform. Its official site describes datasets for training, evaluation and RAG, webpage/search/video data APIs and network resources, primarily supplying data for models and applications.
Category: AI Models
FlagEvalOfficial source checked
FlagEval is an evaluation toolkit and open platform for large models. Its public project covers foundation models, training and fine-tuning algorithms, with language, vision, audio and multimodal evaluation directions.
Category: Model Evaluation
HELMOfficial source checked
Stanford CRFM’s open-source holistic model-evaluation framework, unifying datasets, model interfaces and multiple metrics. Its README states that it entered maintenance mode on 2026-06-01.
Category: Model Evaluation