CMMLU

A Chinese multitask language-understanding benchmark covering 67 subjects for analysis of knowledge, reasoning and China-specific contexts.

Features described on the official site

• Subjects include natural sciences, humanities, social sciences and everyday knowledge.
• The repository links to datasets, papers and evaluation implementations.
• Results distinguish zero-shot and five-shot settings and describe instruction-following and contamination checks for some models.

Suggested starting workflow

• Select subjects and splits relevant to your Chinese-language task.
• Fix prompts and model configurations; retain item outputs and scoring settings.
• Analyze error types and add task samples outside the training data.

These are editorial starting suggestions based on product purpose, not workflows tested by this site.

Before you use it

• Not tested hands-on here; feature descriptions come from official material.
• Pricing, regions, languages, commercial use, watermarks and export conditions have not been verified for a single plan.
• Contamination checks and leaderboard results have not been independently reviewed here. Chinese subject scores do not establish all-language or task performance; data and rankings are not interchangeable with C-Eval or MMLU.