Model Evaluation
MMBench
An OpenCompass Community multimodal-understanding benchmark using multiple-choice questions and fine-grained ability dimensions to analyze vision-language models, with Chinese and English data and evaluation-tool links.
Official identity and purpose checked · 2026-10-24
- Plans
- 0
- Official sources
- 1
- Hands-on tests
- 0 not tested here
- Data date
- 2026-10-24
Purpose
An OpenCompass Community multimodal-understanding benchmark using multiple-choice questions and fine-grained ability dimensions to analyze vision-language models, with Chinese and English data and evaluation-tool links.
Features described on the official site
- Ability dimensions cover visual perception and reasoning.
- Describes Chinese and English Dev/Test splits, CCBench and data formats.
- Explains CircularEval option rotation, answer extraction and VLMEvalKit.
Suggested starting workflow
- Select the dataset and version for your language and task.
- Match VLMEvalKit or legacy data formats and scoring protocols.
- Distinguish single-pass and circular evaluation; record models, extractors and all settings.
These are editorial starting suggestions based on product purpose, not workflows tested by this site.
Before you use it
- Not tested hands-on here; feature descriptions come from official material.
- Pricing, regions, languages, commercial use, watermarks and export conditions have not been verified for a single plan.
- Robustness, reproducibility and extraction-success claims are author descriptions, not reproduced here. Evaluation may use an external model to extract answers, with separate costs. Sample images are not copied, and historical tables are not current rankings.
Pricing & plans
Plan pricing has not been checked. Verify it with the provider; a starting price may not cover your task.
Official sources
Sources with the same saved document, review date and validity period are shown together. Expand to see the plans that cite each source. Plan conditions are checked separately; public sources do not establish hands-on task results.
More tools in this category
Explore categories ↗
ArenaOfficial source checked
Arena, formerly LMArena / Chatbot Arena, is a model-output comparison and evaluation platform with preference voting, agent-category performance and cost views, and a Code Arena development mode.
Category: Model Evaluation
C-EvalOfficial source checked
A comprehensive Chinese benchmark for foundation models. The official README describes 13,948 multiple-choice questions across 52 subjects and four difficulty levels.
Category: Model Evaluation
CMMLUOfficial source checked
A Chinese multitask language-understanding benchmark covering 67 subjects for analysis of knowledge, reasoning and China-specific contexts.
Category: Model Evaluation
DataifyOfficial source checked
Dataify is an AI data-services platform. Its official site describes datasets for training, evaluation and RAG, webpage/search/video data APIs and network resources, primarily supplying data for models and applications.
Category: AI Models
FlagEvalOfficial source checked
FlagEval is an evaluation toolkit and open platform for large models. Its public project covers foundation models, training and fine-tuning algorithms, with language, vision, audio and multimodal evaluation directions.
Category: Model Evaluation
HELMOfficial source checked
Stanford CRFM’s open-source holistic model-evaluation framework, unifying datasets, model interfaces and multiple metrics. Its README states that it entered maintenance mode on 2026-06-01.
Category: Model Evaluation