Model Evaluation
FlagEval
FlagEval is an evaluation toolkit and open platform for large models. Its public project covers foundation models, training and fine-tuning algorithms, with language, vision, audio and multimodal evaluation directions.
Official identity and purpose checked · 2026-10-24
- Plans
- 0
- Official sources
- 1
- Hands-on tests
- 0 not tested here
- Data date
- 2026-10-24
Purpose
FlagEval is an evaluation toolkit and open platform for large models. Its public project covers foundation models, training and fine-tuning algorithms, with language, vision, audio and multimodal evaluation directions.
Features described on the official site
- mCLIPEval supports classification, retrieval and composition tasks for vision-language models.
- ImageEval-prompt organizes text-to-image evaluation prompts by entities, style and details.
- C-SEM evaluates Chinese semantic understanding at lexical and sentence levels.
Suggested starting workflow
- Choose the subproject for your evaluation target.
- Check data, dependencies, versions and subproject licenses.
- Fix evaluation settings and analyze detailed results, then validate with task samples.
These are editorial starting suggestions based on product purpose, not workflows tested by this site.
Before you use it
- Not tested hands-on here; feature descriptions come from official material.
- Pricing, regions, languages, commercial use, watermarks and export conditions have not been verified for a single plan.
- Fairness, objectivity and efficiency goals have not been tested here. The main project license does not automatically cover data or every component. The public project does not establish full platform availability, current rankings or task qualification.
Pricing & plans
Plan pricing has not been checked. Verify it with the provider; a starting price may not cover your task.
Official sources
Sources with the same saved document, review date and validity period are shown together. Expand to see the plans that cite each source. Plan conditions are checked separately; public sources do not establish hands-on task results.
More tools in this category
Explore categories ↗
ArenaOfficial source checked
Arena, formerly LMArena / Chatbot Arena, is a model-output comparison and evaluation platform with preference voting, agent-category performance and cost views, and a Code Arena development mode.
Category: Model Evaluation
C-EvalOfficial source checked
A comprehensive Chinese benchmark for foundation models. The official README describes 13,948 multiple-choice questions across 52 subjects and four difficulty levels.
Category: Model Evaluation
CMMLUOfficial source checked
A Chinese multitask language-understanding benchmark covering 67 subjects for analysis of knowledge, reasoning and China-specific contexts.
Category: Model Evaluation
DataifyOfficial source checked
Dataify is an AI data-services platform. Its official site describes datasets for training, evaluation and RAG, webpage/search/video data APIs and network resources, primarily supplying data for models and applications.
Category: AI Models
HELMOfficial source checked
Stanford CRFM’s open-source holistic model-evaluation framework, unifying datasets, model interfaces and multiple metrics. Its README states that it entered maintenance mode on 2026-06-01.
Category: Model Evaluation
MMBenchOfficial source checked
An OpenCompass Community multimodal-understanding benchmark using multiple-choice questions and fine-grained ability dimensions to analyze vision-language models, with Chinese and English data and evaluation-tool links.
Category: Model Evaluation