HELM

Stanford CRFM’s open-source holistic model-evaluation framework, unifying datasets, model interfaces and multiple metrics. Its README states that it entered maintenance mode on 2026-06-01.

Features described on the official site

• Standardizes benchmark datasets and interfaces to different model providers.
• Includes metrics beyond accuracy, such as efficiency, bias and toxicity.
• Provides interfaces for individual prompts and responses, leaderboards and reproduction documentation.

Suggested starting workflow

• Read the maintenance-mode policy and confirm the needed adapter remains suitable.
• Fix benchmark, model, suite and metric configurations.
• Retain raw outputs and metrics; verify external-model and compute costs separately.

These are editorial starting suggestions based on product purpose, not workflows tested by this site.

Before you use it

• Not tested hands-on here; feature descriptions come from official material.
• Pricing, regions, languages, commercial use, watermarks and export conditions have not been verified for a single plan.
• Maintenance mode does not mean closure, but update and support scopes need policy review. Leaderboards and metrics do not establish task results, compliance or safety certification. The framework was not installed, benchmarks run or provider models called.