Inspect vs OpenCompass

Inspect vs OpenCompass
Fact	Inspect	OpenCompass
Language	Python	Python
Licence	MIT	Apache-2.0
Stars	3k	7.5k
Latest	0.3.281	0.5.4
Last push	2026-10-17	2026-10-16
Release cadence	about 2 days between releases	about 89 days between releases
Active contributors	81+ commit authors on the default branch in the last 90 days	26 commit authors on the default branch in the last 90 days
Flags	none	none

How they relate

Both replace LM Evaluation Harness. Alternatives to LM Evaluation Harness →
PartialInspectRuns benchmark tasks, but is built around custom evals and agents.
Full replacementOpenCompassRuns benchmark suites over local and API models from a config.

0.3.2812026-10-15v0.3.281
0.3.2802026-10-14v0.3.280
0.3.2792026-10-12v0.3.279
0.3.2782026-10-09Tasks and samples can carry a description of what they ask of the agent, recorded in the eval log and in evals_df() and…

0.5.42026-08-26✨ Multimodal Evaluation with VLMEvalKit: OpenCompass now integrates VLMEvalKit for native multimodal dataset loading, O…
0.5.32026-06-29✨ 🧪 Extensive New Benchmarks and Dataset Support: Added support for a wide range of new benchmarks and datasets, inclu…
0.5.22026-02-14✨ 🧪 Extensive New Benchmarks Support: We have introduced comprehensive support for Scientific and General Benchmarks,…
0.5.1.post12025-10-17✨ A New Method to quickly Integrate and Evaluate Your Datasets: Added a fast dataset integration and evaluation method…
0.5.02025-09-01✨ Comprehensive Scientific Benchmarks: Integrated 10+ specialized datasets (MedXpertQA, ClimaQA, SmolInstruct, etc.),…

✓ signed The latest release, 0.3.281, carries a signature GitHub verified.

Loading the security report

✓ signed The latest release, 0.5.4, carries a signature GitHub verified.

Loading the security report