Test prompts, models and agents against datasets, assertions and red-team probes, locally or in CI.
Changes in this category (RSS)
| Tool | Stars | Licence | Terms | Self-hosted | Language | Latest | Last push |
|---|---|---|---|---|---|---|---|
| promptfoo | 26k | MIT | Open source | No | TypeScript | code-scan-action-0.2.1 | 2026-10-06 |
| DeepEval | 19k | Apache-2.0 | Open source | No | Python | python-v4.2.4 | 2026-10-05 |
| LM Evaluation Harness | 14k | MIT | Open source | No | Python | v0.4.13 | 2026-09-14 |
| garak | 9.5k | Apache-2.0 | Open source | No | Python | v0.17.0 | 2026-10-02 |
| OpenCompass | 7.5k | Apache-2.0 | Open source | No | Python | 0.5.4 | 2026-09-28 |
| Giskard | 5.9k | Apache-2.0 | Open source | No | Python | v3.0.1 | 2026-10-06 |
| Inspect | 2.9k | MIT | Open source | No | Python | 0.3.276 | 2026-10-06 |
7 tools
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
The LLM Evaluation Framework
A framework for few-shot evaluation of language models.
the LLM vulnerability scanner
OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.
🐢 Open-Source Evaluation & Testing library for LLM Agents
Inspect: A framework for large language model evaluations