Promptfoo

Promptfoo is an open-source CLI and library for evaluating and red-teaming LLM applications. It replaces trial-and-error prompting with repeatable test suites: define your prompts, models, and assertions in simple declarative configs, then measure quality and security before you ship. Promptfoo has over 26,000 stars on GitHub, is used by teams at major AI labs, and was actively developed in October 2026.

Overview

Promptfoo covers the two questions every LLM team faces: does it work well, and can it be broken? On the evaluation side, it compares models such as GPT, Claude, Gemini, and DeepSeek side by side, scores outputs with metrics and model-graded assertions, and plugs into CI/CD so regressions fail the build. On the security side, its red-teaming tools generate adversarial tests for prompt injection, jailbreaks, data leakage, and other vulnerabilities, giving agents and RAG systems a proper security workout before production.

Key Features

• Declarative eval configs for prompts, agents, and RAG pipelines
• Side-by-side model and prompt comparison with scoring
• Automated red teaming for injections, jailbreaks, and leaks
• Works with all major model providers and local models
• CI/CD integration to catch regressions before release
• Command-line and library usage for flexible workflows
• Open source under the MIT license

💳 Pricing

Promptfoo’s open-source CLI and library are free under the MIT license. A hosted cloud offering with team collaboration and reporting features is available for organizations that outgrow the local tool, making it a freemium product overall.

Who Is It For

Promptfoo is for AI engineers, LLM app developers, and security-minded teams who need evidence that their prompts and agents behave. If you ship LLM features to real users, it gives you the testing discipline that traditional software teams take for granted.