Rogue is an open-source AI agent evaluator and red team platform from Qualifire, published on GitHub with about 1,070 stars and actively developed into 2026. Its pitch is simple: stress-test your AI agents before attackers do. Rogue probes agents systematically to surface security vulnerabilities and reliability flaws before they reach production.
Rogue works in two modes. In automatic evaluation mode, it tests an agent against defined business policies and expected behaviors, running scenarios and producing detailed pass/fail reports with reasoning. In red team mode, it simulates adversarial attacks using more than 75 known vulnerabilities across 12 security categories and around 20 attack techniques, including encoding tricks, social engineering, and prompt injection. Findings are scored with CVSS-based risk ratings and mapped to eight compliance frameworks, including OWASP, MITRE, NIST, GDPR, and the EU AI Act. A client-server architecture with multiple interfaces lets teams watch live conversations as Rogue probes their agent.
Rogue meets testers where they work. It offers a modern terminal UI built in Go, a non-interactive CLI mode designed for CI/CD pipelines, and an MCP interface so other agents and tools can drive evaluations programmatically. It can test agents exposed over common agent protocols as well as agents implemented as plain Python functions, which makes it practical to add agent security testing to an existing development workflow rather than bolting it on afterwards.
Rogue itself is free and open source. It is developed by Qualifire, which sells a commercial AI reliability platform adding real-time guardrails, continuous evaluation, and observability, so the listing is marked Freemium.
Teams shipping LLM agents, RAG systems, or chatbots who need pre-production security audits, regression testing, and compliance-ready evidence that their agent behaves safely.