Garak

Garak is NVIDIA’s open-source vulnerability scanner for large language models, available on GitHub with more than 9,600 stars and active development through 2026. Think of it as a security audit toolkit that systematically probes an LLM or chatbot for weaknesses before attackers do.

Overview

Named after a famously untrustworthy tailor, garak works much like a network vulnerability scanner, but for language models. It sends crafted probes to a target model, covering families such as prompt injection, jailbreaks, data leakage, hallucination induction, toxicity elicitation and encoding-based bypasses, then uses detectors to judge whether the model’s responses reveal a failure. Results are written up as structured reports, so security and ML teams can track risk over model versions and compare configurations objectively instead of relying on ad hoc red-teaming.

Key Features

• A large, growing library of probes across many vulnerability families, from DAN-style jailbreaks to leak and injection tests
• Automatic detectors that grade model responses and flag likely failures
• Works against REST APIs, Hugging Face models, local models and popular hosted services
• Structured reporting in JSON and other formats for audits and regression tracking
• Extensible plugin design: add your own probes, detectors, generators and harnesses in Python
• Maintained by NVIDIA with regular updates tracking new attack research

💳 Pricing

Garak is free and open source under the Apache License 2.0. There is no paid version; the only costs are compute and any API charges for the models you choose to test.

Who Is It For

Garak is built for AI security engineers, red teams and ML platform teams who need repeatable evidence about how a model behaves under attack. It fits naturally into pre-deployment evaluation and ongoing model governance, whether you are shipping a customer-facing chatbot or validating a vendor’s model before procurement.