Our Features

Direct focus on LLM security

garak focuses primarily on LLM security. While other tools might look at generic machine learning security, or app security, garak specifically focuses on risks that are inherent in and unique to LLM deployment, such as prompt injection, jailbreaks, guardrail bypass, text replay, and so on.

Automated scanning

garak has a range of probes but doesn't need supervision - it will run each of these over the model, and manage things like finding appropriate detectors and handling rate limiting itself. You can override many aspects of the config to get a custom scan, but out of the box, it can do a full standard scan and report without intervention.

Connect to many different LLMs

garak supports a ton of LLMs - including OpenAI, Hugging Face, Cohere, Replicate - as well as custom Python integrations. It's a community project, so even more LLM support is always coming.

Structured reporting

garak keeps track of everything found, and outputs four kinds of log:

  1. Screen output - useful for monitoring scan progress; a precise description of what's happening at any time during the scan, including a list of everything on the schedule

  2. Report log - detailing the run down to every single prompt, response, and the evaluation of that response

  3. Hit log - describing each time that garak managed to get through and find a vulnerability

  4. Debug log - a logfile for troubleshooting and keeping track of garak's operations

Last updated