Welcome to LLM Canary!
The LLM Canary tool is an easy-to-use open-source security benchmarking test suite. This tool empowers developers to test, evaluate, and benchmark LLMs. Developers can distinguish between security trade offs when selecting a model, and are positioned to resolve LLM vulnerabilities before integration. Test groups are based on OWASP Top 10 for LLMs and the tool is designed to be kept up-to-date with the most current and relevant threats.
With the LLM Canary tool, you can:
- Identify potential vulnerabilities based on the OWASP top 10 for LLMs
- Define risk for identified vulnerabilities
- Run multiple LLMs at once to improve speed and effectiveness of tests
- Compare your results to our benchmark or to past runs
- Create custom tests to customize your testing experience and ensure thoroughness of testing.
To get started, please refer to the Quick Start Guide.