CodeGate: Security, Workspaces and Muxing for AI Applications, coding assistants, and agentic frameworks.

Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪

the LLM vulnerability scanner

The Security Toolkit for LLM Interactions

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

Ancillary open source software to support confidential computing on NVIDIA GPUs

CLI tool for creating GitHub issues from Snyk project issues

a prompt injection scanner for custom LLM applications

Set of tools to assess and improve LLM security.

This repository provides implementation to formalize and benchmark Prompt Injection attacks and defenses

The jailbreak-evaluation is an easy-to-use Python package for language model jailbreak evaluation.

I leverage OpenAI and ChatGPT to do hackerish things

Make your GenAI Apps Safe & Secure 🚀 Test & harden your system prompt

The automated prompt injection framework for LLM-integrated applications.

LLM Prompt Injection Detector

Universal and Transferable Attacks on Aligned Language Models

Whistleblower is a offensive security tool for testing against system prompt leakage and capability discovery of an AI application exposed through API. Built for AI engineers, security researchers …

source for llmsec.net

source code for the offsecml framework

Dropbox LLM Security research code and results

PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. 🏆 Best Paper Awards @ NeurIPS ML …

🧠 LLMFuzzer - Fuzzing Framework for Large Language Models 🧠 LLMFuzzer is the first open-source fuzzing framework specifically designed for Large Language Models (LLMs), especially for their integra…

⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs

An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.

A toolkit for detecting and protecting against vulnerabilities in Large Language Models (LLMs).

BeaverTails is a collection of datasets designed to facilitate research on safety alignment in large language models (LLMs).

a CLI that provides a generic automation layer for assessing the security of ML models