Hacktor Release v0.8 - Comprehensive Prompt Injection Rules

This release introduces significant upgrades to Hacktor, our powerful LLM security testing tool, with new datasets, improved performance, and easier accessibility.

Key Features:

• Integrated Hackprompt and LLM-Rules datasets: Leverage the latest adversarial prompts and attack patterns from these cutting-edge datasets to uncover vulnerabilities in your LLMs.
• Upgraded Evaluation Service: Experience faster and more robust evaluations with our redesigned architecture, optimized for performance and scalability.
• Enhanced Parsing Capabilities: Handle large tokens and complex inputs efficiently, ensuring accurate and comprehensive testing.
• Simplified Upgrade Process: Update Hacktor with ease using either git and poetry or by pulling the latest Docker image.

Detoxio.AI.Hacktor.v0.8.Preview.mp4

Upgrade Instructions:

Using Git and Poetry:

git pull origin main@v0.8
poetry install

Using Docker:

docker pull detoxio/hacktor:latest

Usage Examples:

Testing with the HACKPROMPT module:

export DETOXIO_API_KEY=xxxx

hacktor llm HF_SPACE https://huggingface.co/spaces/detoxioai/Pokebot \
    --attack_module HACKPROMPT \
    --no_of_tests 10 -v

Using Docker with API Key:

Note Make sure to replace xxxx with Detoxio API Key

docker run --rm -e DETOXIO_API_KEY=xxxxx \
    detoxio/hacktor:latest llm HF_SPACE https://huggingface.co/spaces/detoxioai/Pokebot \
    --attack_module HACKPROMPT \
    --no_of_tests 10 -v

Example Hackor Output

Get Started Today:

Upgrade to Hacktor v0.8 and enhance your LLM security testing with the latest advancements in adversarial attacks.

Stay Connected:

• Follow us on https://www.linkedin.com/company/detoxio-ai/
• Report issues and contribute on https://github.com/detoxio-ai/hacktor/issues/new

Red Teaming LLM deployed as Hugging Face Spaces

Enhancement:

Added support for scanning Hugging Face Spaces (HF_SPACE) URLs as models in Hacktor.

Description:

Currently, Hacktor doesn't support scanning models hosted on Hugging Face Spaces. This enhancement will enable Hacktor to directly interact with and assess the security of models deployed on HF_SPACE.

Proposed Solution:

Implement functionality to parse HF_SPACE URLs, extract the underlying model, and perform security scans using the specified parameters (e.g., --model_id, --attack_module, etc.).

Usage

Install or upgrade hacktor

git

git pull origin main
poetry install

docker

docker pull docker.io/detoxio/hacktor:latest

Export API Key

export DETOXIO_API_KEY=xxx

Run it

Example Command using installed hacktor:

hacktor llm HF_SPACE https://3b9822dd204a5d2b8e.gradio.live/ --model_id "google/gemma-2-2b-it" --use_ai --max_crawling_steps 1 --attack_module ADVBENCH --no_of_tests 10 -v

Example Command using docker:

docker run --rm -e DETOXIO_API_KEY=$DETOXIO_API_KEY  docker.io/detoxio/hacktor:latest llm HF_SPACE https://b9371c32552ab2dda6.gradio.live  --model_id "google/gemma-2-2b-it"  --use_ai   --max_crawling_steps 1   --attack_module ADVBENCH   --no_of_tests 10 -v

URL (https://3b9822dd204a5d2b8e.gradio.live/): This points to the specific Hugging Face Space where the model is hosted and running. It's like the web address of the application that uses the model. This is necessary because models on HF Spaces are often integrated into interactive demos or applications.

model_id (google/gemma-2-2b-it): This is the identifier of the underlying language model itself. Think of it like the model's official name. It tells Hacktor which pre-trained model architecture and weights are being used within that Hugging Face Space. This ID is important for categorizing, analyzing the model's properties, and displaying it meaningfully in the portal.

v0.3.0

Hacktor v0.3.0 Release Notes

What's New

HTTP Request Fuzzing (Feat #9)

• Added support for fuzzing HTTP requests recorded using Burp Suite.
• Users can now perform automated testing and fuzzing of GenAI applications using Burp-recorded requests.
• The tool automatically detects fuzz markers like [[FUZZ]] and generates test cases to evaluate the robustness of the application.
• This feature enhances Hacktor’s ability to integrate with existing testing workflows and provides a seamless way to test APIs recorded in Burp Suite.

New CLI Command:

• Introduced the burp subcommand for handling Burp Suite request files.

  Example Usage:

  poetry run hacktor burp <URL> -r <Request file path>

Full Changelog:

For a comprehensive list of changes, please visit: Full Changelog

New Contributors:

• @jc-detoxio made their first contribution in #10