Skip to content

Releases: tophant-ai/promptbeat

v0.1.2

31 May 17:06
@holtmiu holtmiu
v0.1.2
a65f874

Choose a tag to compare

View all tags
v0.1.2 Latest
Latest

promptbeat v0.1.2

Red-teaming toolkit for LLM safety evaluation. Generate adversarial test cases from curated seeds and evaluate model robustness.

Features

  • Seed-driven red-team test generation via promptfoo backend
  • Multi-provider evaluation with side-by-side comparison
  • Attack Success Rate (ASR) metrics by risk category and provider
  • HTML report generation with interactive filtering
  • Self-contained binary — no Python/Node.js installation required

Installation

Download the tarball for your platform, extract, and run:

tar xf promptbeat-0.1.0-<platform>.tar.gz
./promptbeat-0.1.0-<platform>/bin/promptbeat --version

Available platforms

File Platform
promptbeat-0.1.0-darwin-arm64.tar.gz macOS Apple Silicon (M1/M2/M3/M4)
promptbeat-0.1.0-darwin-x64.tar.gz macOS Intel
promptbeat-0.1.0-linux-x64.tar.gz Linux x86_64

Quick Start

export OPENAI_API_KEY="sk-..."

# Generate adversarial test cases
./bin/promptbeat generate \
  --target target.yaml \
  --scenarios scenarios.yaml \
  --seed-file seeds.yaml \
  --provider openai:gpt-4o \
  --count 5

# Evaluate
./bin/promptbeat eval \
  --config artifacts/generate/generated_redteam.yaml

# Generate HTML report
./bin/promptbeat report \
  --eval-result artifacts/eval/generated_redteam/evaluation_result.json

Requirements

  • OPENAI_API_KEY or other LLM provider API key
  • No other dependencies — Python, Node.js, and promptfoo are bundled

Bundled Components

Component Version
Python 3.12.13
Node.js 22.22.2
promptfoo 0.121.9

Deployment

Build the image:

./deploy/buildImage.sh -v 1.0.0 -s harbor.tophant.com -p tophant

Run the server image with runtime env from api/.env.server:

./deploy/runImage.sh -i harbor.tophant.com/tophant/promptbeat-api:1.0.0
Assets 5
Loading

v0.1.0

09 May 14:35
@holtmiu holtmiu
v0.1.0
1b1c912

Choose a tag to compare

View all tags
v0.1.0

promptbeat v0.1.0

Red-teaming toolkit for LLM safety evaluation. Generate adversarial test cases from curated seeds and evaluate model robustness.

Features

  • Seed-driven red-team test generation via promptfoo backend
  • Multi-provider evaluation with side-by-side comparison
  • Attack Success Rate (ASR) metrics by risk category and provider
  • HTML report generation with interactive filtering
  • Self-contained binary — no Python/Node.js installation required

Installation

Download the tarball for your platform, extract, and run:

tar xf promptbeat-0.1.0-<platform>.tar.gz
./promptbeat-0.1.0-<platform>/bin/promptbeat --version

Available platforms

File Platform
promptbeat-0.1.0-darwin-arm64.tar.gz macOS Apple Silicon (M1/M2/M3/M4)
promptbeat-0.1.0-darwin-x64.tar.gz macOS Intel
promptbeat-0.1.0-linux-x64.tar.gz Linux x86_64

Quick Start

export OPENAI_API_KEY="sk-..."

# Generate adversarial test cases
./bin/promptbeat generate \
  --target target.yaml \
  --scenarios scenarios.yaml \
  --seed-file seeds.yaml \
  --provider openai:gpt-4o \
  --count 5

# Evaluate
./bin/promptbeat eval \
  --config artifacts/generate/generated_redteam.yaml

# Generate HTML report
./bin/promptbeat report \
  --eval-result artifacts/eval/generated_redteam/evaluation_result.json

Requirements

  • OPENAI_API_KEY or other LLM provider API key
  • No other dependencies — Python, Node.js, and promptfoo are bundled

Bundled Components

Component Version
Python 3.12.13
Node.js 22.22.2
promptfoo 0.121.9
Loading