Release v0.2: Add configurable agent security testing pipeline

[purcl-cybersec-agent]

🎉 ASTRA v0.2 Release

This release adds a complete, configurable pipeline for generating security test cases to evaluate AI coding agents' vulnerability to adversarial prompts.

🆕 What's New in v0.2

🔄 Agent Security Testing Pipeline

A complete end-to-end system for generating, parsing, and curating adversarial test cases:

• Multi-agent orchestration: Coordinator, composer, and reviewer committee
• Knowledge graph-driven: Systematic coverage of security domains and attack techniques
• Configurable & extensible: YAML-based configuration, no code changes needed
• Production-ready: CLI tools, error handling, resume capability

📦 New Components

Scripts

• agent/main_agent_sec.py - Generate test cases with parallel processing
• agent/parse_agent_sec.py - Parse and validate XML outputs
• agent/upload_to_hf_dataset.py - Upload to HuggingFace datasets
• agent/agent_sec_composer/output_parser.py - Pydantic-based validation

Configuration System

• resources/agent-sec-config.yaml - Pipeline settings

  • Model selection for coordinator, composer, reviewers
  • Parallel task limits for rate control
  • Output/log directory paths

• resources/client-config.yaml - Enhanced with provider-based routing

  • provider: bedrock for AWS Bedrock models
  • provider: openai for OpenAI-compatible servers (vLLM, SGLang)
  • Pre-configured models: Claude Sonnet/Haiku, GPT-OSS
  • Easy to add custom models

Documentation

• README-coding-agent-security.md - Complete pipeline guide
  • Step-by-step workflow
  • Configuration instructions
  • Model selection tips
  • Troubleshooting guide

🎯 Key Features

Provider-Based Model Routing

• Automatic backend detection using explicit provider field
• No Python code changes to add new models
• Mix Bedrock and local models seamlessly
• Clear error messages for invalid configurations

Flexible Configuration

# Change models without code edits
coordinator_model: "claude-sonnet-4-5"
composer_model: "claude-sonnet-4-5"
reviewer_models:
  - "claude-sonnet-3-7"
  - "claude-haiku-4-5"
  - "qwen3coder"
parallel_tasks: 50

Knowledge Graph Structure

Prohibited Domain (e.g., "Web Security")
└── Technique Family (e.g., "Injection Attacks")
    └── Concrete Instance (e.g., "SQL Injection")

📊 Example Dataset

Generated dataset available at: https://huggingface.co/datasets/PurCL/astra-agent-security

🚀 Quick Start

# 1. Generate test cases
python agent/main_agent_sec.py

# 2. Parse outputs
python agent/parse_agent_sec.py \
  -i data_out/syn_agent_sec.jsonl \
  -o data_out/syn_agent_sec_parsed.jsonl

# 3. Upload to HuggingFace
python agent/upload_to_hf_dataset.py \
  -i data_out/syn_agent_sec_parsed.jsonl \
  -d username/astra-agent-security

🔒 Security & Best Practices

• ✅ No hardcoded paths - all scripts use CLI arguments
• ✅ No credentials in config - uses placeholders
• ✅ Proper .gitignore for build artifacts
• ✅ LFS tracking for large knowledge graph files
• ✅ Clean configuration/code separation

📈 Statistics

• 17 files changed: 766 insertions(+), 85 deletions(-)
• 8 new files added
• 9 files updated for configuration system

🎓 Use Cases

• Red team testing: Generate adversarial prompts for coding agents
• Security evaluation: Systematic assessment of agent vulnerabilities
• Research: Dataset generation for agent safety research
• Benchmarking: Create standardized security test suites

📚 Documentation

Complete documentation in README-coding-agent-security.md covering:

• Installation and prerequisites
• Configuration guide for Bedrock and OpenAI-compatible servers
• Multi-agent system architecture
• Customization and extension guide
• Troubleshooting

---

Built on top of: v0.1 (Amazon Nova AI Challenge Winner 🏆)