refactor: split monolithic binary into dedicated orchestrator, worker, and cli crates#222
Merged
refactor: split monolithic binary into dedicated orchestrator, worker, and cli crates#222
Conversation
… template set **Added:** - Introduced Rust-native agent templates for Ares, including: - ares-rust-acl-agent: AD ACL exploitation agent with Rust binary and Ansible tooling - ares-rust-blue-agent: Blue team defensive agent (Rust, embedded Python) - ares-rust-blue-triage-agent: Blue triage agent (Rust, Grafana MCP) - ares-rust-blue-threat-hunter-agent: Blue threat hunter (Rust, Grafana MCP) - ares-rust-blue-lateral-analyst-agent: Blue lateral analyst (Rust, Grafana MCP) - ares-rust-coercion-agent: NTLM relay/coercion agent (Rust, Ansible) - ares-rust-cracker-agent: Password cracker agent (Rust, hashcat/john, Ansible) - ares-rust-cracker-agent-gpu: GPU-accelerated password cracker (Rust, CUDA/OpenCL) - ares-rust-credential-access-agent: Credential access agent (Rust, Kerberos, Ansible) - ares-rust-lateral-movement-agent: Lateral movement agent (Rust, Ansible) - ares-rust-orchestrator: Orchestrator binary with embedded Python for LLM ops - ares-rust-privesc-agent: Privilege escalation agent (Rust, Ansible) - ares-rust-recon-agent: Reconnaissance agent (Rust, AD/network tools, Ansible) - ares-rust-worker: Generic worker agent (Rust binary, embedded Python) - ares-rust-cli: Pure Rust CLI for Ares framework (no Python) - Added ares-golden-image AMI template for full red team toolset golden image - Added comprehensive README and documentation for each new template - Provided .gitignore, pre-commit config, and linter configs for template repo - Added shell hooks and formatting utilities for pre-commit and workflow checks **Changed:** - Modernized template directory structure: moved all existing agent templates to `templates/` and replaced Python-based agent templates with Rust-based equivalents - Updated main repository README with new template list, features, usage, and architectural overview reflecting Rust migration and new capabilities - Improved documentation for blue team, GPU, and CLI templates, clarifying runtime, build, and orchestration differences - Updated CI workflows and validation logic to support new Rust-based templates and ensure build/test consistency - Updated workflow and pre-commit scripts to check for consistency in workflow repository lists and enforce formatting/linters - Enhanced template build chains to support multi-arch, Rust compilation, and embedded Python agent operation - Added AMI/golden image build for red team labs covering recon, credential access, privesc, cracking, lateral movement, ACL abuse, and coercion **Removed:** - Removed all legacy Python-based agent templates: - ares-acl-agent, ares-blue-agent, ares-blue-triage-agent, ares-blue-threat-hunter-agent, ares-blue-lateral-analyst-agent, ares-coercion-agent, ares-cracker-agent, ares-cracker-agent-gpu, ares-cracker-base-gpu, ares-credential-access-agent, ares-lateral-movement-agent, ares-privesc-agent, ares-recon-agent, ares-worker, ares-base, ares-orchestrator and their corresponding READMEs and warpgate.yaml files - Removed Python-centric documentation and notes from main and template-specific READMEs - Removed template duplication and legacy shell/Ansible-only build steps in favor of Rust-native agent binaries with embedded Python support
**Changed:** - Updated all GitHub Actions workflows and labeler configuration to reference 'warpgate-templates' instead of 'templates' for template-related operations, including build, validation, and test processes - Adjusted shell and Python scripts within workflows to search, loop, and operate on 'warpgate-templates' paths, ensuring correct discovery and validation of template files - Modified labeler rules to apply the area/templates label to changes under 'warpgate-templates', maintaining accurate PR labeling
ci: remove GITHUB_TOKEN build-arg from warpgate build steps
**Changed:**
- Updated build-and-push-templates workflow to no longer pass GITHUB_TOKEN as a
build-arg to warpgate build commands, removing both the flag and related
comments. This simplifies the build invocation and eliminates unnecessary
exposure of the token.
- Updated test-template-builds workflow to remove the --build-arg
"GITHUB_TOKEN=${GITHUB_TOKEN}" parameter from warpgate build steps, aligning
test builds with the new approach and reducing secrets usage.
**Removed:**
- Eliminated passing of GITHUB_TOKEN as a build argument to container builds in
all relevant GitHub Actions workflows.
```
**Added:** - Specify `kalilinux/kali-rolling:latest` as the base image in the ares-golden-image template to standardize environment for builds
…nsistent indentation **Changed:** - Updated workflow to check required files in `warpgate-templates/templates/*/` instead of `warpgate-templates/*/` for correct directory structure in `validate-templates.yaml` - Reformatted `warpgate.yaml` for the ares-golden-image template to use consistent two-space indentation across all sections, improving readability and aligning with common YAML conventions
…oducibility
**Added:**
- Introduced `ares-cracker-base-gpu` Warpgate template for CUDA/OpenCL GPU-accelerated
hashcat builds targeting NVIDIA GPUs (T4/g4dn) - `warpgate-templates/templates/ares-cracker-base-gpu/warpgate.yaml`
- Added explicit installation and verification steps for regsecrets in NetExec pipx
venv, with fallback logic for direct impacket source copy, enhancing reliability
of blue team recon tools - `ansible/roles/recon_tools/tasks/netexec_pipx.yml`
**Changed:**
- Updated CI/CD workflows to use local checkouts for Ansible collection instead
of cloning or downloading via GitHub App tokens, improving determinism and
reliability - `.github/workflows/build-and-push-templates.yaml`, `.github/workflows/test-template-builds.yaml`
- Reworked template discovery, matrix generation, and path usage in GitHub Actions
to support new templates directory layout and GPU builds
- Changed all image push/pull authentication to use `${{ secrets.GITHUB_TOKEN }}`
instead of a custom PAT for improved secret management
- Made swap file creation more robust by handling pre-existing swap files before
allocation in CI scripts
- Updated provisioners in multiple Ares Rust agent templates to directly build
Rust binaries from source and clean up toolchain and build artifacts, reducing
image size and complexity
- Upgraded package dependencies for lateral movement tools, switching to
`freerdp3-x11` from `freerdp2-x11` for compatibility with Ubuntu 24.04
- Enhanced blue team agent and worker cleanup steps: aggressively remove build
dependencies and clean apt caches for smaller, more reproducible images
- Modified golden image template to install NVIDIA drivers and select GPU
instance type for AMI builds, enabling hashcat GPU acceleration out-of-the-box
- Ensured all agent templates use consistent Rust toolchain installation and
cleanup procedures
- Improved NetExec pipx venv detection to support both XDG and legacy pipx paths,
making binary symlinking and validation more robust
- Refined Rust orchestrator base image to use `debian:trixie-slim` instead of
`python:3.13.7-slim` for a more controlled build environment
**Removed:**
- Eliminated all usage of GitHub App token generation and token-authenticated
API downloads for Ansible collection, simplifying workflow authentication
- Removed redundant cleanup shell provisioners that duplicated logic now handled
during build and final cleanup steps in agent templates
- Dropped direct pipx install of Ares in golden image build, as this is now
handled elsewhere or is unnecessary
**Added:** - Provided a detailed README with usage, requirements, and customization steps for building and provisioning Ares Base images using Warp Gate - Introduced `warpgate.yaml` template for building multi-arch Docker images with Python 3.13, core dependencies, and Ansible provisioning steps - Enabled support for both `amd64` and `arm64` Docker targets, including privilege and environment configuration for CI or Kubernetes use - Included shell and Ansible provisioners for system setup and Ares role installation via the nimbus_range collection - Documented post-build steps, image pushing, and cleanup procedures for reproducible and customizable base images
**Changed:** - Switch ares repository source from `feature/rust-cli` branch to `main` branch across all agent, orchestrator, and CLI templates for consistency and to track upstream development - Unify cargo build process to compile from top-level source directory using `cargo build --release --bin ares` instead of building in subdirectories, simplifying build steps for all Rust-based templates - Update binary installation steps to copy the built `ares` binary to the correct destination and rename as needed (e.g., `ares-worker`, `ares-cli`, `ares-orchestrator`) to standardize artifact handling - Remove redundant file copy and manual build steps for agent templates that now depend on prebuilt or installed `ares` binaries, reducing image complexity - Update documentation in all affected README.md files to reflect the switch to the `main` branch and the new build process, improving clarity for users and maintainers
**Changed:** - Change GitHub Actions runner for arm64 builds from `ubuntu-latest` to `ubuntu-24.04-arm64` to ensure compatibility and correct platform usage in the build-and-push-templates workflow
**Changed:** - Updated the GitHub Actions workflow to use 'ubuntu24.04-arm64-8-core' as the runner label for arm64 jobs, replacing 'ubuntu-24.04-arm64', to match the correct available runner name
**Changed:** - Added `apt-get update && apt-get install -y python3-dev || true` to the shell provisioner to ensure Python development headers are available during provisioning for docker.arm64 targets in warpgate.yaml
**Changed:** - Changed the arm64 runner label from `ubuntu24.04-arm64-8-core` to `ubuntu-24.04-arm` to match updated GitHub Actions runner naming conventions in the build and push workflow
Changed: - Consolidated multiple apt-get install commands into a single line to improve build efficiency and reduce layer count in the Docker image - Expanded installed packages to include development, debugging, networking, and utility tools (e.g., libffi-dev, wget, cargo, netcat-traditional, strace, lsof, tcpdump, net-tools, iproute2, telnet, vim, jq, htop, tmux, acl, bind9-dnsutils) for broader functionality and troubleshooting support - Removed redundant separate installation of python3-dev, now included in the consolidated install command
dreadnode-renovate-bot
Bot
added
area/templates
Resolve all merge conflicts: - Accept main's unified template naming (ares-X, not ares-rust-X) - Accept main's action version bumps and checksum verification - Accept main's golden-image CUDA toolkit provisioner - Keep new ares-base template from this branch - Fix duplicate YAML keys in goad_attack_box.yml
**Changed:** - Updated GPU dependency chain in the build workflow to use `ares-rust-cracker-agent-gpu` instead of `ares-cracker-agent-gpu` for consistency with template naming and to ensure correct build sequencing in `.github/workflows/build-and-push-templates.yaml`
… template set **Added:** - Introduced Rust-native agent templates for Ares, including: - ares-rust-acl-agent: AD ACL exploitation agent with Rust binary and Ansible tooling - ares-rust-blue-agent: Blue team defensive agent (Rust, embedded Python) - ares-rust-blue-triage-agent: Blue triage agent (Rust, Grafana MCP) - ares-rust-blue-threat-hunter-agent: Blue threat hunter (Rust, Grafana MCP) - ares-rust-blue-lateral-analyst-agent: Blue lateral analyst (Rust, Grafana MCP) - ares-rust-coercion-agent: NTLM relay/coercion agent (Rust, Ansible) - ares-rust-cracker-agent: Password cracker agent (Rust, hashcat/john, Ansible) - ares-rust-cracker-agent-gpu: GPU-accelerated password cracker (Rust, CUDA/OpenCL) - ares-rust-credential-access-agent: Credential access agent (Rust, Kerberos, Ansible) - ares-rust-lateral-movement-agent: Lateral movement agent (Rust, Ansible) - ares-rust-orchestrator: Orchestrator binary with embedded Python for LLM ops - ares-rust-privesc-agent: Privilege escalation agent (Rust, Ansible) - ares-rust-recon-agent: Reconnaissance agent (Rust, AD/network tools, Ansible) - ares-rust-worker: Generic worker agent (Rust binary, embedded Python) - ares-rust-cli: Pure Rust CLI for Ares framework (no Python) - Added ares-golden-image AMI template for full red team toolset golden image - Added comprehensive README and documentation for each new template - Provided .gitignore, pre-commit config, and linter configs for template repo - Added shell hooks and formatting utilities for pre-commit and workflow checks **Changed:** - Modernized template directory structure: moved all existing agent templates to `templates/` and replaced Python-based agent templates with Rust-based equivalents - Updated main repository README with new template list, features, usage, and architectural overview reflecting Rust migration and new capabilities - Improved documentation for blue team, GPU, and CLI templates, clarifying runtime, build, and orchestration differences - Updated CI workflows and validation logic to support new Rust-based templates and ensure build/test consistency - Updated workflow and pre-commit scripts to check for consistency in workflow repository lists and enforce formatting/linters - Enhanced template build chains to support multi-arch, Rust compilation, and embedded Python agent operation - Added AMI/golden image build for red team labs covering recon, credential access, privesc, cracking, lateral movement, ACL abuse, and coercion **Removed:** - Removed all legacy Python-based agent templates: - ares-acl-agent, ares-blue-agent, ares-blue-triage-agent, ares-blue-threat-hunter-agent, ares-blue-lateral-analyst-agent, ares-coercion-agent, ares-cracker-agent, ares-cracker-agent-gpu, ares-cracker-base-gpu, ares-credential-access-agent, ares-lateral-movement-agent, ares-privesc-agent, ares-recon-agent, ares-worker, ares-base, ares-orchestrator and their corresponding READMEs and warpgate.yaml files - Removed Python-centric documentation and notes from main and template-specific READMEs - Removed template duplication and legacy shell/Ansible-only build steps in favor of Rust-native agent binaries with embedded Python support
…oducibility
**Added:**
- Introduced `ares-cracker-base-gpu` Warpgate template for CUDA/OpenCL GPU-accelerated
hashcat builds targeting NVIDIA GPUs (T4/g4dn) - `warpgate-templates/templates/ares-cracker-base-gpu/warpgate.yaml`
- Added explicit installation and verification steps for regsecrets in NetExec pipx
venv, with fallback logic for direct impacket source copy, enhancing reliability
of blue team recon tools - `ansible/roles/recon_tools/tasks/netexec_pipx.yml`
**Changed:**
- Updated CI/CD workflows to use local checkouts for Ansible collection instead
of cloning or downloading via GitHub App tokens, improving determinism and
reliability - `.github/workflows/build-and-push-templates.yaml`, `.github/workflows/test-template-builds.yaml`
- Reworked template discovery, matrix generation, and path usage in GitHub Actions
to support new templates directory layout and GPU builds
- Changed all image push/pull authentication to use `${{ secrets.GITHUB_TOKEN }}`
instead of a custom PAT for improved secret management
- Made swap file creation more robust by handling pre-existing swap files before
allocation in CI scripts
- Updated provisioners in multiple Ares Rust agent templates to directly build
Rust binaries from source and clean up toolchain and build artifacts, reducing
image size and complexity
- Upgraded package dependencies for lateral movement tools, switching to
`freerdp3-x11` from `freerdp2-x11` for compatibility with Ubuntu 24.04
- Enhanced blue team agent and worker cleanup steps: aggressively remove build
dependencies and clean apt caches for smaller, more reproducible images
- Modified golden image template to install NVIDIA drivers and select GPU
instance type for AMI builds, enabling hashcat GPU acceleration out-of-the-box
- Ensured all agent templates use consistent Rust toolchain installation and
cleanup procedures
- Improved NetExec pipx venv detection to support both XDG and legacy pipx paths,
making binary symlinking and validation more robust
- Refined Rust orchestrator base image to use `debian:trixie-slim` instead of
`python:3.13.7-slim` for a more controlled build environment
**Removed:**
- Eliminated all usage of GitHub App token generation and token-authenticated
API downloads for Ansible collection, simplifying workflow authentication
- Removed redundant cleanup shell provisioners that duplicated logic now handled
during build and final cleanup steps in agent templates
- Dropped direct pipx install of Ares in golden image build, as this is now
handled elsewhere or is unnecessary
**Changed:** - Switch ares repository source from `feature/rust-cli` branch to `main` branch across all agent, orchestrator, and CLI templates for consistency and to track upstream development - Unify cargo build process to compile from top-level source directory using `cargo build --release --bin ares` instead of building in subdirectories, simplifying build steps for all Rust-based templates - Update binary installation steps to copy the built `ares` binary to the correct destination and rename as needed (e.g., `ares-worker`, `ares-cli`, `ares-orchestrator`) to standardize artifact handling - Remove redundant file copy and manual build steps for agent templates that now depend on prebuilt or installed `ares` binaries, reducing image complexity - Update documentation in all affected README.md files to reflect the switch to the `main` branch and the new build process, improving clarity for users and maintainers
l50
force-pushed
the
fix/warpgate-templates
branch
from
c516693 to
b2c10ae
Compare
…y handling
**Changed:**
- Updated all agent and orchestrator README.md files to use non-Rust-prefixed names
(e.g., `ares-worker` instead of `ares-rust-worker`) throughout descriptions,
initialization, build, push, and validation instructions for consistency and
clarity.
- Standardized template metadata and YAML fields in all `warpgate.yaml` files:
- Unified `name` and metadata fields to non-Rust-prefixed naming.
- Removed legacy or redundant tags and streamlined descriptions.
- Consistently reference the `main` branch for source git checkouts.
- Set correct ENTRYPOINTs to use the new binary names (e.g., `ares-worker`,
`ares-orchestrator`).
- Ensured binaries are installed and referenced as `ares-worker` or
`ares-orchestrator` as appropriate.
- Updated shell provisioners to copy and name binaries consistently.
- Added missing build context changes (USER, WORKDIR) for container
consistency.
- Rewrote and clarified documentation tables and build instructions to reflect
new binary names, including differences between Python and Rust versions of
each agent and orchestrator.
- Removed all legacy merge conflict markers and HEAD/branch artifact lines from
documentation and YAML files.
- Ensured all references to Docker tags, image names, and Kubernetes manifests
are consistently using the non-Rust-prefixed convention.
- Updated summary sections and technical details to clarify that all agents and
orchestrator binaries are compiled with PyO3 Python bindings and follow the
new naming scheme.
**Removed:**
- Eliminated all obsolete references to `ares-rust-*` and legacy Rust-prefixed
Docker image names, binaries, and template names from documentation and YAMLs.
- Removed all merge conflict leftovers and duplicate documentation blocks
arising from previous merges.
**Changed:** - Expanded the GPU dependent templates list to include both `ares-cracker-agent-gpu` and `ares-rust-cracker-agent-gpu` for parallel builds, updating the matrix logic and comments for clarity in `build-and-push-templates.yaml`
…collection **Added:** - Added `sources` section in golden image template to securely clone the nimbus_range ansible collection using a GitHub token and avoid exposing credentials in shell commands - `warpgate-templates/templates/ares-golden-image/warpgate.yaml` **Changed:** - Added `unzip` to the apt install list in ares-base and golden-image Docker provisioning to support tasks that require unzipping archives - Set `update_cache: true` for Ansible apt tasks installing unzip to ensure package lists are current and avoid failures due to stale caches in `install_awscli.yml` and privesc_tools' `linux.yml` - Refactored the golden image provisioning process to use a dedicated file copy step for the ansible collection instead of shell cloning with embedded credentials, improving security and maintainability **Removed:** - Removed inline git clone command using embedded GitHub token from the provisioning shell steps in the golden image template to prevent accidental credential leakage
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Key Changes:
ares-orchestratorandares-workeras standalone Rust binariesares-cli,ares-orchestrator, andares-workerAdded:
ares-orchestratorcrate: Rust-native orchestrator binary with all red/blue automation and agent logicares-workercrate: Rust-native worker binary for agent task executionares-rust-orchestrator,ares-rust-worker)warpgate-templates/MANUAL.md)Changed:
ares-cliintoares-orchestratorares-cliintoares-workerares-orchestratorandares-workerares-orchestrator/src/Removed:
aresbinary entrypoint and subcommand routing for orchestrator and worker