Skip to content

ace-v1

Latest
Latest

Choose a tag to compare

View all tags
@UJ2202 UJ2202 released this 27 Oct 09:21
ace-v1
4ef5889
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

ACE v1.0 — First Release

Agent Certification Engine is now generally available.

ACE is an end-to-end platform for evaluating autonomous AI agents under real infrastructure chaos. It injects faults into a Kubernetes cluster, lets an AI agent attempt autonomous remediation, collects the full LLM trace, and produces a statistically rigorous 12-section certification report.

What's included

Certification pipeline

  • 4-phase Python pipeline: fault bucketing → metrics extraction → statistical aggregation → certification report assembly
  • LLM Council: k independent judges + meta-reconciliation to eliminate single-model bias
  • Optional Phase E: H01–H09 statistical hypothesis tests (BCa bootstrap, Kruskal-Wallis, CUSUM drift detection) for n ≥ 30 runs

AgentCert control plane

  • LitmusChaos ChaosCenter v3.0.0 with ACE-specific packages: agent registry, app registry, fault studio, observability, certification orchestrator
  • Full GraphQL API (queries, mutations, subscriptions) with JWT RBAC
  • React 17 + TypeScript frontend with Chaos Studio, Agent Hub, App Hub

Agents evaluated

  • flash-agent — FLASH-methodology SRE ITOps agent (custom ReAct loop + MCP)

Infrastructure

  • One-command setup (./scripts/setup.sh --deploy) across three cluster routes: existing cluster, fresh KinD, cloud (AKS/EKS/GKE)
  • Helm chart deploying the full platform stack (~36 Gi storage, 16 services)
  • LiteLLM unified gateway routing to Azure OpenAI, Gemini, OpenRouter, or local Ollama
  • Langfuse OTEL trace storage for all LLM calls
  • Shared-host isolation — every container, volume, network, and cluster name is instance-scoped to prevent cross-checkout collisions

Quick start

git clone --recurse-submodules https://github.com/AgentCert/ace-monorepo
cd ace-monorepo
./scripts/setup.sh --deploy
Assets 2
Loading