Skip to content

Latest commit

ย 

History

92 Commits

Folders and files

NameName
Last commit message
Last commit date
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

DeployGuard ๐Ÿ›ก๏ธ

CI Quality Gate Python Version FastAPI Google ADK Next.js Dashboard OpenTelemetry License

Fortified Autonomous SRE Fleet for Safe, Policy-Governed CI/CD Operations on Google Cloud.


๐ŸŽฏ The Problem: The Unconstrained AI in Production Dilemma

Production deployments are high-stakes operations. SRE teams face a difficult tradeoff:

  • Manual SRE incident response is slow: Mean Time To Recovery (MTTR) stretches into tens of minutes while engineers manually triage logs, query metrics, and debate rollback thresholds.
  • Unconstrained AI agents are dangerous: Handing production credentials to a single monolithic LLM creates security risks (prompt injection in raw logs, hallucinated commands, zero separation of duties, and un-auditable actions).

๐Ÿš€ The Solution: DeployGuard Governed Fleet

DeployGuard bridges this gap by replacing unconstrained chatbots with a five-agent specialized fleet wrapped in strict enterprise governance gates:

  1. Separation of Duties & Least-Privilege IAM: Reasoning agents cannot execute production rollbacks. Only the authorized RollbackAgent holds Cloud Deploy permissions.
  2. Deterministic Policy Gates: Gemini LLM suggestions are bounded by strict deterministic rule evaluations (confidence thresholds, environment policies, deployment age limits, and verified target releases).
  3. Multi-Stage Security & Model Armor: Untrusted log streams are screened for prompt injection attacks ([PROMPT_INJECTION_BLOCKED]) and PII credentials ([REDACTED_CREDENTIALS]) before reaching model context.
  4. End-to-End Auditability: Every decision generates an immutable DecisionTrace in Firestore with OpenTelemetry root/child span lineage and automated SRE postmortem synthesis.

๐Ÿ—๏ธ System Architecture

Visual Architecture Pipeline (Mermaid)

flowchart TD
    subgraph Telemetry ["๐Ÿ“ก Production Telemetry & Signals"]
        M1["Cloud Monitoring (7-Dim Metrics)"]
        M2["Cloud Logging (Untrusted Logs)"]
    end

    subgraph Security ["๐Ÿ›ก๏ธ Security & Ingestion Boundary"]
        SAN["Multi-Stage LogSanitizer (PII & Injection Redaction)"]
        GW["Agent Gateway (IAM Authorization & Least Privilege)"]
        MA["Vertex AI Model Armor (Prompt & Response Filter)"]
    end

    subgraph Fleet ["๐Ÿค– 5-Agent Autonomous Fleet (Google ADK)"]
        DMA["1. Deploy Monitor Agent
(Baseline Anomaly Detection)"]
        IMA["2. Incident Memory Agent
(Vector Search & Firestore Memory)"]
        DA["3. Decision Agent
(Gemini 2.5 Flash + PolicyEngine)"]
(Gemini 3.5 Flash + PolicyEngine)"]
        RA["4. Rollback Agent
(Cloud Deploy Execution)"]
        PMA["5. Postmortem Agent
(5-Whys RCA & Markdown Synthesis)"]
    end

    subgraph Storage ["๐Ÿ’พ Persistence & Observability"]
        FS[("Cloud Firestore
Incidents & Postmortems")]
        OTEL["OpenTelemetry Traces
(Root/Child Spans)"]
        SSE["FastAPI AsyncEventBroadcaster
(Server-Sent Events /stream)"]
    end

    subgraph Dashboard ["๐Ÿ’ป SRE Operator Dashboard (Next.js 14)"]
        UI1["Live Telemetry Sparklines"]
        UI2["Governance Pipeline Visualizer"]
        UI3["Postmortem Markdown Viewer"]
        UI4["Agent Fleet Registry & IAM Matrix"]
    end

    M1 --> DMA
    M2 --> SAN --> MA --> DA
    DMA -- Anomaly Signal --> IMA
    IMA -- Similar Incidents --> DA
    DA -- Decision Trace --> GW
    GW -- Verified Token --> RA
    RA -- Release Rollback --> M1
    RA -- Rollout Status --> DMA
    DMA -- Recovery Verdict --> PMA
    PMA --> FS
    Fleet --> OTEL
    Fleet --> SSE --> Dashboard
Loading

System Topology (ASCII)

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                            DEPLOYGUARD PLATFORM                             โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚    OPERATOR DASHBOARD (UI)   โ”‚      FASTAPI REST & SSE      โ”‚ PERSISTENCE   โ”‚
โ”‚  โ€ข Live Metric Sparklines    โ”‚  โ€ข /api/v1/events/stream     โ”‚ โ€ข Firestore   โ”‚
โ”‚  โ€ข Governance Pipeline View  โ”‚  โ€ข /api/v1/dashboard/metrics โ”‚ โ€ข Vector DB   โ”‚
โ”‚  โ€ข Trace Stepper & Waterfallsโ”‚  โ€ข /api/v1/traces/{trace_id} โ”‚ โ€ข Cloud Trace โ”‚
โ”‚  โ€ข Agent IAM Matrix Table    โ”‚  โ€ข /api/v1/postmortems       โ”‚ โ€ข Seed DB     โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
               โ”‚                              โ”‚                       โ”‚
               โ–ผ                              โ–ผ                       โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                    GOVERNED AGENT FLEET (GOOGLE ADK)                        โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ 1. Deploy Monitor Agent  โ”‚ 2. Incident Memory Agent โ”‚ 3. Decision Agent     โ”‚
โ”‚    โ€ข 7-dim baseline diff โ”‚    โ€ข Vector embeddings   โ”‚    โ€ข Gemini 2.5 Flash โ”‚
โ”‚    โ€ข 7-dim baseline diff โ”‚    โ€ข Vector embeddings   โ”‚    โ€ข Gemini 3.5 Flash โ”‚
โ”‚    โ€ข Multi-step recovery โ”‚    โ€ข Historical lookup   โ”‚    โ€ข PolicyEngine     โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ 4. Rollback Agent        โ”‚ 5. Postmortem Agent      โ”‚ ๐Ÿ›ก๏ธ Security Gateways  โ”‚
โ”‚    โ€ข Cloud Deploy exec   โ”‚    โ€ข SRE 5-whys RCA      โ”‚    โ€ข Agent Gateway    โ”‚
โ”‚    โ€ข 2-tier auth check   โ”‚    โ€ข Markdown generation โ”‚    โ€ข LogSanitizer     โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

๐Ÿค– Specialized 5-Agent Fleet Specification

Agent Name Agent ID Service Account Identity Permissions Held Risk Level Primary Domain Responsibility
Deploy Monitor deploy-monitor-v1 sa-monitor@deployguard.iam.gserviceaccount.com monitoring.read, logging.read LOW Samples 7-dimensional metric baselines; triggers alerts; verifies post-rollback recovery.
Incident Memory incident-memory-v1 sa-memory@deployguard.iam.gserviceaccount.com datastore.read, datastore.write LOW Manages Firestore incident vector bank; retrieves historical incident matches via semantic similarity.
Decision Engine decision-v2 sa-decision@deployguard.iam.gserviceaccount.com gemini.invoke, memory.read MEDIUM Combines LLM reasoning, Model Armor screening, and deterministic PolicyEngine checks into a signed DecisionTrace.
Rollback Agent rollback-v1 sa-rollback@deployguard.iam.gserviceaccount.com clouddeploy.releaserollback HIGH Enforces two-tier authorization check before executing Cloud Deploy releases.
Postmortem Agent postmortem-v1 sa-postmortem@deployguard.iam.gserviceaccount.com datastore.write, gemini.invoke LOW Deterministically synthesizes SRE postmortems with 5-whys root cause analysis and preventative action items.

โšก 60-Second Quickstart

1. Prerequisites

  • Python 3.12+ (or 3.13)
  • Node.js 18+ & npm
  • uv package manager

2. Installation

git clone https://github.com/sukhada20/DeployGuard.git
cd DeployGuard

# Install backend dependencies and frontend packages
make install

3. Run Development Servers

# Start FastAPI backend & Next.js dashboard proxy on http://localhost:8000
make dev

4. Launch Interactive Demonstration

# In a separate terminal, trigger the full autonomous recovery demonstration
make demo

๐ŸŽฎ Demonstration Scenarios

DeployGuard includes a built-in CLI demonstration orchestrator (src/deployguard/demo/):

Command Mode Description
make demo Interactive Step-by-step presentation mode with Enter key pauses to explain each stage and inspect the live dashboard.
make demo-auto Timed Automated timed playback with 1s delays (ideal for video recordings).
make demo-ci Headless Fast non-interactive run for CI pipelines and automated assertions.
make demo-security Security Runs both Agent Gateway denial and Prompt Injection defense simulations.
make demo-security-gateway Security Demonstrates Agent Gateway rejecting unauthorized action calls by DecisionAgent.
make demo-security-injection Security Demonstrates multi-stage log sanitization neutralizing prompt injections and PII keys.
make demo-clean Reset Clears mock Firestore documents, metrics, and incident memory between runs.

๐Ÿ‘‰ See DEMO.md for the complete SRE Operator Runbook and step-by-step presenter guide.


๐Ÿงช Quality Gates & Verification

DeployGuard maintains strict quality gates across both backend Python code and frontend Next.js assets:

# Run formatters, linters, full unit & integration tests, agent benchmarks, and web production build
make verify

The verification gate executes:

  1. ruff format --check src/ tests/ (PEP 8 code formatting)
  2. ruff check src/ tests/ (Fast Python linting)
  3. mypy src/ tests/ (Strict static typechecking)
  4. pytest tests/ -v -m "not live_gcp" (120 unit, e2e, and security tests)
  5. pytest tests/test_evals.py -v (DeployGuard Agent Evaluation benchmark suite)
  6. npm --prefix web run build (Next.js App Router production build)

โ˜๏ธ Google Cloud Deployment

DeployGuard is designed natively for Google Cloud Platform services:

  • Compute: Cloud Run (Containerized FastAPI API & Web SPA)
  • Agent Intelligence: Vertex AI (Gemini 2.5 Flash) & Google ADK
  • Agent Intelligence: Vertex AI (Gemini 3.5 Flash) & Google ADK
  • Security & Safety: Vertex AI Model Armor & IAM Service Accounts
  • Deployment: Google Cloud Deploy & Cloud Build
  • Telemetry: Google Cloud Monitoring & Cloud Logging
  • Memory & Storage: Cloud Firestore (with Vector Search) & Cloud Trace

๐Ÿ‘‰ See docs/DEPLOYMENT.md for the complete step-by-step GCP production deployment manual.


๐Ÿงช Testing the Deployed Cloud Run App

To test the deployed Cloud Run application continuously, you can create a test repository and configure GitHub Actions:

  1. Create a new GitHub repository (e.g., deployguard-e2e-tests).
  2. Add a GitHub Actions workflow .github/workflows/main.yml:
name: E2E Cloud Run Tests

on:
  push:
    branches: [ main ]
  schedule:
    - cron: '0 */6 * * *' # Run every 6 hours

jobs:
  test-health:
    runs-on: ubuntu-latest
    steps:
      - name: Check Health Endpoint
        run: |
          HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" https://YOUR_CLOUD_RUN_URL/api/v1/health)
          if [ "$HTTP_STATUS" -ne 200 ]; then
            echo "Health check failed with status $HTTP_STATUS"
            exit 1
          fi
          echo "Health check passed!"
  1. Replace YOUR_CLOUD_RUN_URL with the actual URL of your deployed Cloud Run service.
  2. The GitHub Action will automatically run tests on every push and periodically to ensure the deployment remains healthy.

๐Ÿ“„ License

Licensed under the Apache License, Version 2.0. See LICENSE for details.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages