Fortified Autonomous SRE Fleet for Safe, Policy-Governed CI/CD Operations on Google Cloud.
Production deployments are high-stakes operations. SRE teams face a difficult tradeoff:
- Manual SRE incident response is slow: Mean Time To Recovery (MTTR) stretches into tens of minutes while engineers manually triage logs, query metrics, and debate rollback thresholds.
- Unconstrained AI agents are dangerous: Handing production credentials to a single monolithic LLM creates security risks (prompt injection in raw logs, hallucinated commands, zero separation of duties, and un-auditable actions).
DeployGuard bridges this gap by replacing unconstrained chatbots with a five-agent specialized fleet wrapped in strict enterprise governance gates:
- Separation of Duties & Least-Privilege IAM: Reasoning agents cannot execute production rollbacks. Only the authorized
RollbackAgentholds Cloud Deploy permissions. - Deterministic Policy Gates: Gemini LLM suggestions are bounded by strict deterministic rule evaluations (confidence thresholds, environment policies, deployment age limits, and verified target releases).
- Multi-Stage Security & Model Armor: Untrusted log streams are screened for prompt injection attacks (
[PROMPT_INJECTION_BLOCKED]) and PII credentials ([REDACTED_CREDENTIALS]) before reaching model context. - End-to-End Auditability: Every decision generates an immutable
DecisionTracein Firestore with OpenTelemetry root/child span lineage and automated SRE postmortem synthesis.
flowchart TD
subgraph Telemetry ["๐ก Production Telemetry & Signals"]
M1["Cloud Monitoring (7-Dim Metrics)"]
M2["Cloud Logging (Untrusted Logs)"]
end
subgraph Security ["๐ก๏ธ Security & Ingestion Boundary"]
SAN["Multi-Stage LogSanitizer (PII & Injection Redaction)"]
GW["Agent Gateway (IAM Authorization & Least Privilege)"]
MA["Vertex AI Model Armor (Prompt & Response Filter)"]
end
subgraph Fleet ["๐ค 5-Agent Autonomous Fleet (Google ADK)"]
DMA["1. Deploy Monitor Agent
(Baseline Anomaly Detection)"]
IMA["2. Incident Memory Agent
(Vector Search & Firestore Memory)"]
DA["3. Decision Agent
(Gemini 2.5 Flash + PolicyEngine)"]
(Gemini 3.5 Flash + PolicyEngine)"]
RA["4. Rollback Agent
(Cloud Deploy Execution)"]
PMA["5. Postmortem Agent
(5-Whys RCA & Markdown Synthesis)"]
end
subgraph Storage ["๐พ Persistence & Observability"]
FS[("Cloud Firestore
Incidents & Postmortems")]
OTEL["OpenTelemetry Traces
(Root/Child Spans)"]
SSE["FastAPI AsyncEventBroadcaster
(Server-Sent Events /stream)"]
end
subgraph Dashboard ["๐ป SRE Operator Dashboard (Next.js 14)"]
UI1["Live Telemetry Sparklines"]
UI2["Governance Pipeline Visualizer"]
UI3["Postmortem Markdown Viewer"]
UI4["Agent Fleet Registry & IAM Matrix"]
end
M1 --> DMA
M2 --> SAN --> MA --> DA
DMA -- Anomaly Signal --> IMA
IMA -- Similar Incidents --> DA
DA -- Decision Trace --> GW
GW -- Verified Token --> RA
RA -- Release Rollback --> M1
RA -- Rollout Status --> DMA
DMA -- Recovery Verdict --> PMA
PMA --> FS
Fleet --> OTEL
Fleet --> SSE --> Dashboard
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ DEPLOYGUARD PLATFORM โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโค
โ OPERATOR DASHBOARD (UI) โ FASTAPI REST & SSE โ PERSISTENCE โ
โ โข Live Metric Sparklines โ โข /api/v1/events/stream โ โข Firestore โ
โ โข Governance Pipeline View โ โข /api/v1/dashboard/metrics โ โข Vector DB โ
โ โข Trace Stepper & Waterfallsโ โข /api/v1/traces/{trace_id} โ โข Cloud Trace โ
โ โข Agent IAM Matrix Table โ โข /api/v1/postmortems โ โข Seed DB โ
โโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโดโโโโโโโโฌโโโโโโโโ
โ โ โ
โผ โผ โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ GOVERNED AGENT FLEET (GOOGLE ADK) โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโค
โ 1. Deploy Monitor Agent โ 2. Incident Memory Agent โ 3. Decision Agent โ
โ โข 7-dim baseline diff โ โข Vector embeddings โ โข Gemini 2.5 Flash โ
โ โข 7-dim baseline diff โ โข Vector embeddings โ โข Gemini 3.5 Flash โ
โ โข Multi-step recovery โ โข Historical lookup โ โข PolicyEngine โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโค
โ 4. Rollback Agent โ 5. Postmortem Agent โ ๐ก๏ธ Security Gateways โ
โ โข Cloud Deploy exec โ โข SRE 5-whys RCA โ โข Agent Gateway โ
โ โข 2-tier auth check โ โข Markdown generation โ โข LogSanitizer โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโ
| Agent Name | Agent ID | Service Account Identity | Permissions Held | Risk Level | Primary Domain Responsibility |
|---|---|---|---|---|---|
| Deploy Monitor | deploy-monitor-v1 |
sa-monitor@deployguard.iam.gserviceaccount.com |
monitoring.read, logging.read |
LOW |
Samples 7-dimensional metric baselines; triggers alerts; verifies post-rollback recovery. |
| Incident Memory | incident-memory-v1 |
sa-memory@deployguard.iam.gserviceaccount.com |
datastore.read, datastore.write |
LOW |
Manages Firestore incident vector bank; retrieves historical incident matches via semantic similarity. |
| Decision Engine | decision-v2 |
sa-decision@deployguard.iam.gserviceaccount.com |
gemini.invoke, memory.read |
MEDIUM |
Combines LLM reasoning, Model Armor screening, and deterministic PolicyEngine checks into a signed DecisionTrace. |
| Rollback Agent | rollback-v1 |
sa-rollback@deployguard.iam.gserviceaccount.com |
clouddeploy.releaserollback |
HIGH |
Enforces two-tier authorization check before executing Cloud Deploy releases. |
| Postmortem Agent | postmortem-v1 |
sa-postmortem@deployguard.iam.gserviceaccount.com |
datastore.write, gemini.invoke |
LOW |
Deterministically synthesizes SRE postmortems with 5-whys root cause analysis and preventative action items. |
- Python 3.12+ (or 3.13)
- Node.js 18+ & npm
- uv package manager
git clone https://github.com/sukhada20/DeployGuard.git
cd DeployGuard
# Install backend dependencies and frontend packages
make install# Start FastAPI backend & Next.js dashboard proxy on http://localhost:8000
make dev# In a separate terminal, trigger the full autonomous recovery demonstration
make demoDeployGuard includes a built-in CLI demonstration orchestrator (src/deployguard/demo/):
| Command | Mode | Description |
|---|---|---|
make demo |
Interactive | Step-by-step presentation mode with Enter key pauses to explain each stage and inspect the live dashboard. |
make demo-auto |
Timed | Automated timed playback with 1s delays (ideal for video recordings). |
make demo-ci |
Headless | Fast non-interactive run for CI pipelines and automated assertions. |
make demo-security |
Security | Runs both Agent Gateway denial and Prompt Injection defense simulations. |
make demo-security-gateway |
Security | Demonstrates Agent Gateway rejecting unauthorized action calls by DecisionAgent. |
make demo-security-injection |
Security | Demonstrates multi-stage log sanitization neutralizing prompt injections and PII keys. |
make demo-clean |
Reset | Clears mock Firestore documents, metrics, and incident memory between runs. |
๐ See DEMO.md for the complete SRE Operator Runbook and step-by-step presenter guide.
DeployGuard maintains strict quality gates across both backend Python code and frontend Next.js assets:
# Run formatters, linters, full unit & integration tests, agent benchmarks, and web production build
make verifyThe verification gate executes:
ruff format --check src/ tests/(PEP 8 code formatting)ruff check src/ tests/(Fast Python linting)mypy src/ tests/(Strict static typechecking)pytest tests/ -v -m "not live_gcp"(120 unit, e2e, and security tests)pytest tests/test_evals.py -v(DeployGuard Agent Evaluation benchmark suite)npm --prefix web run build(Next.js App Router production build)
DeployGuard is designed natively for Google Cloud Platform services:
- Compute: Cloud Run (Containerized FastAPI API & Web SPA)
- Agent Intelligence: Vertex AI (Gemini 2.5 Flash) & Google ADK
- Agent Intelligence: Vertex AI (Gemini 3.5 Flash) & Google ADK
- Security & Safety: Vertex AI Model Armor & IAM Service Accounts
- Deployment: Google Cloud Deploy & Cloud Build
- Telemetry: Google Cloud Monitoring & Cloud Logging
- Memory & Storage: Cloud Firestore (with Vector Search) & Cloud Trace
๐ See docs/DEPLOYMENT.md for the complete step-by-step GCP production deployment manual.
To test the deployed Cloud Run application continuously, you can create a test repository and configure GitHub Actions:
- Create a new GitHub repository (e.g.,
deployguard-e2e-tests). - Add a GitHub Actions workflow
.github/workflows/main.yml:
name: E2E Cloud Run Tests
on:
push:
branches: [ main ]
schedule:
- cron: '0 */6 * * *' # Run every 6 hours
jobs:
test-health:
runs-on: ubuntu-latest
steps:
- name: Check Health Endpoint
run: |
HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" https://YOUR_CLOUD_RUN_URL/api/v1/health)
if [ "$HTTP_STATUS" -ne 200 ]; then
echo "Health check failed with status $HTTP_STATUS"
exit 1
fi
echo "Health check passed!"- Replace
YOUR_CLOUD_RUN_URLwith the actual URL of your deployed Cloud Run service. - The GitHub Action will automatically run tests on every push and periodically to ensure the deployment remains healthy.
Licensed under the Apache License, Version 2.0. See LICENSE for details.