Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Every compiled pipeline runs as three sequential jobs:
acting directly.
2. **Detection (Stage 2)** — a separate agent inspects Stage 1's proposals for
prompt injection, secret leaks, and other threats.
3. **Execution (Stage 3)** — a non-agent executor applies approved safe outputs
3. **SafeOutputs (Stage 3)** — a non-agent executor applies approved safe outputs
using a write-capable ADO token that the agent never sees.

### Architecture
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ DevOps pipeline with three jobs:

```
┌────────────────────────┐ ┌──────────────────────┐ ┌───────────────────────┐
│ Agent │────▶│ Detection │────▶│ Execution
│ Agent │────▶│ Detection │────▶│ SafeOutputs
│ (Stage 1 — Agent) │ │ (Stage 2 — Threats) │ │ (Stage 3 — Executor) │
│ │ │ │ │ │
│ • Runs inside AWF │ │ • Reviews proposed │ │ • Creates PRs │
Expand Down Expand Up @@ -173,7 +173,7 @@ project. To maintain security isolation between the agent and the executor,

The agent runs in a network-isolated sandbox (AWF) with only the read token.
Even if the agent were compromised or prompt-injected, it cannot perform write
operations. Write actions are only executed in Stage 3 (`Execution`)
operations. Write actions are only executed in Stage 3 (`SafeOutputs`)
after threat analysis, using a completely separate token that the agent never
sees.

Expand Down
6 changes: 3 additions & 3 deletions docs/targets.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ The `target` field in the front matter determines the output format and executio
### `standalone` (default)

Generates a self-contained Azure DevOps pipeline with:
- Full 3-job pipeline: `Agent` → `Detection` → `Execution`
- Full 3-job pipeline: `Agent` → `Detection` → `SafeOutputs`
- AWF (Agentic Workflow Firewall) L7 domain whitelisting via Squid proxy + Docker
- MCP Gateway (MCPG) for MCP routing with SafeOutputs HTTP backend
- Setup/teardown job support
Expand All @@ -22,7 +22,7 @@ This is the recommended target for maximum flexibility and security controls.
Generates a pipeline that extends the 1ES Unofficial Pipeline Template:
- Uses `templateContext.type: buildJob` with Copilot CLI + AWF + MCPG (same execution model as standalone)
- Integrates with 1ES SDL scanning and compliance tools
- Full 3-job pipeline: Agent → Detection → Execution
- Full 3-job pipeline: Agent → Detection → SafeOutputs
- Requires 1ES Pipeline Templates repository access

Example:
Expand All @@ -38,7 +38,7 @@ Generates a **job-level ADO YAML template** with `jobs:` at root. This is a
reusable template that can be included in an existing pipeline — it does not
generate a complete pipeline.

The output contains the same 3-job chain (Agent → Detection → Execution) as
The output contains the same 3-job chain (Agent → Detection → SafeOutputs) as
`standalone`, with:
- Job names prefixed with the agent name for uniqueness (e.g., `DailyReview_Agent`)
- No triggers, pipeline name, or resource declarations (the parent pipeline owns those)
Expand Down
8 changes: 4 additions & 4 deletions docs/template-markers.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ Examples of fuzzy schedule → cron conversion:

Should be replaced with the `checkout: self` step. This generates a simple checkout of the triggering branch.

All checkout steps across all jobs (Agent, Detection, Execution, Setup, Teardown) use this marker.
All checkout steps across all jobs (Agent, Detection, SafeOutputs, Setup, Teardown) use this marker.

## {{ checkout_repositories }}
Should be replaced with checkout steps for additional repositories the agent will work with. The behavior depends on the `repos:` front-matter field (each entry's `checkout:` flag, which defaults to `true`):
Expand Down Expand Up @@ -173,7 +173,7 @@ If `setup` is empty, this is replaced with an empty string.
## {{ teardown_job }}

Generates a separate teardown job YAML if `teardown` contains steps. The job:
- Runs after `Execution` (depends on it)
- Runs after `SafeOutputs` (depends on it)
- Uses the same pool as the main agentic task
- Includes a checkout of self
- Display name: `Teardown`
Expand Down Expand Up @@ -432,7 +432,7 @@ If `permissions.read` is not configured, this marker is replaced with an empty s

## {{ acquire_write_token }}

Generates an `AzureCLI@2` step that acquires a write-capable ADO-scoped access token from the ARM service connection specified in `permissions.write`. This token is used only by the executor in Stage 3 (`Execution` job) and is never exposed to the agent.
Generates an `AzureCLI@2` step that acquires a write-capable ADO-scoped access token from the ARM service connection specified in `permissions.write`. This token is used only by the executor in Stage 3 (`SafeOutputs` job) and is never exposed to the agent.

The step:
- Uses the ARM service connection from `permissions.write`
Expand Down Expand Up @@ -525,7 +525,7 @@ jobs:
- job: DailyCodeReview_Agent
- job: DailyCodeReview_Detection
dependsOn: DailyCodeReview_Agent
- job: DailyCodeReview_Execution
- job: DailyCodeReview_SafeOutputs
dependsOn: [DailyCodeReview_Agent, DailyCodeReview_Detection]
```

Expand Down
4 changes: 2 additions & 2 deletions prompts/create-ado-agentic-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Produce a single `.md` file containing two parts:
The `ado-aw` compiler turns this into a three-job Azure DevOps pipeline:

```
Agent → Detection → Execution
Agent → Detection → SafeOutputs
(Stage 1: Agent) (Stage 2: Threat (Stage 3: Executor)
analysis)
```
Expand Down Expand Up @@ -409,7 +409,7 @@ setup: # Separate job BEFORE Agent
- bash: echo "Provisioning resources..."
displayName: "Setup"

teardown: # Separate job AFTER Execution
teardown: # Separate job AFTER SafeOutputs
- bash: echo "Cleanup..."
displayName: "Teardown"
```
Expand Down
18 changes: 9 additions & 9 deletions prompts/debug-ado-agentic-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ If the Azure DevOps `pipelines` MCP toolset (`@azure-devops/mcp`) is configured
Every `ado-aw` pipeline compiles into a three-job Azure DevOps pipeline:

```
Agent → Detection → Execution
Agent → Detection → SafeOutputs
(Stage 1: Agent) (Stage 2: Threat (Stage 3: Executor)
Analysis)
```
Expand All @@ -29,11 +29,11 @@ Agent → Detection → Execution
|-----|---------|-------|-------------|
| **Agent** | Runs the AI agent inside an AWF network sandbox (Squid proxy + Docker). Agent proposes actions via safe-output MCP tools. | Read-only (`permissions.read`) | Network-isolated via AWF |
| **Detection** | Threat analysis on proposed safe outputs — checks for prompt injection, secret leaks, malicious patches. | None | Standard ADO agent |
| **Execution** | Executes approved safe outputs (create PRs, work items, wiki pages, etc.) | Write (`permissions.write`) | Standard ADO agent |
| **SafeOutputs** | Executes approved safe outputs (create PRs, work items, wiki pages, etc.) | Write (`permissions.write`) | Standard ADO agent |

Additional optional jobs:
- **Setup** — runs before `Agent` (from `setup:` front matter)
- **Teardown** — runs after `Execution` (from `teardown:` front matter)
- **Teardown** — runs after `SafeOutputs` (from `teardown:` front matter)

---

Expand All @@ -48,7 +48,7 @@ You need minimal context from the user:
- **If multiple recent failed builds exist** → list them and ask the user which one to investigate. Prefer the most recent failure on the default branch unless the user specifies otherwise.

**If you don't have ADO MCP pipeline tools**, also ask the user for:
- Which job failed (Agent, Detection, Execution, Setup, Teardown)
- Which job failed (Agent, Detection, SafeOutputs, Setup, Teardown)
- Error messages or log snippets from the failing step
- The agent source `.md` file (or path) and the compiled `.lock.yml` (or path)

Expand Down Expand Up @@ -84,7 +84,7 @@ Map the failing timeline record to one of these categories:
| `Agent` — MCPG/MCP steps | Tool routing failure | [MCPG Issues](#mcp-gateway-mcpg-issues) |
| `Agent` — engine/run step | Agent runtime failure | [Stage 1: Agent Failures](#stage-1-agent-failures) |
| `Detection` | Threat analysis issue | [Stage 2: Detection Failures](#stage-2-detection-failures) |
| `Execution` | Safe output execution issue | [Stage 3: Execution Failures](#stage-3-execution-failures) |
| `SafeOutputs` | Safe output execution issue | [Stage 3: SafeOutputs Failures](#stage-3-safeoutputs-failures) |
| `Teardown` | Post-execution failure | [Setup/Teardown Failures](#setupteardown-failures) |
| Pipeline queued/cancelled | Resource/authorization issue | [Common Cross-Stage Issues](#common-cross-stage-issues) |

Expand Down Expand Up @@ -309,7 +309,7 @@ If genuinely a false positive, adjust the agent's instructions to produce output

### No Safe Outputs Produced

**Symptoms**: `Detection` succeeds but `Execution` has nothing to do. The agent completed without producing any mutations.
**Symptoms**: `Detection` succeeds but `SafeOutputs` has nothing to do. The agent completed without producing any mutations.

**Common causes**:

Expand All @@ -320,7 +320,7 @@ If genuinely a false positive, adjust the agent's instructions to produce output

---

## Stage 3: Execution Failures
## Stage 3: SafeOutputs Failures

This job executes the approved safe outputs using the write token. Failures here are usually ADO API errors or validation issues.

Expand Down Expand Up @@ -390,7 +390,7 @@ This job executes the approved safe outputs using the write token. Failures here

## Setup/Teardown Failures

**Setup** runs before `Agent`; **Teardown** runs after `Execution`.
**Setup** runs before `Agent`; **Teardown** runs after `SafeOutputs`.

- These use the same pool as the main agentic task — check `pool:` configuration
- They include a `checkout: self` step — check that the repository is accessible
Expand Down Expand Up @@ -488,7 +488,7 @@ If downloads fail:

## Analysis

- **Stage classification**: Stage 1 (Agent) / Stage 2 (Detection) / Stage 3 (Execution) / Setup / Teardown / Cross-stage
- **Stage classification**: Stage 1 (Agent) / Stage 2 (Detection) / Stage 3 (SafeOutputs) / Setup / Teardown / Cross-stage
- **Why this stage failed**: <detailed explanation>

## Root Cause
Expand Down
2 changes: 1 addition & 1 deletion prompts/update-ado-agentic-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ setup: # Separate job BEFORE Agent
- bash: echo "Provisioning..."
displayName: "Setup"

teardown: # Separate job AFTER Execution
teardown: # Separate job AFTER SafeOutputs
- bash: echo "Cleanup..."
displayName: "Teardown"
```
Expand Down