From 1784a5757337c255f88ada44d1497243726012b8 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 15 May 2026 13:17:06 +0000 Subject: [PATCH] docs: fix Stage 3 job name from Execution to SafeOutputs across all docs The Stage 3 job in all pipeline templates (base.yml, 1es-base.yml, job-base.yml, stage-base.yml) is named 'SafeOutputs', but documentation consistently referred to it as 'Execution'. This caused: - Wrong YAML example in docs/template-markers.md (DailyCodeReview_Execution instead of DailyCodeReview_SafeOutputs) - Incorrect dependsOn description for teardown job - Misleading job-name lists in debugging prompts (agents/users looking for an 'Execution' job in the ADO UI would not find it) Updated files: - docs/template-markers.md: fix YAML example, job-name list, and acquire_write_token description - docs/targets.md: fix 3-job chain descriptions - README.md: fix diagram box and inline reference - AGENTS.md: fix three-stage pipeline model description - prompts/create-ado-agentic-workflow.md: fix flow diagram and teardown comment - prompts/debug-ado-agentic-workflow.md: fix job table, job-name list, section header, and teardown reference - prompts/update-ado-agentic-workflow.md: fix teardown comment Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- AGENTS.md | 2 +- README.md | 4 ++-- docs/targets.md | 6 +++--- docs/template-markers.md | 8 ++++---- prompts/create-ado-agentic-workflow.md | 4 ++-- prompts/debug-ado-agentic-workflow.md | 18 +++++++++--------- prompts/update-ado-agentic-workflow.md | 2 +- 7 files changed, 22 insertions(+), 22 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 36076adb..5888c70d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -36,7 +36,7 @@ Every compiled pipeline runs as three sequential jobs: acting directly. 2. **Detection (Stage 2)** — a separate agent inspects Stage 1's proposals for prompt injection, secret leaks, and other threats. -3. **Execution (Stage 3)** — a non-agent executor applies approved safe outputs +3. **SafeOutputs (Stage 3)** — a non-agent executor applies approved safe outputs using a write-capable ADO token that the agent never sees. ### Architecture diff --git a/README.md b/README.md index 92fce68c..d0619b90 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ DevOps pipeline with three jobs: ``` ┌────────────────────────┐ ┌──────────────────────┐ ┌───────────────────────┐ -│ Agent │────▶│ Detection │────▶│ Execution │ +│ Agent │────▶│ Detection │────▶│ SafeOutputs │ │ (Stage 1 — Agent) │ │ (Stage 2 — Threats) │ │ (Stage 3 — Executor) │ │ │ │ │ │ │ │ • Runs inside AWF │ │ • Reviews proposed │ │ • Creates PRs │ @@ -173,7 +173,7 @@ project. To maintain security isolation between the agent and the executor, The agent runs in a network-isolated sandbox (AWF) with only the read token. Even if the agent were compromised or prompt-injected, it cannot perform write -operations. Write actions are only executed in Stage 3 (`Execution`) +operations. Write actions are only executed in Stage 3 (`SafeOutputs`) after threat analysis, using a completely separate token that the agent never sees. diff --git a/docs/targets.md b/docs/targets.md index 47e146c6..3501a6d3 100644 --- a/docs/targets.md +++ b/docs/targets.md @@ -9,7 +9,7 @@ The `target` field in the front matter determines the output format and executio ### `standalone` (default) Generates a self-contained Azure DevOps pipeline with: -- Full 3-job pipeline: `Agent` → `Detection` → `Execution` +- Full 3-job pipeline: `Agent` → `Detection` → `SafeOutputs` - AWF (Agentic Workflow Firewall) L7 domain whitelisting via Squid proxy + Docker - MCP Gateway (MCPG) for MCP routing with SafeOutputs HTTP backend - Setup/teardown job support @@ -22,7 +22,7 @@ This is the recommended target for maximum flexibility and security controls. Generates a pipeline that extends the 1ES Unofficial Pipeline Template: - Uses `templateContext.type: buildJob` with Copilot CLI + AWF + MCPG (same execution model as standalone) - Integrates with 1ES SDL scanning and compliance tools -- Full 3-job pipeline: Agent → Detection → Execution +- Full 3-job pipeline: Agent → Detection → SafeOutputs - Requires 1ES Pipeline Templates repository access Example: @@ -38,7 +38,7 @@ Generates a **job-level ADO YAML template** with `jobs:` at root. This is a reusable template that can be included in an existing pipeline — it does not generate a complete pipeline. -The output contains the same 3-job chain (Agent → Detection → Execution) as +The output contains the same 3-job chain (Agent → Detection → SafeOutputs) as `standalone`, with: - Job names prefixed with the agent name for uniqueness (e.g., `DailyReview_Agent`) - No triggers, pipeline name, or resource declarations (the parent pipeline owns those) diff --git a/docs/template-markers.md b/docs/template-markers.md index ed186424..0373801f 100644 --- a/docs/template-markers.md +++ b/docs/template-markers.md @@ -81,7 +81,7 @@ Examples of fuzzy schedule → cron conversion: Should be replaced with the `checkout: self` step. This generates a simple checkout of the triggering branch. -All checkout steps across all jobs (Agent, Detection, Execution, Setup, Teardown) use this marker. +All checkout steps across all jobs (Agent, Detection, SafeOutputs, Setup, Teardown) use this marker. ## {{ checkout_repositories }} Should be replaced with checkout steps for additional repositories the agent will work with. The behavior depends on the `repos:` front-matter field (each entry's `checkout:` flag, which defaults to `true`): @@ -173,7 +173,7 @@ If `setup` is empty, this is replaced with an empty string. ## {{ teardown_job }} Generates a separate teardown job YAML if `teardown` contains steps. The job: -- Runs after `Execution` (depends on it) +- Runs after `SafeOutputs` (depends on it) - Uses the same pool as the main agentic task - Includes a checkout of self - Display name: `Teardown` @@ -432,7 +432,7 @@ If `permissions.read` is not configured, this marker is replaced with an empty s ## {{ acquire_write_token }} -Generates an `AzureCLI@2` step that acquires a write-capable ADO-scoped access token from the ARM service connection specified in `permissions.write`. This token is used only by the executor in Stage 3 (`Execution` job) and is never exposed to the agent. +Generates an `AzureCLI@2` step that acquires a write-capable ADO-scoped access token from the ARM service connection specified in `permissions.write`. This token is used only by the executor in Stage 3 (`SafeOutputs` job) and is never exposed to the agent. The step: - Uses the ARM service connection from `permissions.write` @@ -525,7 +525,7 @@ jobs: - job: DailyCodeReview_Agent - job: DailyCodeReview_Detection dependsOn: DailyCodeReview_Agent - - job: DailyCodeReview_Execution + - job: DailyCodeReview_SafeOutputs dependsOn: [DailyCodeReview_Agent, DailyCodeReview_Detection] ``` diff --git a/prompts/create-ado-agentic-workflow.md b/prompts/create-ado-agentic-workflow.md index d32ea012..ffa21f45 100644 --- a/prompts/create-ado-agentic-workflow.md +++ b/prompts/create-ado-agentic-workflow.md @@ -38,7 +38,7 @@ Produce a single `.md` file containing two parts: The `ado-aw` compiler turns this into a three-job Azure DevOps pipeline: ``` -Agent → Detection → Execution +Agent → Detection → SafeOutputs (Stage 1: Agent) (Stage 2: Threat (Stage 3: Executor) analysis) ``` @@ -409,7 +409,7 @@ setup: # Separate job BEFORE Agent - bash: echo "Provisioning resources..." displayName: "Setup" -teardown: # Separate job AFTER Execution +teardown: # Separate job AFTER SafeOutputs - bash: echo "Cleanup..." displayName: "Teardown" ``` diff --git a/prompts/debug-ado-agentic-workflow.md b/prompts/debug-ado-agentic-workflow.md index eb6b8e97..41bff089 100644 --- a/prompts/debug-ado-agentic-workflow.md +++ b/prompts/debug-ado-agentic-workflow.md @@ -20,7 +20,7 @@ If the Azure DevOps `pipelines` MCP toolset (`@azure-devops/mcp`) is configured Every `ado-aw` pipeline compiles into a three-job Azure DevOps pipeline: ``` -Agent → Detection → Execution +Agent → Detection → SafeOutputs (Stage 1: Agent) (Stage 2: Threat (Stage 3: Executor) Analysis) ``` @@ -29,11 +29,11 @@ Agent → Detection → Execution |-----|---------|-------|-------------| | **Agent** | Runs the AI agent inside an AWF network sandbox (Squid proxy + Docker). Agent proposes actions via safe-output MCP tools. | Read-only (`permissions.read`) | Network-isolated via AWF | | **Detection** | Threat analysis on proposed safe outputs — checks for prompt injection, secret leaks, malicious patches. | None | Standard ADO agent | -| **Execution** | Executes approved safe outputs (create PRs, work items, wiki pages, etc.) | Write (`permissions.write`) | Standard ADO agent | +| **SafeOutputs** | Executes approved safe outputs (create PRs, work items, wiki pages, etc.) | Write (`permissions.write`) | Standard ADO agent | Additional optional jobs: - **Setup** — runs before `Agent` (from `setup:` front matter) -- **Teardown** — runs after `Execution` (from `teardown:` front matter) +- **Teardown** — runs after `SafeOutputs` (from `teardown:` front matter) --- @@ -48,7 +48,7 @@ You need minimal context from the user: - **If multiple recent failed builds exist** → list them and ask the user which one to investigate. Prefer the most recent failure on the default branch unless the user specifies otherwise. **If you don't have ADO MCP pipeline tools**, also ask the user for: -- Which job failed (Agent, Detection, Execution, Setup, Teardown) +- Which job failed (Agent, Detection, SafeOutputs, Setup, Teardown) - Error messages or log snippets from the failing step - The agent source `.md` file (or path) and the compiled `.lock.yml` (or path) @@ -84,7 +84,7 @@ Map the failing timeline record to one of these categories: | `Agent` — MCPG/MCP steps | Tool routing failure | [MCPG Issues](#mcp-gateway-mcpg-issues) | | `Agent` — engine/run step | Agent runtime failure | [Stage 1: Agent Failures](#stage-1-agent-failures) | | `Detection` | Threat analysis issue | [Stage 2: Detection Failures](#stage-2-detection-failures) | -| `Execution` | Safe output execution issue | [Stage 3: Execution Failures](#stage-3-execution-failures) | +| `SafeOutputs` | Safe output execution issue | [Stage 3: SafeOutputs Failures](#stage-3-safeoutputs-failures) | | `Teardown` | Post-execution failure | [Setup/Teardown Failures](#setupteardown-failures) | | Pipeline queued/cancelled | Resource/authorization issue | [Common Cross-Stage Issues](#common-cross-stage-issues) | @@ -309,7 +309,7 @@ If genuinely a false positive, adjust the agent's instructions to produce output ### No Safe Outputs Produced -**Symptoms**: `Detection` succeeds but `Execution` has nothing to do. The agent completed without producing any mutations. +**Symptoms**: `Detection` succeeds but `SafeOutputs` has nothing to do. The agent completed without producing any mutations. **Common causes**: @@ -320,7 +320,7 @@ If genuinely a false positive, adjust the agent's instructions to produce output --- -## Stage 3: Execution Failures +## Stage 3: SafeOutputs Failures This job executes the approved safe outputs using the write token. Failures here are usually ADO API errors or validation issues. @@ -390,7 +390,7 @@ This job executes the approved safe outputs using the write token. Failures here ## Setup/Teardown Failures -**Setup** runs before `Agent`; **Teardown** runs after `Execution`. +**Setup** runs before `Agent`; **Teardown** runs after `SafeOutputs`. - These use the same pool as the main agentic task — check `pool:` configuration - They include a `checkout: self` step — check that the repository is accessible @@ -488,7 +488,7 @@ If downloads fail: ## Analysis -- **Stage classification**: Stage 1 (Agent) / Stage 2 (Detection) / Stage 3 (Execution) / Setup / Teardown / Cross-stage +- **Stage classification**: Stage 1 (Agent) / Stage 2 (Detection) / Stage 3 (SafeOutputs) / Setup / Teardown / Cross-stage - **Why this stage failed**: ## Root Cause diff --git a/prompts/update-ado-agentic-workflow.md b/prompts/update-ado-agentic-workflow.md index e6592bff..e811f812 100644 --- a/prompts/update-ado-agentic-workflow.md +++ b/prompts/update-ado-agentic-workflow.md @@ -216,7 +216,7 @@ setup: # Separate job BEFORE Agent - bash: echo "Provisioning..." displayName: "Setup" -teardown: # Separate job AFTER Execution +teardown: # Separate job AFTER SafeOutputs - bash: echo "Cleanup..." displayName: "Teardown" ```