feat(compile): runtime prompt loading via {{#runtime-import}} markers

.github/workflows/ado-script.yml

@@ -69,13 +69,13 @@ jobs:
         working-directory: scripts/ado-script
         run: npm run typecheck
 
-      - name: Build bundle (gate.js)
+      - name: Build bundles
         working-directory: scripts/ado-script
         run: npm run build
 
-      - name: Smoke-test bundle
+      - name: Smoke-test bundles
         working-directory: scripts/ado-script
-        run: npx vitest run -c vitest.config.smoke.ts
+        run: npm run test:smoke
 
       - name: E2E gate test
         run: cargo test --test gate_e2e -- --ignored --nocapture

AGENTS.md

@@ -61,7 +61,7 @@ Every compiled pipeline runs as three sequential jobs:
 │   │   │   ├── mod.rs    # Trait, Extension enum, collect_extensions(), re-exports
 │   │   │   ├── github.rs # Always-on GitHub MCP extension
 │   │   │   ├── safe_outputs.rs # Always-on SafeOutputs MCP extension
-│   │   │   ├── trigger_filters.rs # Trigger filter extension (gate evaluator delivery)
+│   │   │   ├── ado_script.rs # Always-on ado-script extension (gate evaluator + runtime-import resolver, per-job downloads)
 │   │   │   └── tests.rs  # Extension integration tests
 │   │   ├── codemods/     # Front-matter codemods (one file per transformation)
 │   │   │   ├── mod.rs    # Codemod struct, CODEMODS registry, runner
@@ -156,7 +156,9 @@ Every compiled pipeline runs as three sequential jobs:
 │   ├── update-ado-agentic-workflow.md # Guide for modifying an existing agentic pipeline
 │   └── debug-ado-agentic-workflow.md  # Guide for troubleshooting a failing agentic pipeline
 ├── scripts/              # Supporting scripts shipped as release artifacts
-│   ├── ado-script/       # TypeScript workspace for bundled gate.js (and future bundles)
+│   ├── ado-script/       # TypeScript workspace for bundled gate.js, import.js, and future bundles
+│   │   └── src/
+│   │       └── import/   # Runtime prompt resolver bundle
 │   └── gate.js           # Bundled gate evaluator (built from scripts/ado-script/, see docs/ado-script.md)
 ├── tests/                # Integration tests and fixtures
 ├── docs/                 # Per-concept reference documentation (see index below)
@@ -169,7 +171,7 @@ Every compiled pipeline runs as three sequential jobs:
 - **Language**: Rust (2024 edition) - Note: Rust 2024 edition exists and is the edition used by this project
 - **CLI Framework**: clap v4 with derive macros
 - **Error Handling**: anyhow for ergonomic error propagation
-- **Bundled scripts**: TypeScript + ncc (`scripts/ado-script/`) — compiled gate evaluator and future internal helpers; see [`docs/ado-script.md`](docs/ado-script.md).
+- **Bundled scripts**: TypeScript + ncc (`scripts/ado-script/`) — compiled gate evaluator, runtime import resolver, and future internal helpers; see [`docs/ado-script.md`](docs/ado-script.md).
 - **Async Runtime**: tokio with full features
 - **YAML Parsing**: serde_yaml
 - **MCP Server**: rmcp with server and transport-io features
@@ -193,6 +195,8 @@ index to jump to the right page.
 
 - [`docs/front-matter.md`](docs/front-matter.md) — full agent file format
   (markdown body + YAML front matter grammar) with every supported field.
+- [`docs/runtime-imports.md`](docs/runtime-imports.md) — runtime prompt import
+  markers, path resolution, and `inlined-imports:` behavior.
 - [`docs/schedule-syntax.md`](docs/schedule-syntax.md) — fuzzy schedule time
   syntax (`daily around 14:00`, `weekly on monday`, timezones, scattering).
 - [`docs/engine.md`](docs/engine.md) — `engine:` configuration (model,
@@ -239,7 +243,7 @@ index to jump to the right page.
   adding codemods.
 - [`docs/ado-script.md`](docs/ado-script.md) — `ado-script` workspace
   (`scripts/ado-script/`): the bundled TypeScript runtime helpers (today:
-  `gate.js`), schemars-driven type codegen, and the A2 design decision.
+  `gate.js` and `import.js`), schemars-driven type codegen, and the A2 design decision.
 - [`docs/local-development.md`](docs/local-development.md) — local development
   setup notes.
 

docs/ado-script.md

@@ -3,8 +3,9 @@
 `ado-script` is the umbrella name for the TypeScript workspace at
 [`scripts/ado-script/`](../scripts/ado-script/). It produces small,
 ncc-bundled Node programs that the **compiler injects into every emitted
-pipeline** as runtime helpers. The first (and currently only) bundle is
-`gate.js`, the trigger-filter gate evaluator.
+pipeline** as runtime helpers. Today it produces `gate.js`, the
+trigger-filter gate evaluator, and `import.js`, the runtime prompt
+resolver described in [`runtime-imports.md`](runtime-imports.md).
 
 > **Internal-only.** `ado-script` is not a user-facing front-matter
 > feature. Authors never write an `ado-script:` block in their agent
@@ -33,6 +34,35 @@ discriminated union. There is no `eval`, no `Function`, no `vm` — a
 compromised compiler cannot use the spec to run arbitrary code on the
 pipeline runner.
 
+## What `import.js` does
+
+`import.js` is a single-shot Node program. It reads the prompt file path
+from `argv[2]` and resolves `{{#runtime-import path}}` markers in place.
+The compiler runs it as a post-prepare-prompt step when
+[`inlined-imports: false`](front-matter.md#inlined-imports). See
+[`runtime-imports.md`](runtime-imports.md) for the author-facing marker
+syntax.
+
+### Env-var contract
+
+`import.js` takes no environment variables. Relative-path markers
+resolve against `dirname(argv[2])`; in pipeline use this is irrelevant
+because the compiler always embeds an absolute marker path and
+`import.js` is single-pass (nested markers inside the inlined body are
+not re-expanded).
+
+The bundle lives at `dist/import/index.js` and ships in the same
+`ado-script.zip` release asset as `gate.js`, so pipelines download it
+through the same Setup-job asset flow. `import.js` uses only the Node
+standard library, so the ncc bundle is small (~1.5 KB) and carries no
+SDK dependency.
+
+The Stage-2 threat-analysis prompt is **not** runtime-imported.
+`src/data/threat-analysis.md` is `include_str!`'d into the `ado-aw`
+binary and inlined into the emitted YAML at compile time, matching
+gh-aw's pattern (their `threat_detection.md` ships with the setup
+action and is read directly from disk — no marker, no resolver).
+
 ## End-to-end data flow
 
 ```
@@ -147,14 +177,19 @@ scripts/ado-script/
 │   │   ├── env-facts.ts         # Pipeline-variable readers + ENV_BY_FACT + BRANCH_FACTS + ref-prefix stripping
 │   │   ├── policy.ts            # PolicyTracker state machine
 │   │   └── vso-logger.ts        # ##vso[…] emitters with property/message escaping; complete() is idempotent
-│   └── gate/                    # gate.js entry point + per-concern modules
-│       ├── index.ts             # main(): decode → preflight → bypass → facts → eval → emit
-│       ├── bypass.ts            # build-reason auto-pass
-│       ├── facts.ts             # fact acquisition (env + REST)
-│       ├── predicates.ts        # 11 predicate evaluators + validatePredicateTree + glob ReDoS hardening
-│       └── selfcancel.ts        # best-effort build cancellation
+│   ├── gate/                    # gate.js entry point + per-concern modules
+│   │   ├── index.ts             # main(): decode → preflight → bypass → facts → eval → emit
+│   │   ├── bypass.ts            # build-reason auto-pass
+│   │   ├── facts.ts             # fact acquisition (env + REST)
+│   │   ├── predicates.ts        # 11 predicate evaluators + validatePredicateTree + glob ReDoS hardening
+│   │   └── selfcancel.ts        # best-effort build cancellation
+│   └── import/                  # import.js entry point + runtime prompt resolver
+│       ├── index.ts             # main(): expand runtime-import markers in place
+│       └── __tests__/           # marker, path-resolution, and single-pass coverage
 ├── test/                        # End-to-end smoke tests
-└── dist/gate/index.js           # ncc bundle output (gitignored)
+└── dist/                        # ncc bundle output (gitignored)
+    ├── gate/index.js
+    └── import/index.js
 ```
 
 The release workflow (`.github/workflows/release.yml`) runs
@@ -195,11 +230,18 @@ The Rust subcommand that emits the schema is intentionally hidden:
 cargo run -- export-gate-schema --output schema/gate-spec.schema.json
 ```
 
-## How the gate bundle is wired into emitted pipelines
+## How the bundles are wired into emitted pipelines
+
+`AdoScriptExtension`
+(`src/compile/extensions/ado_script.rs`) is the always-on single
+extension that owns all `ado-script` wiring. It has two independent
+features, each emitted **into the job that actually consumes the
+bundle**:
 
-`TriggerFiltersExtension`
-(`src/compile/extensions/trigger_filters.rs`) injects three Setup-job
-steps when any `filters:` block is active:
+### Setup job (gate evaluator)
+
+When `filters:` lowers to non-empty checks, `setup_steps()` returns
+three step strings into the Setup job:
 
 1. **`NodeTool@0`** — installs Node 20.x LTS, capped at
    `timeoutInMinutes: 5`.
@@ -209,9 +251,42 @@ steps when any `filters:` block is active:
    `unzip -o /tmp/ado-aw-scripts/ado-script.zip -d /tmp/ado-aw-scripts/`.
    Also capped at `timeoutInMinutes: 5`.
 3. **`bash: node '/tmp/ado-aw-scripts/ado-script/dist/gate/index.js'`** —
-   runs the gate with `GATE_SPEC` and the env-var contract above.
-
-The IR-to-bash codegen that produces these steps is
+   runs the gate with `GATE_SPEC` and the env-var contract documented
+   above.
+
+### Agent job (runtime-import resolver)
+
+When `inlined-imports: false` (the default), `prepare_steps()` returns
+the same install + download pair plus the resolver invocation, into
+the Agent job's existing `{{ prepare_steps }}` block:
+
+1. **`NodeTool@0`** — same shape as above.
+2. **`curl` download + verify + extract** — same artefact, same
+   verification.
+3. **`bash: node '/tmp/ado-aw-scripts/ado-script/dist/import/index.js'`** —
+   expands `{{#runtime-import …}}` markers in
+   `/tmp/awf-tools/agent-prompt.md` in place. See
+   [`runtime-imports.md`](runtime-imports.md) for marker syntax.
+
+### Per-job download (NOT a duplication bug)
+
+ADO jobs use **isolated VMs** — `/tmp` is not shared between jobs.
+The `ado-script.zip` bundle therefore has to be downloaded once per
+job that consumes it. When both features are active (a pipeline with
+both `filters:` and `inlined-imports: false`), install + download
+steps appear in **both** Setup and Agent. That's correct architecture
+given ADO's topology, not waste.
+
+### What gets emitted, by case
+
+| `filters:` | `inlined-imports` | Setup-job steps | Agent-job extra steps |
+|---|---|---|---|
+| inactive   | `true`  | (none)                              | (none)                              |
+| inactive   | `false` | (no Setup job)                      | install + download + resolver       |
+| active     | `true`  | install + download + gate           | (none)                              |
+| active     | `false` | install + download + gate           | install + download + resolver       |
+
+The IR-to-bash codegen that produces the gate step is
 `compile_gate_step_external` in `src/compile/filter_ir.rs`.
 
 ## Modifying `ado-script`

docs/filter-ir.md

@@ -350,24 +350,30 @@ The bash shim exports only the ADO macros needed by the spec's facts:
 
 ## Integration Points
 
-### TriggerFiltersExtension
+### AdoScriptExtension
 
-When Tier 2/3 filters are configured, the `TriggerFiltersExtension`
-(`src/compile/extensions/trigger_filters.rs`) activates via
-`collect_extensions()`. It implements `CompilerExtension` and controls:
+When `filters:` is configured (and lowers to non-empty checks), the
+always-on `AdoScriptExtension`
+(`src/compile/extensions/ado_script.rs`) emits the gate-side steps via
+the `setup_steps()` trait hook. The extension also owns the unrelated
+runtime-import resolver — see [`runtime-imports.md`](runtime-imports.md).
+
+For the gate path it controls:
 
 1. **Node install step** — emits a `NodeTool@0` step pinned to Node 20.x
-   LTS so `gate.js` has a runtime
+   LTS so `gate.js` has a runtime.
 2. **Download step** — fetches `ado-script.zip` from the ado-aw release
    artifacts, verifies its SHA256 checksum via `checksums.txt`, then
-   extracts `gate.js` to `/tmp/ado-aw-scripts/ado-script/dist/gate/index.js`
+   extracts `gate.js` to `/tmp/ado-aw-scripts/ado-script/dist/gate/index.js`.
 3. **Gate step** — calls `compile_gate_step_external()` to generate a step
-   that runs `node /tmp/ado-aw-scripts/ado-script/dist/gate/index.js` (no inline heredoc)
+   that runs `node /tmp/ado-aw-scripts/ado-script/dist/gate/index.js` (no inline heredoc).
 4. **Validation** — runs `validate_pr_filters()` / `validate_pipeline_filters()`
-   during compilation via the `validate()` trait method
+   during compilation via the `validate()` trait method.
 
-The extension uses the `setup_steps()` trait method (not `prepare_steps()`)
-because the gate must run in the **Setup job** (before the Agent job).
+The gate-side steps use `setup_steps()` (not `prepare_steps()`)
+because the gate must run in the **Setup job**, before the Agent job.
+Runtime-import resolver steps for the agent body use `prepare_steps()` and
+land in the Agent job — see [`runtime-imports.md`](runtime-imports.md).
 
 ### Tier 1 Inline Path
 
@@ -379,7 +385,7 @@ no Node evaluator and no download step.
 ### Gate Step Injection
 
 Gate steps are injected into the Setup job by `generate_setup_job()` in
-`common.rs`. When the `TriggerFiltersExtension` is active, its
+`common.rs`. When the `AdoScriptExtension` is active, its
 `setup_steps()` are collected and injected first (download + gate). When
 only Tier 1 filters are present, the inline gate step is injected directly.
 

docs/front-matter.md

@@ -234,6 +234,27 @@ becomes a `repos:` entry, with `checkout: false` added for entries
 that weren't listed under `checkout:`. Mixing the legacy fields with
 an existing `repos:` block is rejected; pick one shape.
 
+## Inlined Imports
+
+The `inlined-imports:` field controls when `{{#runtime-import ...}}`
+markers in the markdown body are resolved. It defaults to `false`.
+See [`runtime-imports.md`](runtime-imports.md) for the full marker
+syntax, path resolution rules, and runtime behavior.
+
+When `inlined-imports: false`, the compiler leaves runtime-import
+markers to be resolved on the pipeline runner. This is the default
+behavior, and it means prompt-body edits do not require recompiling the
+generated YAML.
+
+When `inlined-imports: true`, the compiler resolves all runtime-import
+markers at compile time, including the implicit top-level marker that
+normally reloads the body itself. The emitted YAML contains the fully
+expanded prompt body, so the pipeline file is self-contained.
+
+The trade-off is that the generated YAML is larger, and prompt-body
+edits require `ado-aw compile` plus committing the updated pipeline
+file.
+
 ## Filter Validation
 
 The compiler validates filter configurations at compile time and will emit

docs/runtime-imports.md

@@ -0,0 +1,88 @@
+# Runtime Imports
+
+_Part of the [ado-aw documentation](../AGENTS.md)._
+
+Runtime imports let agent prompts pull in snippet files with gh-aw-compatible
+`{{#runtime-import ...}}` markers. They are available in the markdown body and
+are controlled by the [`inlined-imports:` field](front-matter.md#inlined-imports).
+The runtime bundle that expands them is documented in [`ado-script.md`](ado-script.md).
+
+## Marker syntax
+
+Use `{{#runtime-import path}}` for a required import. If the target file is
+missing, resolution fails.
+
+```markdown
+## Repository policy
+{{#runtime-import docs/policy.md}}
+```
+
+Use `{{#runtime-import? path}}` for an optional import. If the target file is
+missing, the marker is replaced with an empty string.
+
+```markdown
+## Local notes
+{{#runtime-import? docs/local-notes.md}}
+```
+
+## Where markers can appear
+
+Authors can place runtime-import markers anywhere in the agent markdown body.
+When `inlined-imports: false` (the default), the compiler also injects an
+implicit top-level runtime-import marker that reloads the body itself at
+pipeline runtime instead of embedding it into the generated YAML. When
+`inlined-imports: true`, that implicit body marker is resolved at compile time
+along with any author-written markers.
+
+## Path resolution
+
+- **Author-written markers** must use **relative paths** rooted at the agent
+  `.md` file's directory. Absolute paths and `..` segments are rejected. This
+  protects the compile host (`ado-aw compile`, which may run on a CI agent
+  carrying privileged material like SSH keys and service-connection tokens)
+  from untrusted PR branches embedding host files into the compiled YAML —
+  e.g. `{{#runtime-import /home/runner/.ssh/id_rsa}}` or
+  `{{#runtime-import ../../../../etc/passwd}}` are both compile-time errors.
+- **Compiler-generated marker for the agent body** uses an absolute path
+  (`$(Build.SourcesDirectory)/…`) built from the trigger-repo checkout root,
+  so the runtime resolver never has to resolve a relative path. The
+  compile-time restriction does not apply here because the path is
+  tooling-generated, not author-supplied.
+
+## Single-pass behavior
+
+Runtime imports are expanded in a single pass. Imported snippets are inserted
+verbatim, and any nested `{{#runtime-import ...}}` or
+`{{#runtime-import? ...}}` markers inside those snippets are **not** expanded.
+This matches gh-aw's runtime-import behavior.
+
+## Resolver ordering
+
+The runtime-import resolver runs first. Any extension supplements that are
+appended later with `cat >>` — including SafeOutputs guidance, GitHub MCP
+guidance, runtime guidance, and cache-memory guidance — are added after import
+resolution and are left untouched.
+
+## Failure modes
+
+| Marker kind | Missing file behavior |
+|---|---|
+| `{{#runtime-import path}}` | Resolver exits with status 1 and the pipeline fails. |
+| `{{#runtime-import? path}}` | Marker is silently replaced with an empty string. |
+
+When `inlined-imports: true`, the same required/optional rules are applied at
+compile time instead of on the pipeline runner.
+
+## Implementation notes
+
+- **Runtime**: `dist/import/index.js` is ncc-bundled into `ado-script.zip`.
+  The always-on `AdoScriptExtension`'s `prepare_steps()` injects three
+  steps into the Agent job's existing `{{ prepare_steps }}` block:
+  `NodeTool@0` install, the `ado-script.zip` download/verify/extract,
+  and the `node import.js` resolver invocation. All three run on the
+  same VM as the agent — ADO jobs are VM-isolated, so the bundle must
+  be downloaded inside whichever job consumes it.
+- **Compile time**: `resolve_imports_inline()` in
+  `src/compile/extensions/ado_script.rs` performs the inline expansion
+  when `inlined-imports: true`.
+