Skip to content

feat(compile): add awf_path_prepends for chroot PATH injection#359

Merged
jamesadevine merged 2 commits intomainfrom
feat/awf-path-prepends
Apr 29, 2026
Merged

feat(compile): add awf_path_prepends for chroot PATH injection#359
jamesadevine merged 2 commits intomainfrom
feat/awf-path-prepends

Conversation

@jamesadevine
Copy link
Copy Markdown
Collaborator

@jamesadevine jamesadevine commented Apr 29, 2026

Summary

Adds awf_path_prepends() to the CompilerExtension trait so extensions can declare directories that should be on PATH inside the AWF chroot. The Lean runtime uses this to make lean, lake, and elan discoverable without absolute paths.

Problem

After PR #354 added the $HOME/.elan AWF mount, the agent could access Lean binaries at ~/.elan/bin/lean — but lean wasn't on PATH. The agent had to do an extra step (which lean fails → ~/.elan/bin/lean -h works) before it could use Lean.

Root cause: sudo resets PATH via secure_path. AWF reconstructs PATH internally from process.env.PATH (post-sudo, missing elan). On GitHub Actions, AWF reads the $GITHUB_PATH file to recover lost paths, but no equivalent existed for ADO.

Solution

  • New CompilerExtension::awf_path_prepends() trait method — extensions declare bin directories to inject into the chroot PATH
  • LeanExtension returns ["$HOME/.elan/bin"]
  • New generate_awf_path_step() generates a dedicated pipeline step ("Generate GITHUB_PATH file") that writes path entries to a heredoc file and sets GITHUB_PATH via ##vso[task.setvariable]
  • generate_awf_path_env() appends GITHUB_PATH: $(GITHUB_PATH) to the {{ engine_env }} block on the AWF step for explicit env passthrough
  • AWF natively reads $GITHUB_PATH at startup and merges entries into the chroot PATH

Generated step (when Lean is enabled):

- bash: |
    AWF_PATH_FILE="/tmp/awf-tools/ado-path-entries"
    cat > "$AWF_PATH_FILE" << AWF_PATH_EOF
    $HOME/.elan/bin
    AWF_PATH_EOF
    echo "##vso[task.setvariable variable=GITHUB_PATH]$AWF_PATH_FILE"
  displayName: "Generate GITHUB_PATH file"

The heredoc uses an unquoted delimiter so $HOME is expanded by bash at write time — AWF reads the file as literal paths and does not perform shell expansion itself.

Changes

File Change
src/compile/extensions/mod.rs awf_path_prepends() trait method + macro dispatch
src/runtimes/lean/extension.rs Returns ["$HOME/.elan/bin"]
src/compile/common.rs generate_awf_path_step(), generate_awf_path_env(), appends to engine_env + tests
src/compile/standalone.rs Wire up {{ awf_path_step }} replacement
src/compile/onees.rs Wire up {{ awf_path_step }} replacement
src/data/base.yml {{ awf_path_step }} marker after {{ prepare_steps }}
src/data/1es-base.yml {{ awf_path_step }} marker after {{ prepare_steps }}
src/compile/extensions/tests.rs Trait default + Lean awf_path_prepends tests
docs/extending.md Add method to trait listing
docs/template-markers.md Document {{ awf_path_step }} marker

Testing

  • All 1014 tests pass
  • Clippy clean (no new warnings)
  • New tests: awf_path_prepends() default + Lean impl, generate_awf_path_step() with/without Lean, generate_awf_path_env() with/without Lean

@jamesadevine jamesadevine force-pushed the feat/awf-path-prepends branch 2 times, most recently from 079891a to e834a85 Compare April 29, 2026 16:44
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Good feature with one concrete bug that will silently corrupt generated YAML when multiple extensions contribute path entries, plus a docs/code mismatch.

Findings

🐛 Bugs / Logic Issues

  • src/compile/common.rs:1728 — Multi-path YAML corruption

    path_lines = paths.join("\n") embeds bare newlines into the format string at position {path_lines}. The first path line inherits 4-space indentation from the literal, but every subsequent path line has 0-space indentation:

    // Current — produces uneven indentation for the 2nd+ paths
    let path_lines = paths.join("\n");
    // format string: "    {path_lines}" → "    /path/one\n/path/two"

    After replace_with_indent applies the template's per-line indent (e.g., 6 spaces in base.yml), the second path becomes a 6-space line while surrounding bash lines have 10 spaces. YAML | block scalars terminate when a content line has less indentation than the block's established level — so the second path ends the block scalar, orphaning AWF_PATH_EOF and echo as top-level YAML nodes and breaking compilation.

    Today only LeanExtension uses this and it returns exactly one path, so this doesn't manifest. But the trait returns Vec<String>, and a second extension or a second Lean path would trigger the defect.

    Fix: pre-indent every path line in the format string consistently:

    let path_lines = paths.iter().map(|p| format!("    {p}")).collect::<Vec<_>>().join("\n");

⚠️ Suggestions

  • docs/template-markers.md — Example doesn't match generated output

    The documentation example shows individual echo "..." >> appends, but the implementation generates a cat > ... << AWF_PATH_EOF heredoc. The docs should show the heredoc form that is actually emitted, both for accuracy and to explain why $HOME expansion happens at write time.

  • src/compile/common.rs:1750awf_path_prepends() called twice per compile

    generate_awf_path_env independently checks awf_path_prepends() across all extensions, and generate_awf_path_step does the same. For the current single-path Lean case this is trivial, but if awf_path_prepends() were ever non-trivial, passing the paths: Vec<String> as a parameter to both functions (collected once in compile_shared) would be cleaner.

✅ What Looks Good

  • replace_with_indent already handles multi-line marker expansion cleanly — the architecture is sound; the bug is purely in path_lines not carrying consistent indentation.
  • The heredoc with unquoted delimiter (<< AWF_PATH_EOF) correctly lets bash expand $HOME at write time before AWF reads the file as literal paths — this is the right approach.
  • ##vso[task.setvariable] variable name and file path are fully hardcoded; no user-controlled input reaches the VSO command.
  • Extension-level trait default (vec![]) and macro dispatch wiring in extension_enum! are clean and follow the existing pattern exactly.
  • Test coverage is solid for the single-path happy path and the no-extension case.

Generated by Rust PR Reviewer for issue #359 · ● 423K ·

…PATH injection

Add a new CompilerExtension trait method awf_path_prepends() that lets
extensions declare directories to prepend to PATH inside the AWF chroot.

The compiler collects these paths and generates a dedicated pipeline step
(Generate GITHUB_PATH file) that writes them to a file and sets the
GITHUB_PATH env var via ##vso[task.setvariable]. AWF natively reads this
file at startup and merges entries into the chroot PATH, bypassing the
sudo secure_path reset that strips custom PATH entries.

LeanExtension declares \C:\Users\devinejames/.elan/bin so lean, lake, and elan are
discoverable by the agent without requiring absolute paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jamesadevine jamesadevine force-pushed the feat/awf-path-prepends branch from e834a85 to 7a317b1 Compare April 29, 2026 16:59
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Looks good — solid, well-scoped feature with correct design throughout. A couple of minor nits worth cleaning up before merge.

Findings

⚠️ Suggestions

  • src/compile/common.rs:1749generate_awf_path_env(awf_paths: &[String]) accepts a &[String] but only uses .is_empty() on it; the actual path contents play no role in the output. The parameter read as has_awf_paths: bool (or usize) would make the API clearer and avoid the impression that the paths influence the returned string. Low stakes, but the current signature is quietly misleading.

  • src/compile/standalone.rs:59 / src/compile/onees.rs:56let enabled_tools_args= generate_enabled_tools_args(...) is missing a space before = in both files. Looks like an accidental side-effect of the line insertion.

  • No end-to-end fixture test — the unit tests verify generate_awf_path_step / generate_awf_path_env in isolation but nothing in tests/ (or compiler integration tests) compiles a Lean pipeline and asserts that the final YAML contains both the awf_path_step block and GITHUB_PATH: $(GITHUB_PATH) in the AWF step's env: block. The latter is produced by compile_shared, which is the only place the two halves are joined. A round-trip test would close that gap.

✅ What Looks Good

  • replace_with_indent makes indentation correct automatically — I initially flagged the 0-based indentation in generate_awf_path_step, but replace_with_indent reads the template's leading whitespace and prepends it to every line of the replacement. So the step is correctly indented at 6 spaces (standalone) and 16 spaces (1ES), and the YAML block-scalar content and heredoc delimiter both end up at the right columns after YAML's own indentation stripping. No issue here.

  • /tmp/awf-tools/ existence is guaranteed — the "Prepare tooling" step (mkdir -p /tmp/awf-tools/staging, line 80 in base.yml) runs before both {{ prepare_steps }} and {{ awf_path_step }}, so the directory is always present when the path-entries file is written.

  • No injection risk from user inputawf_path_prepends() returns hardcoded strings in every current implementation; no front-matter value reaches path_lines, so there's no way for a user-controlled value to escape the heredoc or inject a ##vso[ command.

  • ##vso[task.setvariable] + explicit env: passthrough is the right two-step approach; ADO's auto-mapping is unreliable under sudo, and the explicit GITHUB_PATH: $(GITHUB_PATH) makes the intent unambiguous.

  • Trait default (vec![]) and macro dispatch are consistent with the existing required_awf_mounts pattern. Documentation in both template-markers.md and extending.md is thorough.

Generated by Rust PR Reviewer for issue #359 · ● 997.9K ·

@jamesadevine
Copy link
Copy Markdown
Collaborator Author

@copilot implement suggestions from feedback

…e collection, e2e test

Agent-Logs-Url: https://github.com/githubnext/ado-aw/sessions/7947ef47-4f5c-4df7-88bc-461d12b9f9ff

Co-authored-by: jamesadevine <4742697+jamesadevine@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 29, 2026

@copilot implement suggestions from feedback

Done in 9091383. Changes made based on review feedback:

  • Spacing fix: let enabled_tools_args=let enabled_tools_args = in standalone.rs and onees.rs
  • Cleaner API: generate_awf_path_env(&[String])generate_awf_path_env(bool) since the parameter was only used for .is_empty()
  • Avoid double collection: Added has_awf_paths: bool to CompileConfig — set once in standalone/onees from !awf_paths.is_empty(), passed into compile_shared so it no longer re-collects path prepends from extensions
  • End-to-end test: Added assertions to the existing test_lean_runtime_compiled_output test verifying that Generate GITHUB_PATH file step and GITHUB_PATH: $(GITHUB_PATH) env passthrough both appear in the compiled Lean pipeline output

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • spsprodeus21.vssps.visualstudio.com
    • Triggering command: /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-6ba70617ace46357 /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-6ba70617ace46357 /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.0fhcit/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.1uctenx4jccaludioiq5rta8c.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.0i8hln/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.1x0st2sbki7ttxff64xcexpz6.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.0nlib9/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.1xqwv2pdjusslmldctmf8oc4v.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.0w9wwa/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.27csmyo8iqueid22yd9du11tr.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.0wp51y/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.2ecbhmmuim3ifjuxng52dg4r4.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.0yrpm1/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.2fnzf4wj1ikhja5wlvtfi4n16.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.113wls/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.2h5qrtu02g6o08stvsx0zaxgz.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.11x8be/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.2r4pjpg1fvb946ialm774lyyy.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.12bxqd7epp�� /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.143gex7etni0u68do3f29mg76.0bwm8vz.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.14zchfmsadzugcyg64s6mltaw.0bwm8vz.rcgu.o g/de�� lib/rustlib/x86_--error-format=json lib/rustlib/x86_--json=diagnostic-rendered-ansi,artifacts,future-incompat lib/rustlib/x86_64-REDACTED-linux-gnu/bin/cc res-0d1f7d618793cc res-0d1f7d618793-m64 res-0d1f7d618793/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/rustcAAYQh2/symbols.o res-0d1f7d618793/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.012deb/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.0j2ue9jne345w8lkloxxzuez4.0xivby8.rcgu.o (dns block)
    • Triggering command: /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-6ba70617ace46357 /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-6ba70617ace46357 s/paste-410ab7e9compile s/paste-410ab7e9/tmp/agentic-pipeline-params-markers-9644/markers-agent.md s/paste-410ab7e9-o s/pa�� s/rustc8EX4vH/symbols.o s/mcp_http_tests-c92dab1c72107817.0btxrikvj34sniulxhqolzk4a.04hqbj7.rcgu.o s/mcp_http_tests-c92dab1c72107817.0xgpfyee70dj80pvajvxsg9ly.04hqbj7.rcgu.o s/mcp_http_testsgit s/mcp_http_testsremote s/mcp_http_testsget-url s/mcp_http_testsorigin s/mc�� s/mcp_http_tests-c92dab1c72107817.24f509c89mthguuur2wb0m1t2.04hqbj7.rcgu.o s/mcp_http_tests-c92dab1c72107817.2gjq9yxrsd5tvdrptess95h13.04hqbj7.rcgu.o s/mcp_http_tests-c92dab1c72107817.2jhhn84ik8y2ax6f9z21ecsfg.04hqbj7.rcgu.o s/mcp_http_testsgit s/mcp_http_testsremote s/mcp_http_testsget-url s/mcp_http_testsorigin (dns block)
  • spsprodweu4.vssps.visualstudio.com
    • Triggering command: /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-6ba70617ace46357 /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-6ba70617ace46357 /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.0fhcit/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.1uctenx4jccaludioiq5rta8c.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.0i8hln/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.1x0st2sbki7ttxff64xcexpz6.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.0nlib9/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.1xqwv2pdjusslmldctmf8oc4v.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.0w9wwa/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.27csmyo8iqueid22yd9du11tr.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.0wp51y/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.2ecbhmmuim3ifjuxng52dg4r4.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.0yrpm1/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.2fnzf4wj1ikhja5wlvtfi4n16.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.113wls/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.2h5qrtu02g6o08stvsx0zaxgz.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.11x8be/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.2r4pjpg1fvb946ialm774lyyy.0xivby8.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.12bxqd7epp�� /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.143gex7etni0u68do3f29mg76.0bwm8vz.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.14zchfmsadzugcyg64s6mltaw.0bwm8vz.rcgu.o g/de�� lib/rustlib/x86_--error-format=json lib/rustlib/x86_--json=diagnostic-rendered-ansi,artifacts,future-incompat lib/rustlib/x86_64-REDACTED-linux-gnu/bin/cc res-0d1f7d618793cc res-0d1f7d618793-m64 res-0d1f7d618793/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/rustcAAYQh2/symbols.o res-0d1f7d618793/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-b8bb5046e3077361.012deb/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/compiler_tests-8cc29ae2534471de.0j2ue9jne345w8lkloxxzuez4.0xivby8.rcgu.o (dns block)
    • Triggering command: /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-6ba70617ace46357 /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-6ba70617ace46357 s/paste-410ab7e9compile s/paste-410ab7e9/tmp/agentic-pipeline-params-markers-9644/markers-agent.md s/paste-410ab7e9-o s/pa�� s/rustc8EX4vH/symbols.o s/mcp_http_tests-c92dab1c72107817.0btxrikvj34sniulxhqolzk4a.04hqbj7.rcgu.o s/mcp_http_tests-c92dab1c72107817.0xgpfyee70dj80pvajvxsg9ly.04hqbj7.rcgu.o s/mcp_http_testsgit s/mcp_http_testsremote s/mcp_http_testsget-url s/mcp_http_testsorigin s/mc�� s/mcp_http_tests-c92dab1c72107817.24f509c89mthguuur2wb0m1t2.04hqbj7.rcgu.o s/mcp_http_tests-c92dab1c72107817.2gjq9yxrsd5tvdrptess95h13.04hqbj7.rcgu.o s/mcp_http_tests-c92dab1c72107817.2jhhn84ik8y2ax6f9z21ecsfg.04hqbj7.rcgu.o s/mcp_http_testsgit s/mcp_http_testsremote s/mcp_http_testsget-url s/mcp_http_testsorigin (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@jamesadevine jamesadevine merged commit 4576bc3 into main Apr 29, 2026
@jamesadevine jamesadevine deleted the feat/awf-path-prepends branch April 29, 2026 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants