Skip to content

feat(safeoutputs): add work-item filing to noop and missing-tool safe outputs#521

Draft
Copilot wants to merge 6 commits into
mainfrom
copilot/add-optional-configurations-for-azure-devops
Draft

feat(safeoutputs): add work-item filing to noop and missing-tool safe outputs#521
Copilot wants to merge 6 commits into
mainfrom
copilot/add-optional-configurations-for-azure-devops

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 12, 2026

Adds work-item filing behaviour to noop and missing-tool safe outputs. The Stage 3 executor always files an Azure DevOps work item or appends a comment to an existing one when these diagnostic tools are invoked. All fields have sensible defaults so no front-matter configuration is required.

Summary

  • src/safeoutputs/mod.rs — shared infrastructure:

    • WorkItemReportConfig: configures title, work-item-type, area/iteration paths, tags, and stats inclusion
    • wiql_find_work_item_by_title(): WIQL search for a non-closed item matching the configured title
    • file_or_append_work_item(): comment on existing item if found; create new one otherwise. Returns a warning (not a failure) when ADO credentials are absent or the WIQL call fails, so these always-on tools never block the pipeline
  • src/safeoutputs/noop.rsNoopConfig with a work_item: WorkItemReportConfig field that defaults to a Task titled "Agent reported no operation"; executor always calls the shared helper

  • src/safeoutputs/missing_tool.rsMissingToolConfig with a work_item: WorkItemReportConfig field that defaults to a Task titled "Agent encountered missing tool"; executor always calls the shared helper

  • docs/safe-outputs.md — documents the always-on behaviour and all override options

Both tools work with zero configuration. Override any field in front matter to customise the work item:

safe-outputs:
  noop:
    work-item:
      title: "Agent reported no operation"   # default
      work-item-type: Task                   # default
      area-path: "MyProject\\MyTeam"
      tags:
        - agent-noop

  missing-tool:
    work-item:
      title: "Agent encountered missing tool" # default
      work-item-type: Task                    # default
      area-path: "MyProject\\MyTeam"
      tags:
        - agent-missing-tool

Stage 3 runs a WIQL query to find any non-closed item with the configured title. If found, a comment is appended; otherwise a new work item is created. If ADO credentials are not available the executor succeeds with a warning instead of failing hard.

Test plan

cargo test — all existing and new unit tests pass. New tests cover config deserialization, default field values, and the graceful-warning fallback when ADO credentials are not present.

Copilot AI and others added 2 commits May 12, 2026 17:43
…e outputs

Agent-Logs-Url: https://github.com/githubnext/ado-aw/sessions/f7059573-7767-4cc5-b10c-53fac12aa232

Co-authored-by: jamesadevine <4742697+jamesadevine@users.noreply.github.com>
…D handling

Agent-Logs-Url: https://github.com/githubnext/ado-aw/sessions/f7059573-7767-4cc5-b10c-53fac12aa232

Co-authored-by: jamesadevine <4742697+jamesadevine@users.noreply.github.com>
@jamesadevine
Copy link
Copy Markdown
Collaborator

@copilot implement sensible defaults for the workitem. These safeoutputs always should now create or append to a work item.

… items with sensible defaults

Agent-Logs-Url: https://github.com/githubnext/ado-aw/sessions/e309e3af-fbce-4cfe-91d8-420be5233943

Co-authored-by: jamesadevine <4742697+jamesadevine@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 12, 2026

@copilot implement sensible defaults for the workitem. These safeoutputs always should now create or append to a work item.

Done in d0257b8. Both tools now always file or append to a work item with no config required:

  • noop → defaults to a Task titled "Agent reported no operation"
  • missing-tool → defaults to a Task titled "Agent encountered missing tool"

The work-item block in front matter lets you override any field (title, type, area/iteration path, tags). Since these are always-on diagnostic tools, if ADO credentials aren't available or the WIQL search fails, the executor returns a warning (success=true) rather than failing the pipeline hard.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • spsprodeus21.vssps.visualstudio.com
    • Triggering command: /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-3136f96127ec4700 /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-3136f96127ec4700 /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/init_tests-6da1eb8e557c50ba.1r/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/bash_lint_tests-519976ce551c8145.29d554253a7ua8jdy2djiuujt.10kbr5c.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/init_tests-6da1eb8e557c50ba.1w/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/bash_lint_tests-519976ce551c8145.2bikq83kqxzd59u4d0aoev3vs.10kbr5c.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/init_tests-6da1eb8e557c50ba.22/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/bash_lint_tests-519976ce551c8145.2boqvweqww93ilwsx0c22c3r0.10kbr5c.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/init_tests-6da1eb8e557c50ba.2o/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/bash_lint_tests-519976ce551c8145.2d2vuk299nc534pripnny1rfr.10kbr5c.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/init_tests-6da1eb8e557c50ba.2u/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/bash_lint_tests-519976ce551c8145.2hkf7twmmm5z0ktz2vjsee9ue.10kbr5c.rcgu.o f.006607excnbrprqp637f7y53i.0ywnp6d.rcgu.o f.04ihlpyw73h878m5ii3u14z4w.0ywnp6d.rcgu.o f.05s581wfkj8h57yyfztyfopmr.0ywnp6d.rcgu.o f.0a5uivvqg0yudio4gs881kurk.0ywnp6d.rcgu.o f.0avko3xidh2wk8781opolbaij.0ywnp6d.rcgu.o f.0g5xkpur9po42a3k6jocdfert.0ywnp6d.rcgu.o f.0gsn8ev1cnkk8096ejd9eaqnc.0ywnp6d.rcgu.o f.0hnyc0p3974v12h5g8kkwz19v.0ywnp6d.rcgu.o f.0xonaogqri1qh024baqfa7x2d.0ywnp6d.rcgu.o f.12nfqzyx5t92gdliu6u9j2t23.0ywnp6d.rcgu.o f.1bcwhu0k0xfbj80ouktlgq7gg.0ywnp6d.rcgu.o f.1bhjf3uobqgssktgi0e14aqnn.0ywnp6d.rcgu.o f.1hqz8k3c94266bfgrfbuybt2z.0ywnp6d.rcgu.o f.1huk6ich8t6qwyn0rf9m9q3l0.0ywnp6d.rcgu.o (dns block)
    • Triggering command: /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-3136f96127ec4700 /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-3136f96127ec4700 f.0a5uivvqg0yudio4gs881kurk.0ywnp6d.rcgu.o f.0avko3xidh2wk8781opolbaij.0ywnp6d.rcgu.o f.0g5xkpur9po42a3k6jocdfert.0ywnp6d.rcgu.o f.0gsn8ev1cnkk8096ejd9eaqnc.0ywnp6d.rcgu.o f.0hnyc0p3974v12h5g8kkwz19v.0ywnp6d.rcgu.o f.0xonaogqri1qh024baqfa7x2d.0ywnp6d.rcgu.o f.12nfqzyx5t92gdliu6u9j2t23.0ywnp6d.rcgu.o f.1bcwhu0k0xfbj80ouktlgq7gg.0ywnp6d.rcgu.o f.1bhjf3uobqgssktgi0e14aqnn.0ywnp6d.rcgu.o f.1hqz8k3c94266bfgrfbuybt2z.0ywnp6d.rcgu.o f.1huk6ich8t6qwyn0rf9m9q3l0.0ywnp6d.rcgu.o f.1t�� f.270d89qd8zd7pddtw479jvwb6.0ywnp6d.rcgu.o f.2i0rx46105zwgayhinkt9jur2.0ywnp6d.rcgu.o mbols.o d12dc65cce50cb.agit d12dc65cce50cb.aremote d12dc65cce50cb.aget-url d12dc65cce50cb.aorigin (dns block)
  • spsprodweu4.vssps.visualstudio.com
    • Triggering command: /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-3136f96127ec4700 /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-3136f96127ec4700 /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/init_tests-6da1eb8e557c50ba.1r/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/bash_lint_tests-519976ce551c8145.29d554253a7ua8jdy2djiuujt.10kbr5c.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/init_tests-6da1eb8e557c50ba.1w/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/bash_lint_tests-519976ce551c8145.2bikq83kqxzd59u4d0aoev3vs.10kbr5c.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/init_tests-6da1eb8e557c50ba.22/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/bash_lint_tests-519976ce551c8145.2boqvweqww93ilwsx0c22c3r0.10kbr5c.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/init_tests-6da1eb8e557c50ba.2o/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/bash_lint_tests-519976ce551c8145.2d2vuk299nc534pripnny1rfr.10kbr5c.rcgu.o /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/init_tests-6da1eb8e557c50ba.2u/home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/bash_lint_tests-519976ce551c8145.2hkf7twmmm5z0ktz2vjsee9ue.10kbr5c.rcgu.o f.006607excnbrprqp637f7y53i.0ywnp6d.rcgu.o f.04ihlpyw73h878m5ii3u14z4w.0ywnp6d.rcgu.o f.05s581wfkj8h57yyfztyfopmr.0ywnp6d.rcgu.o f.0a5uivvqg0yudio4gs881kurk.0ywnp6d.rcgu.o f.0avko3xidh2wk8781opolbaij.0ywnp6d.rcgu.o f.0g5xkpur9po42a3k6jocdfert.0ywnp6d.rcgu.o f.0gsn8ev1cnkk8096ejd9eaqnc.0ywnp6d.rcgu.o f.0hnyc0p3974v12h5g8kkwz19v.0ywnp6d.rcgu.o f.0xonaogqri1qh024baqfa7x2d.0ywnp6d.rcgu.o f.12nfqzyx5t92gdliu6u9j2t23.0ywnp6d.rcgu.o f.1bcwhu0k0xfbj80ouktlgq7gg.0ywnp6d.rcgu.o f.1bhjf3uobqgssktgi0e14aqnn.0ywnp6d.rcgu.o f.1hqz8k3c94266bfgrfbuybt2z.0ywnp6d.rcgu.o f.1huk6ich8t6qwyn0rf9m9q3l0.0ywnp6d.rcgu.o (dns block)
    • Triggering command: /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-3136f96127ec4700 /home/REDACTED/work/ado-aw/ado-aw/target/debug/deps/ado_aw-3136f96127ec4700 f.0a5uivvqg0yudio4gs881kurk.0ywnp6d.rcgu.o f.0avko3xidh2wk8781opolbaij.0ywnp6d.rcgu.o f.0g5xkpur9po42a3k6jocdfert.0ywnp6d.rcgu.o f.0gsn8ev1cnkk8096ejd9eaqnc.0ywnp6d.rcgu.o f.0hnyc0p3974v12h5g8kkwz19v.0ywnp6d.rcgu.o f.0xonaogqri1qh024baqfa7x2d.0ywnp6d.rcgu.o f.12nfqzyx5t92gdliu6u9j2t23.0ywnp6d.rcgu.o f.1bcwhu0k0xfbj80ouktlgq7gg.0ywnp6d.rcgu.o f.1bhjf3uobqgssktgi0e14aqnn.0ywnp6d.rcgu.o f.1hqz8k3c94266bfgrfbuybt2z.0ywnp6d.rcgu.o f.1huk6ich8t6qwyn0rf9m9q3l0.0ywnp6d.rcgu.o f.1t�� f.270d89qd8zd7pddtw479jvwb6.0ywnp6d.rcgu.o f.2i0rx46105zwgayhinkt9jur2.0ywnp6d.rcgu.o mbols.o d12dc65cce50cb.agit d12dc65cce50cb.aremote d12dc65cce50cb.aget-url d12dc65cce50cb.aorigin (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title feat(safeoutputs): add work-item config for noop and missing-tool safe outputs feat(safeoutputs): add work-item filing to noop and missing-tool safe outputs May 12, 2026
Copilot AI requested a review from jamesadevine May 12, 2026 20:01
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Has one correctness bug (silent config loss) that should be fixed before merge; everything else looks solid.

Findings

🐛 Bugs / Logic Issues

  • src/safeoutputs/mod.rs (WorkItemReportConfig)title is a bare String with no #[serde(default = ...)]. If a user provides a partial work-item: override (e.g., only work-item-type: Bug) without specifying title:, serde fails to deserialize WorkItemReportConfig, and get_tool_config's unwrap_or_default() silently falls back to the full struct default — losing all the user's overrides with no error or warning. The user has no way to know their configuration was discarded.

    Fix: either give title an explicit #[serde(default)] via a context-appropriate fallback function (but since the struct is shared across noop/missing-tool, this is tricky), or make title Option<String> and resolve the effective title at call sites with the context-appropriate default.

⚠️ Suggestions

  • src/safeoutputs/mod.rs:wiql_find_work_item_by_title — A fresh reqwest::Client is created per call. This is inconsequential in practice (at most one call per pipeline run), but it's a minor divergence from ADO REST callers elsewhere in the codebase which also create their own clients. No action required — just noting for consistency.

  • src/safeoutputs/mod.rs:file_or_append_work_item (~line 560)ExecutionResult::failure() is returned (not a hard ? bail) when the comment POST or work item POST returns a non-2xx status. This is the right call given the always-on nature of the tools, and it's consistent with the credential-missing path. Just make sure callers/tests cover the failure-result path.

✅ What Looks Good

  • Graceful-warning fallback for missing ADO credentials is correct and necessary — noop/missing-tool should never block a pipeline.
  • Sanitization chain is complete: execute_sanitized() calls sanitize_content_fields() before execute_impl() so agent-supplied context/tool_name are sanitized before entering the work item body; get_tool_config calls sanitize_config_fields() on the operator-controlled config.
  • WIQL injection protection: single-quote doubling (replace('\'', "''")) is the correct WIQL escaping convention, and the title comes from sanitized operator-controlled front matter, not agent input.
  • multilineFieldsFormat path follows the established pattern already used in create_work_item.rs and update_work_item.rs.
  • Test coverage is good: defaults, partial deserialization, and the no-credentials warning path are all exercised.

Generated by Rust PR Reviewer for issue #521 · ● 791.4K ·

jamesadevine and others added 2 commits May 12, 2026 23:22
… file

When create-work-item has no explicit assignee configured in front
matter, Stage 3 now falls back to the email of the person who last
committed changes to the agent source markdown file. The lookup uses
git log -1 --format=%ae and degrades gracefully (no assignee set)
when git history is unavailable (e.g. shallow clone).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
WorkItemReportConfig.title was a bare String with no serde default.
When users provided a partial work-item config (e.g. only
work-item-type: Bug), serde deserialization failed and
get_tool_config's unwrap_or_default() silently discarded all
overrides with no error or warning.

Fix: change title to Option<String> with #[serde(default)]. Each
caller (noop, missing-tool) passes its context-specific default
title to file_or_append_work_item, which resolves the effective
title via .unwrap_or(default_title).

Add regression tests for partial work-item configs to both noop
and missing-tool.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Has one meaningful bug that breaks the stated design guarantee — the "never blocks the pipeline" promise is violated in HTTP error paths.

Findings

🐛 Bugs / Logic Issues

  • src/safeoutputs/mod.rs — HTTP failure paths return hard failure, breaking the "always-on" promise

    The PR description and docs state these tools "never block the pipeline" — the missing-credentials and WIQL-failure cases correctly return ExecutionResult::warning. But two HTTP error paths return ExecutionResult::failure (success=false), which causes main.rs:292-293 to call std::process::exit(1) and fail Stage 3:

    // comment-append failure (line ~614)
    Ok(ExecutionResult::failure(format!(
        "Failed to append comment to work item #{} (HTTP {}): {}",
        ...
    )))
    
    // work-item creation failure (line ~680)
    Ok(ExecutionResult::failure(format!(
        "Failed to create work item (HTTP {}): {}",
        ...
    )))

    Both should be ExecutionResult::warning(...) to match the pattern established for the missing-credentials and WIQL-failure paths above them. Any transient ADO 5xx or permissions issue would otherwise silently kill the pipeline — the exact scenario this design is supposed to be immune to.

  • src/execute.rs — test assertion removed without replacement

    The assertion assert!(result.message.contains("No operation")) was deleted. While the behaviour has changed (good), nothing in the test now verifies the warning message content, so a regression that produces an empty or misleading message would pass undetected. Consider asserting exec.message.contains("not set") or similar.

⚠️ Suggestions

  • src/main.rs:discover_last_committerpath.file_name()? silently returns None for unusual paths

    Paths like Path::new("") or paths ending in .. return None from file_name(). The ? in an Option-returning async block handles this correctly (returns None), but the failure is never logged. A log::debug!("git log skipped: no file_name for {}", path.display()) before the ? return would aid diagnostics — consistent with how all other failure cases in this function are logged.

  • src/safeoutputs/mod.rs:wiql_find_work_item_by_title — WIQL single-quote escaping is correct but worth a test

    The comment correctly explains that doubling single quotes is the WIQL convention. Since the title is operator-controlled front-matter (not agent input), injection risk is low. A unit test for a title containing ' (e.g. "agent's task") confirming the escaping produces "agent''s task" would guard against future regressions.

✅ What Looks Good

  • The three-tier graceful-degradation structure (no credentials → WIQL failure → HTTP call) is well-designed and clearly commented.
  • #[derive(SanitizeConfig)] on WorkItemReportConfig properly sanitizes all String/Option<String>/Vec<String> fields including title, area_path, and tags before they reach the WIQL query.
  • discover_last_committer runs git in a subprocess and handles all error variants without panicking — the fallback-to-None approach is exactly right for an optional enrichment feature.
  • Config deserialization tests cover defaults, overrides, and the empty-map case — good coverage for the serde layer.

Generated by Rust PR Reviewer for issue #521 · ● 768.5K ·

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Looks good overall — the feature is well-structured and the graceful-degradation path is a solid design choice. Two findings worth addressing before merge.

Findings

🐛 Bugs / Logic Issues

  • src/execute.rs:582-597 — integration tests don't assert is_warning()
    Both the noop and missing-tool execute-level tests were updated to remove the old message-content assertion but only assert result.success. The comment even says "returns a warning (success=true) rather than failing hard", yet neither test calls assert!(result.is_warning()). The per-module unit tests in noop.rs/missing_tool.rs do check this, so the integration tests are weaker than they could be. A no-credentials execution should produce a warning, and success=true alone doesn't distinguish ExecutionResult::success from ExecutionResult::warning.

⚠️ Suggestions

  • src/main.rs:370discover_last_committer runs git log against bare filename only
    path.file_name()? extracts just the filename and current_dir(dir) changes into the parent. This works correctly for any path where the parent directory is the actual working tree location. However, path.parent() on a relative path with no directory component (e.g. Path::new("agent.md")) returns Some(""), and Command::current_dir("") will fail with ENOENT — causing the fallback Err(e) branch to fire and return None. In practice source is always an absolute path from clap, but a defensive let dir = path.parent().filter(|p| !p.as_os_str().is_empty()).unwrap_or(Path::new(".")); would make the intent explicit and remove the latent edge case.

  • src/safeoutputs/mod.rs:525-535 — unbounded ADO error body included in warning messages
    The WIQL failure path does anyhow::bail!("WIQL query failed (HTTP {}): {}", status, error_body) where error_body is the raw HTTP response body. This propagates into the ExecutionResult::warning message and gets printed to stdout (neutralized by neutralize_pipeline_commands, so no injection risk). The concern is simply size — a verbose ADO error page could produce a very noisy pipeline log line. Truncating to, say, the first 512 bytes would keep the diagnostic value while keeping logs readable.

✅ What Looks Good

  • Sanitization chain is correct: WorkItemReportConfig derives SanitizeConfig, get_tool_config calls sanitize_config_fields() before returning, and WIQL single-quote escaping has a clear comment explaining the WIQL escaping convention and why it is sufficient for operator-controlled config.
  • Graceful degradation design: Returning ExecutionResult::warning instead of failing hard when ADO credentials are absent is exactly right for always-on diagnostic tools — other pipeline stages shouldn't be blocked by a missing write token.
  • discover_last_committer error handling: All three failure branches (Ok(non-success), Ok(empty), Err) log at debug and return None cleanly. No panics, no hard failures.
  • multilineFieldsFormat patch op is consistent with the existing create_work_item.rs and update_work_item.rs patterns — not a new pattern introduced by this PR.

Generated by Rust PR Reviewer for issue #521 · ● 1.1M ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants