Skip to content

[agent-issue]: Agents cannot supply work-item fields; custom-fields is compile-time only, so runtime-computed values never reach ADO #2044

Description

@Krasios

Submission requirements

  • I generated this issue with an agent that used .github/agents/ado-aw.agent.md.
  • I reviewed the generated issue and confirm it is being filed directly in githubnext/ado-aw.

Problem summary

There is no route by which a value computed at runtime can reach any work-item field. CreateWorkItemParams is #[serde(deny_unknown_fields)] over {title, description, tags}, and UpdateWorkItemParams exposes only a fixed set (title, body, state, area_path, iteration_path, assignee, tags). The only mechanism that writes arbitrary fields is custom-fields on CreateWorkItemConfig — a compile-time constant map in front matter.

#1924 already named this limitation in passing: "custom-fields does not help: it is a compile-time constant map." That issue was resolved for the specific case of the description field, but the general gap remains.

Concrete impact. Our daily SFF failure-analysis agent runs a skill that computes the release version at analysis time and emits it on each create proposal (Microsoft.VSTS.Build.FoundIn, One_custom.Version). Under dry_run the skill does not write to ADO — it returns proposals the agent replays as create-work-item safe outputs — so those fields are dropped in transit. Result: 67 of 190 auto-filed bugs had a blank Version and were therefore missing from every triage view that filters on it. We backfilled 33 by hand.

The workaround is to hardcode the value in front matter:

safe-outputs:
  create-work-item:
    custom-fields:
      Microsoft.VSTS.Build.FoundIn: "sff2610"
      One_custom.Version: "SFF2610"

That works, but it makes a compile-time constant stand in for a runtime value: the version must now be duplicated in two places and bumped by hand every release, and nothing detects it going stale. It also cannot express anything genuinely per-item — severity, repro rate, or a found-in-build that differs per failure bucket all have a single value per pipeline or none at all.

The failure mode is silent, which is the worst part. deny_unknown_fields means the extra data is discarded with no error surfaced to the agent; the safe-output executor reports success and the pipeline goes green. This is the same shape as #1924"created successfully, the executor reports success, and the only way to notice is for a human to open the bug." Nobody noticed for weeks.

Worth noting the same silent drop applies to per-item area_path on create-work-item, and that one cost us more. Every auto-filed bug landed in the project root area, our dedup query scoped candidates by area path and so could never see a bug it had previously filed, and the agent re-filed the same failures every day until someone deduplicated 8 bugs by hand after a single run.

Reproduction details

Environment: ado-aw 0.50.0 (Windows), Azure DevOps Server, Bug work item type.

  1. Configure a create-work-item safe output in agent front matter.

  2. Ask the safe-outputs MCP server for its tool definitions:

    ado-aw mcp <out> <bound> --enabled-tools create-work-item
    {"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}
    

    Observed — the agent-facing schema exposes only:

    create-work-item -> title, description, tags   (additionalProperties: false)
    

    Repeating with --enabled-tools update-work-item:

    update-work-item -> id, title, body, state, tags,
                        assignee, area_path, iteration_path
    
  3. There is consequently no field an agent can populate to carry a computed value, on either tool — the attempt is rejected at the tool boundary rather than failing later.

  4. Setting the value via front-matter custom-fields does reach the work item (confirmed by decoding the compiled lock's toolConfigs blob), but only as a constant fixed at compile time.

Expected: an agent can attach arbitrary work-item fields to the item it is creating or updating, so that values determined during the run reach ADO.

Actual: only title, description and tags survive; everything else must be known when the workflow is compiled.

Proposed next step

Add an optional field map to the agent-facing payloads, merged over the operator's custom-fields (agent value winning, or config winning — either is fine, as long as it is documented):

/// Additional work item fields, keyed by full reference name
/// (e.g. "Microsoft.VSTS.Build.FoundIn").
#[serde(default)]
pub fields: std::collections::HashMap<AdoWorkItemFieldRef, String>,

on both CreateWorkItemParams and UpdateWorkItemParams. AdoWorkItemFieldRef already exists and is already used for custom-fields keys, so the validation and sanitization path is in place.

Our preference is for this to be unrestricted, mirroring how custom-fields behaves today — the operator has already opted into create-work-item, and the agent already controls the title, description and tags of the very same work item, so field writes on an item it is authorized to create seem within the existing boundary.

That said, we recognize this widens what a prompt-injected agent can reach, and #369 shows that is territory you have already audited. If you conclude it needs gating, we would be unblocked by whatever mechanism you consider appropriate — the requirement is only that a value computed during the run can reach a field, not that it be ungoverned.

Related: #1924 (same silent-success failure mode, and the origin of the "compile-time constant map" observation) and #1899 (agent-supplied assigned_to, resolved by giving the agent a way to express per-item intent it previously could not).

Two smaller notes while filing:

  • The drop being silent is arguably a defect independent of this request. If deny_unknown_fields is intentional, having the executor surface a warning when a proposal carries data it discards would have saved us the entire investigation.
  • description-field from [agent-issue]: create-work-item hardcodes System.Description, so Bug work items are filed blank #1924 is merged on main but not in v0.50.0, the current latest release (closed 8/20, released 8/14). Not a request, just confirming our read — we are waiting on 0.51.0 for it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestrustPull requests that update rust code

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions