[agent-issue]: support Azure DevOps variable group imports in workflow source

[krubenok]

Submission requirements

• I generated this issue with an agent that used .github/agents/ado-aw.agent.md.
• I reviewed the generated issue and confirm it is being filed directly in githubnext/ado-aw.

Problem summary

ado-aw should support source-level Azure DevOps variable group imports so generated pipeline locks can consume project-managed secrets without hand-editing the generated YAML.

Today engine.github-app-token.private-key can reference an ADO pipeline variable name, for example:

engine:
  id: copilot
  github-app-token:
    app-id: 1234567
    owner: example-org
    repositories: [example-repo]
    private-key: AGENTIC_WORKFLOWS_GITHUB_APP_PRIVATE_KEY

That is useful when the GitHub App private key is managed centrally in an ADO Library variable group. However, in Azure DevOps, a variable group being present and authorized for a pipeline definition is not enough to make its variables available to a YAML run. The YAML must explicitly import the group:

variables:
  - group: Agentic Workflows

ado-aw front matter currently has no source-level way to request that import, so a source-clean generated lock cannot consume secrets managed in a project-level variable group. This is especially painful for GitHub App private keys and other shared agentic-workflow secrets that should be managed once at the project level, not copied onto every pipeline definition.

Reproduction details

1. Create an ADO Library variable group, for example Agentic Workflows, containing a secret variable such as AGENTIC_WORKFLOWS_GITHUB_APP_PRIVATE_KEY.

2. Author an ado-aw workflow that references that variable from engine.github-app-token.private-key:

engine:
  id: copilot
  github-app-token:
    app-id: 1234567
    owner: example-org
    repositories: [example-repo]
    private-key: AGENTIC_WORKFLOWS_GITHUB_APP_PRIVATE_KEY

3. Compile with ado-aw.

Observed behavior: the generated lock references $(AGENTIC_WORKFLOWS_GITHUB_APP_PRIVATE_KEY) in the GitHub App token mint step, but does not import the variable group. At runtime, the variable is unavailable unless it was also copied directly onto the pipeline definition or the generated lock was hand-patched.

Expected behavior: ado-aw source should support a variable group import, for example:

variables:
  - group: Agentic Workflows

and generated YAML should preserve that import while maintaining lock integrity:

variables:
  - group: Agentic Workflows

Pipeline-definition authorization is still required, but it is not sufficient by itself. Authorization grants the pipeline permission to use the group; the YAML import actually brings the group variables into the run.

Current workarounds are all undesirable:

1. Store the secret directly as a pipeline variable on each generated pipeline definition, duplicating project-level secret management.
2. Hand-patch the generated lock with variables: - group: ... and disable or bypass generated-lock integrity.
3. Build a parent/wrapper YAML outside ado-aw that imports the group and includes an ado-aw-generated job/stage template, adding complexity and target-specific wrapper behavior.

Proposed next step

Add source-level support for ADO variable group imports. A simple front matter shape could mirror ADO YAML:

variables:
  - group: Agentic Workflows

If variables: is too broad or conflicts with future literal variable support, an ado-aw-specific shape could be used instead, such as:

variable-groups:
  - Agentic Workflows

Security considerations:

• The source should contain only variable group names, never secret values.
• ado-aw should not resolve, print, log, or serialize variable values.
• Validation should treat group names as identifiers/references and reject YAML forms that inline secret values if this feature is intended only for groups.
• Existing secret handling remains unchanged: steps still reference secret variables by macro name, such as $(AGENTIC_WORKFLOWS_GITHUB_APP_PRIVATE_KEY).

Target/wrapper considerations:

• target: standalone: emit top-level variables: with - group: ....
• target: 1es: ensure variable group imports land in the correct top-level/template-supported location for the 1ES wrapper and required template schema.
• target: job / target: stage: decide whether imports are representable in the emitted template target. If not, fail with a clear compile-time error or document that variable group imports require the parent pipeline to import the group.
• Required-template / extends wrappers should preserve the variable group import in a way Azure DevOps evaluates before jobs that reference the macros.

Acceptance criteria / tests:

• Front matter can declare one or more variable group imports.
• Compiled standalone YAML includes top-level `variables:
  • group: `.
• A workflow using engine.github-app-token.private-key: <VAR_NAME> can import a variable group containing <VAR_NAME> without hand-patching the lock.
• Generated locks remain deterministic and integrity-preserving.
• Tests cover at least standalone output shape and the behavior/error for 1es, job, and stage targets.
• Tests verify secret values are never required or emitted, only group names.
• Docs explain that variable group authorization and YAML import are both required in Azure DevOps.

[krubenok]

Additional incompatibility tied to the same variable-group support work: Azure Key Vault-backed ADO Library variable groups naturally expose secret names with hyphens, and ado-aw 0.42 currently rejects those names for engine.github-app-token.private-key.

For example, Key Vault secrets commonly look like:

AGENTIC-WORKFLOWS-GITHUB-APP-PRIVATE-KEY
GH-AW-APP-ID

Azure DevOps YAML/macros can reference those variable names directly:

env:
  GH_APP_PRIVATE_KEY: $(AGENTIC-WORKFLOWS-GITHUB-APP-PRIVATE-KEY)

But this ado-aw source is rejected because private-key is validated as an env-var-style name ([A-Za-z_][A-Za-z0-9_]*):

engine:
  github-app-token:
    private-key: AGENTIC-WORKFLOWS-GITHUB-APP-PRIVATE-KEY

That forces a generated lock patch today:

- GH_APP_PRIVATE_KEY: $(AGENTIC_WORKFLOWS_GITHUB_APP_PRIVATE_KEY)
+ GH_APP_PRIVATE_KEY: $(AGENTIC-WORKFLOWS-GITHUB-APP-PRIVATE-KEY)

Request: support the variable-name syntax Azure DevOps accepts for Key Vault-backed variable groups, while still emitting a safe environment variable key for the step (GH_APP_PRIVATE_KEY). In other words, the source value should be treated as the ADO macro target name, not as the emitted process environment key.

Suggested acceptance criteria:

• engine.github-app-token.private-key accepts ADO variable names with hyphens when used as a macro target.
• Generated YAML uses GH_APP_PRIVATE_KEY: $(<original-name>).
• Validation still rejects pipeline command injection, template expressions, malformed macros, and other unsafe macro targets.
• Docs mention that Key Vault secret names commonly use hyphens and can surface that way in variable groups.
• Tests cover private-key: AGENTIC-WORKFLOWS-GITHUB-APP-PRIVATE-KEY.

This is related to the source-clean variable group import feature because variable groups need to work with the variable names Key Vault actually exposes, not only with underscore-style pipeline variables.

[jamesadevine]

both in 0.43.0