Skip to content

execution: cloud still emits setup-azd and azd extension install steps #375

Description

Problem

When agentops.yaml sets execution: cloud, the generated CI workflow still contains:

      - name: Set up azd
        uses: Azure/setup-azd@v2

      - name: Install pinned azd AI agents extension
        run: azd extension install azure.ai.agents --version "1.0.0-beta.9"

But in cloud mode agentops eval run never shells out to azd. Foundry runs the agent and the evaluators server-side, and AgentOps just creates the eval, polls the run, and downloads the output items:

cloud: using Foundry dataset agentops-helpdesk-smoke@sha256-a22e19374d43b28e
cloud: creating eval (4 criteria, item_schema fields: ['expected', 'id', 'input'])
cloud: starting run for agent helpdeskbot:11
cloud: run status -> completed (elapsed 1m47s, attempt 52/300)
cloud: downloaded 3 output item(s)

So both steps are pure overhead: extra CI minutes, an extra network dependency on the azd extension registry, and an extra failure point that has nothing to do with the evaluation itself. This is exactly how #373 bit every hosted-agent pipeline — the gate died before the eval ever started, in a step it did not need.

Repro

  1. agentops init against a Foundry hosted agent.
  2. Add execution: cloud to agentops.yaml.
  3. agentops workflow generate --kinds pr,dev --force.
  4. Inspect .github/workflows/agentops-pr.yml — the setup-azd and azd extension install steps are still emitted.

--deploy-mode auto does not avoid this: auto resolves to azd for Foundry targets regardless of execution.

Expected

When execution: cloud, the generator should omit Azure/setup-azd@v2 and azd extension install azure.ai.agents. Same applies to the Azure DevOps template (services/cicd.py:639).

Notes

execution: azd and execution: auto (resolving to azd) obviously still need both steps. This is only about the cloud branch.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions