Problem
When agentops.yaml sets execution: cloud, the generated CI workflow still contains:
- name: Set up azd
uses: Azure/setup-azd@v2
- name: Install pinned azd AI agents extension
run: azd extension install azure.ai.agents --version "1.0.0-beta.9"
But in cloud mode agentops eval run never shells out to azd. Foundry runs the agent and the evaluators server-side, and AgentOps just creates the eval, polls the run, and downloads the output items:
cloud: using Foundry dataset agentops-helpdesk-smoke@sha256-a22e19374d43b28e
cloud: creating eval (4 criteria, item_schema fields: ['expected', 'id', 'input'])
cloud: starting run for agent helpdeskbot:11
cloud: run status -> completed (elapsed 1m47s, attempt 52/300)
cloud: downloaded 3 output item(s)
So both steps are pure overhead: extra CI minutes, an extra network dependency on the azd extension registry, and an extra failure point that has nothing to do with the evaluation itself. This is exactly how #373 bit every hosted-agent pipeline — the gate died before the eval ever started, in a step it did not need.
Repro
agentops init against a Foundry hosted agent.
- Add
execution: cloud to agentops.yaml.
agentops workflow generate --kinds pr,dev --force.
- Inspect
.github/workflows/agentops-pr.yml — the setup-azd and azd extension install steps are still emitted.
--deploy-mode auto does not avoid this: auto resolves to azd for Foundry targets regardless of execution.
Expected
When execution: cloud, the generator should omit Azure/setup-azd@v2 and azd extension install azure.ai.agents. Same applies to the Azure DevOps template (services/cicd.py:639).
Notes
execution: azd and execution: auto (resolving to azd) obviously still need both steps. This is only about the cloud branch.
Problem
When
agentops.yamlsetsexecution: cloud, the generated CI workflow still contains:But in cloud mode
agentops eval runnever shells out toazd. Foundry runs the agent and the evaluators server-side, and AgentOps just creates the eval, polls the run, and downloads the output items:So both steps are pure overhead: extra CI minutes, an extra network dependency on the azd extension registry, and an extra failure point that has nothing to do with the evaluation itself. This is exactly how #373 bit every hosted-agent pipeline — the gate died before the eval ever started, in a step it did not need.
Repro
agentops initagainst a Foundry hosted agent.execution: cloudtoagentops.yaml.agentops workflow generate --kinds pr,dev --force..github/workflows/agentops-pr.yml— thesetup-azdandazd extension installsteps are still emitted.--deploy-mode autodoes not avoid this:autoresolves to azd for Foundry targets regardless ofexecution.Expected
When
execution: cloud, the generator should omitAzure/setup-azd@v2andazd extension install azure.ai.agents. Same applies to the Azure DevOps template (services/cicd.py:639).Notes
execution: azdandexecution: auto(resolving to azd) obviously still need both steps. This is only about thecloudbranch.