refactor: ai agent with toolbox and connection

[JerryYang (JerryYangKai)]

Summary

Refactors the Foundry AI extensions so that each extension owns the full lifecycle of its resource:

• azure.ai.connections owns Connection definitions and deployment.
• azure.ai.toolboxes owns Toolbox composition and version deployment.
• azure.ai.agents owns Agent composition and orchestrates its dependencies.

The local YAML definitions are now the source of truth. add commands update local definitions, while deploy commands apply them to an existing Foundry project.

Changes

Connections

• Adds a canonical connection.yaml definition model with strict YAML/JSON loading and atomic writes.
• Adds azd ai connection deploy [path].
• Expands environment-variable references in targets, credentials, metadata, and OAuth fields at deployment time.
• Adds the non-secret connection ARM id to connection show --output json.
• Keeps credentials out of the default show output unless --show-credentials is explicitly requested.
• Reads project context from the key owned by azure.ai.projects, with a legacy fallback.

Toolboxes

• Adds a canonical toolbox.yaml definition model for connections, skills, tools, policies, and metadata.
• Adds local-only composition commands:
  • azd ai toolbox add connection
  • azd ai toolbox add skill
• Adds azd ai toolbox deploy [path], which creates a new immutable Toolbox version.
• Resolves Connection references through the machine-readable azd ai connection show --output json contract.
• Reuses the same request-building path for standalone and azure.yaml deployments.
• Removes the duplicate direct Connection API client from the Toolbox extension.
• Declares azure.ai.connections as a manifest dependency.

Agents

• Adds azd ai agent add toolbox for updating a local agent.yaml.
• Adds standalone azd ai agent deploy [path] for source-code Hosted Agent deployment without requiring azure.yaml.
• Deploys a sibling toolbox.yaml before the Agent when the Toolbox reference is unpinned.
• Skips Toolbox deployment when the Agent references an explicit Toolbox version.
• Passes the resolved Toolbox name, version, and MCP endpoint to the Hosted Agent environment.
• Infers Connection-to-Toolbox edges in the generated deployment graph.
• Removes the duplicate Toolbox client from the Agent extension.
• Declares Connections and Toolboxes as manifest dependencies.

Schemas and compatibility

• Updates the Agent, Connection, and Toolbox schemas for the new definition fields.
• Moves shared project-context ownership to azure.ai.projects.
• Retains a best-effort fallback for the legacy Agent-owned project-context key.
• Preserves existing composed azure.yaml deployment behavior.

Architecture

The resulting dependency graph is:
Project -> Connection -> Toolbox -> Agent

Cross-extension coordination uses machine-readable CLI contracts instead of duplicating sibling resource clients:

• Toolboxes resolve Connections through azd ai connection show --output json.
• Agents deploy unpinned sibling Toolboxes through azd ai toolbox deploy --output json.
• Extension manifests enforce the required dependency versions.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
18 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Refactors Foundry extensions to support source-controlled toolbox and connection definitions and integrate toolbox deployment into hosted-agent workflows.

Changes:

• Adds toolbox and connection definition models, file handling, commands, and cloud deployment.
• Integrates toolbox references and MCP endpoints into agent deployment.
• Migrates shared project context ownership to azure.ai.projects with legacy fallback.

Reviewed changes

Copilot reviewed 61 out of 62 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
cli/azd/extensions/azure.ai.toolboxes/schemas/azure.ai.toolbox.json	Expands toolbox configuration schema.
cli/azd/extensions/azure.ai.toolboxes/internal/foundry/projectctx/types.go	Updates project-context ownership documentation.
cli/azd/extensions/azure.ai.toolboxes/internal/foundry/projectctx/store.go	Adds canonical and legacy context lookup.
cli/azd/extensions/azure.ai.toolboxes/internal/foundry/projectctx/store_test.go	Tests context lookup precedence.
cli/azd/extensions/azure.ai.toolboxes/internal/foundry/projectctx/resolver.go	Documents the canonical config path.
cli/azd/extensions/azure.ai.toolboxes/internal/foundry/connections/types.go	Retains connection category definitions.
cli/azd/extensions/azure.ai.toolboxes/internal/foundry/connections/client.go	Removes the duplicated connection client.
cli/azd/extensions/azure.ai.toolboxes/internal/definition/file.go	Adds strict definition loading and saving.
cli/azd/extensions/azure.ai.toolboxes/internal/definition/definition.go	Defines toolbox configuration types.
cli/azd/extensions/azure.ai.toolboxes/internal/definition/definition_test.go	Tests definition mutation and validation.
cli/azd/extensions/azure.ai.toolboxes/internal/foundry/toolboxes/cloud/types.go	Defines toolbox cloud API models.
cli/azd/extensions/azure.ai.toolboxes/internal/foundry/toolboxes/cloud/client.go	Adds the toolbox data-plane client.
cli/azd/extensions/azure.ai.toolboxes/internal/foundry/toolboxes/cloud/client_test.go	Tests toolbox API requests.
cli/azd/extensions/azure.ai.toolboxes/internal/foundry/toolboxes/cloud/deploy.go	Translates and deploys toolbox definitions.
cli/azd/extensions/azure.ai.toolboxes/internal/foundry/toolboxes/cloud/deploy_test.go	Tests deployment translation.
cli/azd/extensions/azure.ai.toolboxes/internal/output/format.go	Adds structured output handling.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_create.go	Adds toolbox definition creation.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_files.go	Resolves toolbox definition files.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_add_connection.go	Adds connection references to toolboxes.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_add_skill.go	Adds skill references to toolboxes.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_deploy.go	Adds toolbox deployment.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_deploy_test.go	Tests toolbox deployment commands.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_show.go	Updates toolbox display and MCP URLs.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/root.go	Registers the new toolbox commands.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/root_test.go	Tests toolbox command registration.
cli/azd/extensions/azure.ai.agents/schemas/Agent.json	Adds toolbox references to agents.
cli/azd/extensions/azure.ai.agents/internal/project/models/agent_tools.go	Models agent toolbox tools.
cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go	Parses toolbox configuration.
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go	Integrates toolbox dependencies.
cli/azd/extensions/azure.ai.agents/internal/project/tools.go	Translates toolbox references.
cli/azd/extensions/azure.ai.agents/internal/project/tools_test.go	Tests toolbox tool translation.
cli/azd/extensions/azure.ai.agents/internal/project/direct_deploy.go	Adds standalone hosted-agent deployment.
cli/azd/extensions/azure.ai.agents/internal/project/direct_deploy_test.go	Tests direct deployment preparation.
cli/azd/extensions/azure.ai.agents/internal/cmd/deploy.go	Orchestrates agent and toolbox deployment.
cli/azd/extensions/azure.ai.agents/internal/cmd/deploy_test.go	Tests deployment orchestration.
cli/azd/extensions/azure.ai.agents/internal/foundry/projectctx/store.go	Migrates project-context storage.
cli/azd/extensions/azure.ai.agents/internal/foundry/projectctx/store_test.go	Tests context migration behavior.
cli/azd/extensions/azure.ai.agents/internal/version/version.go	Updates the agent extension version.
cli/azd/extensions/azure.ai.agents/go.mod	Updates agent dependencies.
cli/azd/extensions/azure.ai.agents/go.sum	Updates agent dependency checksums.
cli/azd/extensions/azure.ai.agents/extension.yaml	Adds the toolbox extension dependency.
cli/azd/extensions/azure.ai.projects/extension.yaml	Updates extension integration metadata.
cli/azd/extensions/azure.ai.connections/schemas/azure.ai.connection.json	Expands connection configuration fields.
cli/azd/extensions/azure.ai.connections/internal/definition/file.go	Adds strict connection file handling.
cli/azd/extensions/azure.ai.connections/internal/definition/definition.go	Defines connection configuration types.
cli/azd/extensions/azure.ai.connections/internal/definition/definition_test.go	Tests connection definitions.
cli/azd/extensions/azure.ai.connections/internal/cmd/connection_create.go	Adds connection definition creation.
cli/azd/extensions/azure.ai.connections/internal/cmd/connection_create_test.go	Tests connection creation.
cli/azd/extensions/azure.ai.connections/internal/cmd/root.go	Registers connection commands.
cli/azd/extensions/azure.ai.connections/internal/cmd/root_test.go	Tests command registration.
cli/azd/extensions/azure.ai.connections/internal/project/service_target_connection.go	Refactors connection service behavior.
cli/azd/extensions/azure.ai.connections/internal/version/version.go	Updates the connection extension version.
cli/azd/extensions/azure.ai.connections/go.mod	Updates connection dependencies.
cli/azd/extensions/azure.ai.connections/go.sum	Updates connection dependency checksums.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 61 out of 62 changed files in this pull request and generated no new comments.

Suppressed comments (5)

Previously missed (4) — in code that hasn't changed since the last review.

cli/azd/extensions/azure.ai.connections/internal/cmd/connection_deploy.go:117

• OAuth scopes bypass the environment expansion applied to every other OAuth field. A definition such as scopes: ["${GITHUB_SCOPE}"] is sent literally, contradicting the deploy contract and causing OAuth authorization to request the wrong scope. Expand each scope before assigning it to the create flags.

		scopes:           slices.Clone(input.Scopes),

cli/azd/extensions/azure.ai.agents/internal/cmd/deploy.go:178

• An unpinned reference without a sibling toolbox.yaml returns only TOOLBOX_NAME, even though an omitted version means the existing toolbox's default version. The toolbox reference itself is not included in the agent API request, so the workload receives neither a version nor an MCP endpoint and cannot use the referenced toolbox automatically. Resolve the existing default through the toolbox extension's machine-readable show contract, or fail with an actionable error.

	toolboxPath := filepath.Join(filepath.Dir(agentDefinitionPath), "toolbox.yaml")
	if _, err := os.Stat(toolboxPath); err != nil {
		if os.IsNotExist(err) {
			return environment, nil
		}

cli/azd/extensions/azure.ai.agents/internal/project/direct_deploy.go:225

• Prioritize the azd credential in this chain. The command directs users to authenticate with azd auth login, and the existing agent paths use AzureDeveloperCLICredential; if Azure CLI is also logged in as another principal, it can obtain the first token successfully and a later service authorization failure will not fall through to azd. Appending the azd credential first preserves the documented authentication behavior while retaining Azure CLI as fallback.

	credentials := make([]azcore.TokenCredential, 0, 2)
	if azureCLIError == nil {
		credentials = append(credentials, azureCLI)
	}
	if azdError == nil {

cli/azd/extensions/azure.ai.agents/internal/cmd/deploy.go:164

• The environment map omits the established name-scoped endpoint key. Agent initialization writes TOOLBOX_<NAME>_MCP_ENDPOINT into agent.yaml (internal/cmd/init.go:4345-4359), but both branches here only provide TOOLBOX_ENDPOINT; preparation therefore resolves the generated name-scoped placeholder to an empty value. Set envkey.ToolboxMCPEndpoint(name) to the resolved endpoint in both the pinned and deployed branches.

	environment := map[string]string{"TOOLBOX_NAME": name}

cli/azd/extensions/azure.ai.agents/internal/cmd/deploy.go:109

• This deploys the sibling toolbox before any full agent validation, code-directory check, or packaging occurs. For example, an invalid agent.yaml or missing --code directory still creates an immutable toolbox version before the agent deployment fails. Run the local agent/code preflight before this remote mutation to avoid orphan versions and version churn on retries.

	environment, err := deployAgentToolboxDependency(
		ctx, runner, resolved.Endpoint, definitionPath, toolbox,
	)

[Copilot]

Pull request overview

Copilot reviewed 88 out of 89 changed files in this pull request and generated 2 comments.

Suppressed comments (4)

Previously missed (2) — in code that hasn't changed since the last review.

cli/azd/extensions/azure.ai.toolboxes/extension.yaml:13

• This range points to the already-published beta.4, which does not contain the new connection show id contract. The resolver then substitutes the short name as project_connection_id, producing an invalid toolbox request. Require the next Connections release that actually contains this contract, and update the ownership test accordingly.
  cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:130
• Inline azure.yaml cannot populate this field because language is core-owned and stored in ServiceConfig.Language, while agentDefinitionFromStruct receives only AdditionalProperties. Unlike image, no svc.GetLanguage() value is threaded into conversion, so language: python is silently dropped. Pass the core field into the conversion or avoid advertising it for inline services.

	Language                   string                             `json:"language,omitempty"`

cli/azd/extensions/azure.ai.agents/extension.yaml:18

• These ranges reference the already-published Connections beta.4 and Toolboxes beta.5, neither of which contains the new JSON id contract or toolbox deploy command invoked by Agent deployment. The declared dependencies therefore do not enforce the required runtime contracts. Point them at the releases containing this PR and update the manifest test.

  - id: azure.ai.connections
    version: "~1.0.0-beta.4"
  - id: azure.ai.toolboxes
    version: "~1.0.0-beta.5"

cli/azd/extensions/azure.ai.agents/internal/cmd/deploy.go:194

• An unpinned reference without a sibling toolbox.yaml succeeds with only TOOLBOX_NAME; it never resolves the toolbox's default version or MCP endpoint. This contradicts the reference contract and deploys an agent without TOOLBOX_VERSION/TOOLBOX_ENDPOINT. Resolve the existing toolbox through the Toolbox JSON contract, or return an actionable error requiring a version or sibling definition.

	toolboxPath := filepath.Join(filepath.Dir(agentDefinitionPath), "toolbox.yaml")
	if _, err := os.Stat(toolboxPath); err != nil {
		if os.IsNotExist(err) {
			return environment, nil
		}

[Copilot]

Pull request overview

Copilot reviewed 90 out of 91 changed files in this pull request and generated no new comments.

Suppressed comments (4)

Previously missed (1) — in code that hasn't changed since the last review.

cli/azd/extensions/azure.ai.toolboxes/extension.yaml:13

• 1.0.0-beta.4 is already published without the new connection show --output json id field consumed by this PR. Existing beta.4 installations satisfy this dependency and are not guaranteed to upgrade, leaving the resolver with the connection name instead of the ARM ID required for project_connection_id. Depend on the first Connections release containing the new JSON contract and publish it before Toolboxes.

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1373

• This only adds the toolbox to dependency validation; it never adds TOOLBOX_NAME, TOOLBOX_VERSION, or TOOLBOX_ENDPOINT to the environment passed to CreateAgentAPIRequestFromDefinition (that environment is built later from serviceConfig.GetEnvironment() and the legacy list). Consequently, an azure.yaml agent using the new toolbox: field can deploy after its toolbox but the hosted container receives none of the toolbox coordinates. Resolve the reference here (using the pinned version or TOOLBOX_<NAME>_MCP_ENDPOINT) and merge the three generic variables into the request environment, matching standalone deploy.

	addAgentToolboxDependency(serviceTargetConfig, agentDef.Toolbox)

cli/azd/extensions/azure.ai.agents/internal/cmd/deploy.go:194

• An unpinned reference without a sibling toolbox.yaml returns only TOOLBOX_NAME; unlike the sibling and pinned branches, it leaves both version and endpoint unresolved. The deployed agent therefore cannot consume the referenced existing toolbox through the documented TOOLBOX_ENDPOINT contract. Query azd ai toolbox show <name> --output json here to resolve the default version and endpoint (or return an actionable error).

	toolboxPath := filepath.Join(filepath.Dir(agentDefinitionPath), "toolbox.yaml")
	if _, err := os.Stat(toolboxPath); err != nil {
		if os.IsNotExist(err) {
			return environment, nil
		}

cli/azd/extensions/azure.ai.agents/extension.yaml:18

• 1.0.0-beta.5 is already published in cli/azd/extensions/registry.json, before this PR adds the toolbox deploy JSON contract invoked by standalone agent deployment. A machine with that version installed satisfies this range, so the nested command fails as unknown instead of upgrading. Set the minimum to the first toolbox release containing deploy and release Toolboxes before Agents.

  - id: azure.ai.toolboxes
    version: "~1.0.0-beta.5"

[Copilot]

Pull request overview

Copilot reviewed 90 out of 91 changed files in this pull request and generated 2 comments.

Suppressed comments (1)

cli/azd/extensions/azure.ai.connections/internal/cmd/connection_deploy.go:163

• This rejects the canonical composed shape for CustomKeys credentials. The same schema is used by standalone and azure.yaml definitions, and existing synthesis fixtures use credentials.keys.<name>; here the keys object fails the string assertion before deployment. Please preserve/expand the nested credential object (and translate keys to customKeys) so the same connection.yaml works in both deployment paths.

[github-actions]

📋 Prioritization Note

Thanks for the contribution! The linked issue isn't in the current milestone yet.
Thank you for logging this issue; our team is reviewing it. If you need urgent prioritization, tag Rick Winter (@RickWinter) and Kristen Womack (@kristenwomack) to let us know.

[Copilot]

Pull request overview

Copilot reviewed 95 out of 96 changed files in this pull request and generated 4 comments.

Suppressed comments (1)

cli/azd/extensions/azure.ai.connections/internal/cmd/connection_deploy.go:163

• connection.yaml is also used as the canonical $ref shape, where CustomKeys credentials are nested under credentials.keys (for example in the synthesis fixtures). Here keys is a map[string]any, so every such definition is rejected as “must be a string” before deployment. Handle the nested keys object and expand each leaf value before populating customKeys, so the same canonical file works in standalone and composed deployment.

[Copilot]

Pull request overview

Copilot reviewed 95 out of 96 changed files in this pull request and generated 2 comments.

Suppressed comments (3)

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1476

This adds the Toolbox only to dependency validation. The composed azd deploy path still builds the Hosted Agent environment in prepareDeploy without reading agentDef.Toolbox, so unlike standalone deploy it never injects TOOLBOX_NAME, TOOLBOX_VERSION, or TOOLBOX_ENDPOINT. Agents using the new reference through azure.yaml therefore start without the advertised Toolbox environment.

	addAgentToolboxDependency(serviceTargetConfig, agentDef.Toolbox)

cli/azd/extensions/azure.ai.agents/internal/cmd/agent_definition_edit.go:72

Reject kind: prompt-voice before mutating the file. The schema and ValidateAgentDefinition explicitly prohibit a hosted-agent toolbox reference for prompt-voice agents, but this command currently turns a valid voice definition into one that cannot be deployed.

	root, err := agentDefinitionMapping(&document)
	if err != nil {
		return err
	}

cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_connection_resolver.go:104

Do not substitute the connection name when the JSON contract omits id. project_connection_id is populated from this value and requires the ARM resource ID, so this fallback converts an incomplete sibling-extension response into a malformed Toolbox request. Treat a blank ID as an incomplete descriptor instead.

[azure-sdk-automation]

azure.ai.connections PR build

Note

This is an unsigned development build. Install it only if you trust this PR.

Install the extension:

azd ext install "https://azuresdkartifacts.z5.web.core.windows.net/azd/extensions/pr/9735/azure-ai-connections.zip"

• Version: 1.0.0-beta.5.pr.9735.6757237
• Merge commit: d099b87

[azure-sdk-automation]

azure.ai.projects PR build

Note

This is an unsigned development build. Install it only if you trust this PR.

Install the extension:

azd ext install "https://azuresdkartifacts.z5.web.core.windows.net/azd/extensions/pr/9735/azure-ai-projects.zip"

• Version: 1.0.0-beta.8.pr.9735.6757238
• Merge commit: d099b87

[azure-sdk-automation]

azure.ai.toolboxes PR build

Note

This is an unsigned development build. Install it only if you trust this PR.

Install the extension:

azd ext install "https://azuresdkartifacts.z5.web.core.windows.net/azd/extensions/pr/9735/azure-ai-toolboxes.zip"

• Version: 1.0.0-beta.5.pr.9735.6757239
• Merge commit: d099b87