Training extension: support Foundry storage connections and SDK-compatible assets

[yeshsurya]

Problem

The azure.ai.training extension does not have full parity with the Microsoft Foundry Python SDK for command-job storage and asset references.

Local code and input folders are uploaded through the project's default storage. In projects where that default datastore is unavailable or unresolved, command jobs can fail during input preparation before compute starts. The Python SDK avoids this by allowing callers to select a Foundry storage connection for folder uploads.

The extension also needs to preserve SDK-compatible output asset metadata and registered azureai:// dataset references.

A related edge case occurs with content-based upload deduplication: if identical content was previously uploaded through default storage, a later submission requesting a BYOS connection can reuse the earlier dataset version and never request an upload location from the selected connection.

Proposed behavior

• Accept storage_connection_name in command-job YAML.
• Add --storage-connection-name to azd ai training job submit, with the CLI value taking precedence.
• Forward connectionName to the Foundry startPendingUpload request.
• Include the effective storage connection in local-upload deduplication so datasets are reused only within the same connection.
• Preserve azureai:// code and input references without treating them as local paths.
• Preserve output asset_name and optional asset_version in the submitted job payload.
• Add unit tests and user documentation for these behaviors.

Expected result

Users can submit the same command jobs as the Python SDK, including BYOS-backed local uploads, registered Foundry dataset references, and named output assets, without falling back to an unavailable default datastore.