[Cloud] Perform cloud discovery with API Version 2022-09-01 and support `dataplaneEndpoints` for Azure Stack #25834

jiasli · 2023-03-16T04:04:46Z

Related command
az cloud

Description
Close #25815

Testing Guide

Populate dataplaneEndpoints from metadata:

az cloud register -n mycloud --endpoint-resource-manager https://management.azure.com/

Populate dataplaneEndpoints from local cloud config JSON file:

az cloud register -n mycloud --cloud-config @D:\cloud\cloud-config.json

azure-client-tools-bot-prd · 2023-03-16T04:04:48Z

️✔️acr

️✔️acs

️✔️advisor

️✔️ams

️✔️apim

️✔️appconfig

️✔️appservice

️✔️aro

️✔️backup

️✔️batch

️✔️batchai

️✔️billing

️✔️botservice

️✔️cdn

❌cloud

️✔️cognitiveservices

️✔️config

️✔️configure

️✔️consumption

️✔️container

️✔️core

️✔️cosmosdb

️✔️databoxedge

️✔️dla

️✔️dls

️✔️dms

️✔️eventgrid

️✔️eventhubs

️✔️feedback

️✔️find

️✔️hdinsight

️✔️identity

️✔️iot

️✔️keyvault

️✔️kusto

️✔️lab

️✔️managedservices

️✔️maps

️✔️marketplaceordering

️✔️monitor

️✔️netappfiles

️✔️network

️✔️policyinsights

️✔️privatedns

️✔️profile

️✔️rdbms

️✔️redis

️✔️relay

️✔️resource

️✔️role

️✔️search

️✔️security

️✔️servicebus

️✔️serviceconnector

️✔️servicefabric

️✔️signalr

️✔️sql

️✔️sqlvm

️✔️storage

️✔️synapse

️✔️telemetry

️✔️util

️✔️vm

yonzhan · 2023-03-16T04:17:24Z

Cloud enhancement

TheOnlyWei · 2023-03-30T05:25:55Z

Although the main post mentioned that a new nested group called dataplaneEndpoints is being supported, there should also be support for new suffixes and endpoints that is not passed by a cloudconfig.json file during az cloud register. Currently, Azure CLI is using a deprecated method to populate the hard-coded metadata which should be updated to read all endpoints and suffixes:

azure-cli/src/azure-cli/azure/cli/command_modules/cloud/custom.py

Line 69 in 091c6b1

_populate_from_metadata_endpoint(c, arm_endpoint)

The method is from this deprecated library:
https://github.com/Azure/msrestazure-for-python/blob/8e3def8c9927a349596e575de93df76de7a3a2b6/msrestazure/azure_cloud.py#L204

jiasli · 2023-03-30T06:15:10Z

If these new suffixes and endpoints are not passed by a cloudconfig.json file, how will Azure CLI even know them?

We can certainly update _populate_from_metadata_endpoint and make no longer deprecated. That's why this PR copies this function from msrestazure to Azure CLI.

Reading all endpoints and suffixes is simply against our modeling principal which enforces all endpoints to be an attribute of the cloud object. Of course, we can leverage setattr to programmatically change the attributes of the cloud object, this defeats the purpose of modeling.

TheOnlyWei · 2023-03-30T09:09:57Z

@jiasli

The new endpoints and suffixes will appear in the GET request to the metadata endpoint URL.

Is there a reason there is an exception made for reading all endpoints and suffixes from the cloudconfig.json but not from the metadata endpoint?
For example, the following code just adds all endpoints from cloudconfig.json:

azure-cli/src/azure-cli/azure/cli/command_modules/cloud/custom.py

Line 46 in 091c6b1

endpoints = cloud_args['endpoints']

jiasli · 2023-03-30T10:29:38Z

I didn't write that code, but I guess it is because cloudconfig.json is designed/used specifically for Azure CLI. On the other hand, metadata endpoint is merely a supplemental way to extract information we want.

toefraz · 2023-07-12T19:39:29Z

@jiasli Bumping this.

FYI 2022-09-01 is a breaking change for the metadata endpoint. Prior versions returned a JSON array of cloud endpoints for multiple environments. This version returns a single JSON object for the endpoints associated with the ARM instance being queried.

Is the goal to support both schemas with this for backwards compatibility?

jiasli · 2024-05-10T02:58:40Z

src/azure-cli/azure/cli/command_modules/cloud/custom.py

+
+
+# The exact API version doesn't matter too much right now. It just has to be YYYY-MM-DD format.
+METADATA_ENDPOINT_SUFFIX = '/metadata/endpoints?api-version=2022-09-01'


msrestazure current uses 2015-01-01 API version: https://management.azure.com/metadata/endpoints?api-version=2015-01-01

> az cloud register -n myazurestack --endpoint-resource-manager https://management.azure.com/ --debug ... urllib3.connectionpool: https://management.azure.com:443 "GET //metadata/endpoints?api-version=2015-01-01 HTTP/1.1" 200 295

cloud-dataplane-endpoints

80aa3ef

yonzhan assigned jiasli Mar 16, 2023

yonzhan added this to the April 2023 (2023-04-25) milestone Mar 16, 2023

jiasli modified the milestones: April 2023 (2023-04-25), Backlog Mar 30, 2023

jiasli commented May 10, 2024

View reviewed changes

This was referenced Jun 26, 2024

Error Encountered with 'az cloud register' when Adding ARM Endpoint and API Version #28716

Open

Add Azure Service Fabrik Cluster Endpoint to az cloud show command #29283

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Cloud] Perform cloud discovery with API Version 2022-09-01 and support `dataplaneEndpoints` for Azure Stack #25834

[Cloud] Perform cloud discovery with API Version 2022-09-01 and support `dataplaneEndpoints` for Azure Stack #25834

jiasli commented Mar 16, 2023

azure-client-tools-bot-prd bot commented Mar 16, 2023 •

edited

Loading

yonzhan commented Mar 16, 2023

TheOnlyWei commented Mar 30, 2023 •

edited

Loading

jiasli commented Mar 30, 2023

TheOnlyWei commented Mar 30, 2023 •

edited

Loading

jiasli commented Mar 30, 2023

toefraz commented Jul 12, 2023

jiasli May 10, 2024



		# The exact API version doesn't matter too much right now. It just has to be YYYY-MM-DD format.
		METADATA_ENDPOINT_SUFFIX = '/metadata/endpoints?api-version=2022-09-01'

[Cloud] Perform cloud discovery with API Version 2022-09-01 and support dataplaneEndpoints for Azure Stack #25834

Are you sure you want to change the base?

[Cloud] Perform cloud discovery with API Version 2022-09-01 and support dataplaneEndpoints for Azure Stack #25834

Conversation

jiasli commented Mar 16, 2023

azure-client-tools-bot-prd bot commented Mar 16, 2023 • edited Loading

yonzhan commented Mar 16, 2023

TheOnlyWei commented Mar 30, 2023 • edited Loading

jiasli commented Mar 30, 2023

TheOnlyWei commented Mar 30, 2023 • edited Loading

jiasli commented Mar 30, 2023

toefraz commented Jul 12, 2023

jiasli May 10, 2024

Choose a reason for hiding this comment

[Cloud] Perform cloud discovery with API Version 2022-09-01 and support `dataplaneEndpoints` for Azure Stack #25834

[Cloud] Perform cloud discovery with API Version 2022-09-01 and support `dataplaneEndpoints` for Azure Stack #25834

azure-client-tools-bot-prd bot commented Mar 16, 2023 •

edited

Loading

TheOnlyWei commented Mar 30, 2023 •

edited

Loading

TheOnlyWei commented Mar 30, 2023 •

edited

Loading