Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

az curl or az rest for custom REST support (coverage) #7618

Closed
mxplusb opened this issue Oct 19, 2018 · 25 comments
Closed

az curl or az rest for custom REST support (coverage) #7618

mxplusb opened this issue Oct 19, 2018 · 25 comments
Assignees
Labels
Milestone

Comments

@mxplusb
Copy link

@mxplusb mxplusb commented Oct 19, 2018

Is your feature request related to a problem? Please describe.

The Azure docs have a REST API explorer, and that is helpful for one-off testing, but isn't very helpful for scripting or for REST APIs that haven't made their way into the Azure CLI.

Describe the solution you'd like

I'd like to propose adding a new subcommand, such as az curl or az rest, that allows a user to call a resource provider with a custom payload. I could see the feature looking something like this:

az {curl,rest} /subscriptions/{subscriptionId}/providers/{Resource.Provider}/{endpoint} \
  --api-version {version} \
  --payload '{"json": "payload"}'

or

az {curl,rest} /{Resource.Provider}/{endpoint} \
  --subscription {sub-id} \
  --api-version {version} \
  --payload '{"json": "payload"}'

or

az {curl,rest} {endpoint} \
  --subscription {sub-id} \
  --resource-provider {Resource.Provder} \
  --api-version {version} \
  --payload '{"json": "payload"}'

This would allow users to pass JSON payloads for scripting or for testing and validating of the REST API for custom clients.

Describe alternatives you've considered

Currently you can curl the Azure REST API, but you have to pass a lot of authentication context that will persist in your shell history, which can be a security concern. This would alleviate that and allow a user to interact with the raw REST API while letting the Azure CLI handle authentication contexts on behalf of the user.

Additional context

You can see a good example of this with the Cloud Foundry CLI, cf. You can cf curl to interact with raw API resources the CLI doesn't support out of the box.

@ppanyukov
Copy link

@ppanyukov ppanyukov commented Nov 22, 2018

I would like this for setting IP restrictions for webapps. From the docs here: https://docs.microsoft.com/en-us/azure/app-service/app-service-ip-restrictions :

There currently is no CLI or PowerShell for the new IP Restrictions capability but the values can be set manually with a PUT operation on the app configuration in Resource Manager.

I'm currently scratching my head how exactly to achieve this considering authentication etc.

Loading

@tjprescott
Copy link
Member

@tjprescott tjprescott commented Nov 26, 2018

@mxplusb have you tried that az resource create/update commands? That is essentially what those do.

@ppanyukov if there isn't already, please open a feature request for webapps to support these properties. The article is likely referring to use of --set on generic update. Opening a specific issue will put it on their radar to add this, and may get you a workaround using --set in the meantime.

Loading

@nickwalkmsft
Copy link
Contributor

@nickwalkmsft nickwalkmsft commented Feb 14, 2019

I'm struggling to achieve some of the REST calls made in https://docs.microsoft.com/en-gb/azure/storage/common/storage-metrics-in-azure-monitor using an az resource command; as far as I can tell they can't be achieved. This feature would be great in the CLI, as armclient isn't available in the Cloud Shell.

Loading

@yugangw-msft
Copy link
Contributor

@yugangw-msft yugangw-msft commented May 7, 2019

I recommend we prioritize this work as it becomes helpful not just on ARM but also any rest calls on data plane APIs like AAD Graph. Having such support will not only handle the thorny part like propagating access tokens, but also getting CLI built-in supports for free such as different output mode, jmespath query, etc.

Regarding the design, I suggest add an argument of HTTP Method like get/put/post/delete

With that, CLI can be used for more scripting scenarios even with temporary service coverage gaps.

Loading

@tjprescott tjprescott added this to Triage in Key Vault Features via automation May 7, 2019
@tjprescott tjprescott moved this from Triage to High Priority in Key Vault Features May 7, 2019
@yugangw-msft yugangw-msft self-assigned this May 25, 2019
@yugangw-msft
Copy link
Contributor

@yugangw-msft yugangw-msft commented May 25, 2019

i will take this

Loading

@yugangw-msft
Copy link
Contributor

@yugangw-msft yugangw-msft commented May 28, 2019

folks, check out my #9490 and let me know if you have any comments. The command signature looks like:

(env) C:\sdk\azure-cli>az rest -h

Command
    az rest : Invoke custom request.

Arguments
    --method         [Required] : HTTP request method.  Allowed values: delete, get, head, options,
                                  patch, post, put.  Default: get.
    --url            [Required] : Request url.
    --body                      : Request body.
    --headers                   : Space-separated appsettings in KEY=VALUE format or JSON string.
                                  Use @{file} to load from a file.
    --output-file               : Save response payload to a file.
    --resource                  : Resource which CLI would acquire token so to put on the
                                  "Authorization" header. By default, CLI can figure this out based
                                  on --url, unless you use ones not in the list of "az cloud show".
    --skip-authorization-header : Do not auto append "Authorization" header.
    --uri-parameters            : Space-separated appsettings in KEY=VALUE format or JSON string.
                                  Use @{file} to load from a file.
Examples
    Get Audit log through Microsoft Graph
        az rest --method get --url https://graph.microsoft.com/beta/auditLogs/directoryAudits


    Update a Azure Active Directory Graph User's display name
        az rest --method patch --headers "{"Content-Type": "application/json"}" --url
        "https://graph.microsoft.com/v1.0/users/johndoe@azuresdkteam.onmicrosoft.com" --body
        "{"displayName":"jondoe2"}"

Loading

@mxplusb
Copy link
Author

@mxplusb mxplusb commented May 28, 2019

@yugangw-msft that looks fantastic, it definitely solves my original ask, thanks for following through!

Loading

@achandmsft achandmsft added this to To do in Compute Features via automation May 30, 2019
@achandmsft achandmsft changed the title az curl or az rest for custom REST support az curl or az rest for custom REST support (coverage) May 30, 2019
@achandmsft achandmsft moved this from To do to In progress in Compute Features May 30, 2019
Compute Features automation moved this from In progress to Done Jun 8, 2019
@yugangw-msft
Copy link
Contributor

@yugangw-msft yugangw-msft commented Jun 8, 2019

I have merged it to release on June 18. //CC @achandmsft
@nickwalkmsft, the command for your scenario would be:

az rest --method get --url https://management.azure.com/subscriptions/0b1f6471-1bf0-4dda-aec3-cb9272f09590/resourceGroups/yugangw/providers/Microsoft.Storage/storageAccounts/yugangwstor22/providers/microsoft.insights/metricdefinitions?api-version=2018-01-01

Loading

@bigdatamoore
Copy link

@bigdatamoore bigdatamoore commented Jun 21, 2019

Hi @yugangw-msft can you provide an example of how to use the resource argument? I am not clear on what I am supposed to pass to that.

Loading

@yugangw-msft
Copy link
Contributor

@yugangw-msft yugangw-msft commented Jun 22, 2019

Sure, most of times, you don't need to supply values to --resource as Azure CLI is able to figure out from --uri. e.g.

az rest --method get --uri https://graph.microsoft.com/beta/auditLogs/directoryAudits

CLI will default the resource to https://graph.microsoft.com/, which is the right one, hence you have nothing to worry about.

--resource by itself is a bit confusing, but you can understand it as a service endpoint, and to send a request to it, CLI works underneath to get the appropriate token issued for accessing the service endpoint. When CLI tries to acquire the token, Azure AD requires the resource argument, you can always supply it for CLI to pass over to it, or leave to CLI to figure out.
JFYI, --resource is also called relying party, web api, audience uri, etc based on the context.

Hope this helps.

Loading

@Slowdive-Aideron
Copy link

@Slowdive-Aideron Slowdive-Aideron commented Jan 24, 2020

How do i specify the Authorization : bearer $token in the --headers ? i haven't found anything useful . Also is it possible to user az rest -m get on a private keyvault for example ?

Loading

@brandonh-msft
Copy link
Member

@brandonh-msft brandonh-msft commented Jan 28, 2020

@yugangw-msft how do I use this? az rest throws an error

Loading

@minascasiou
Copy link

@minascasiou minascasiou commented Apr 25, 2020

Nice! Thank you.
Does az rest support --query?

I can't seem to get any results when I try it

Loading

@minascasiou
Copy link

@minascasiou minascasiou commented Apr 25, 2020

Nice! Thank you.
Does az rest support --query?

I can't seem to get any results when I try it

This is what I'm trying...

az rest -m get -u "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/HaNeCan/providers/Microsoft.Web/sites/HaNeCan/functions?api-version=2019-08-01" --query "[].{id: [].invoke_url_template}"

Loading

@bbkane
Copy link

@bbkane bbkane commented May 20, 2021

@Slowdive-Aideron , I was able to run the following successfully:

readonly vaultBaseUrl="https://concert-kv-dev-bbk-01.vault.azure.net/"
readonly issuerName="digicert-bbkane-com-03"
readonly resource="https://vault.azure.net"

az rest \
    --body "@body.json" \
    --method put \
    --resource "$resource" \
    --uri "${vaultBaseUrl}/certificates/issuers/${issuerName}" \
    --uri-parameters 'api-version=7.2'

The key is to use --resource "https://vault.azure.net"

Loading

@jsntcy jsntcy removed this from High Priority in Key Vault Features Jul 27, 2021
@BhargaviAnnadevara-MSFT

@yugangw-msft @bbkane Can az rest be used to invoke Azure DevOps REST APIs? I haven't been successful in doing so since ADO REST APIs require one to pass a PAT as part of --headers and I couldn't.

Just see a 403 when I run this:

az rest --method get --url https://dev.azure.com/xxx/xxx/_apis/wit/workitems/456?api-version=5.0 --headers Authorization="Basic <PAT>"

Loading

@jiasli
Copy link
Member

@jiasli jiasli commented Aug 30, 2021

@BhargaviAnnadevara-MSFT, az rest definitely works with Azure DevOps REST APIs:

You can see the 'Authorization': 'Basic...' header is set correctly:

> az rest --method get --url https://dev.azure.com/xxx/xxx/_apis/wit/workitems/456?api-version=5.0 --headers Authorization="Basic <PAT>" --verbose
Request URL: 'https://dev.azure.com/xxx/xxx/_apis/wit/workitems/456?api-version=5.0'
Request method: 'GET'
Request headers:
    ...
    'Authorization': 'Basic...'
    ...
Request body:
None
Response status: 401
...

Please verify if the PAT is valid. You may use other REST API client, like postman, to cross check.

Loading

@BhargaviAnnadevara-MSFT

@jiasli Yes, the PAT is a valid one since I was able to make the call via Postman. After some more troubleshooting, I realized the PAT is supposed to be a base64 encoded string. Did that and the az rest command got through. Thank you very much for the prompt response!

Loading

@bbkane
Copy link

@bbkane bbkane commented Aug 30, 2021

@BhargaviAnnadevara-MSFT , in my experience (I'm not an az developer), the benefits of az rest are automatic authentication ( which you aren't using, you're using a PAT to authenticate) and url resolution (which you also aren't using, you're using a full url).

If az rest works for you, that's great, but if you're going to distribute this script to team members, you might consider using a tool more likely to be installed by default, such as curl or Invoke-WebRequest .

az is kind of a heavy dependency especially if you don't need its advantages :) . Of course, if your team already has a lot of experience with az, then az rest might be the best for you just because of that.

Loading

@BhargaviAnnadevara-MSFT
Copy link

@BhargaviAnnadevara-MSFT BhargaviAnnadevara-MSFT commented Aug 30, 2021

@bbkane Agree. :) However, this effort is for a customer who has their setup scripted all in az and doesn't want to switch for the one API call. Hence the need. Thanks much for the info!

Loading

@jiasli
Copy link
Member

@jiasli jiasli commented Aug 31, 2021

@bbkane, nice explanation! You just said everything I want to say. 😄

@BhargaviAnnadevara-MSFT, glad to know az rest is working for you.

Indeed, az rest is designed to automatically authenticate the request using an AAD Bearer access token, furtherly optimized for ARM resource ID as the URL. If you use Basic authentication, using curl or Invoke-WebRequest would be a better choice in environments without az. That being said, az rest is still a fully functional REST client that you may use anywhere.

Loading

@jamesrapadmi
Copy link

@jamesrapadmi jamesrapadmi commented Aug 31, 2021

@yugangw-msft I have the same issue as @BhargaviAnnadevara-MSFT. Is there a --resource value that will allow access to the https://dev.azure.com/_apis? Or can you only use the PAT method for this? The az command clearly can access other devops apis.

Loading

@jiasli
Copy link
Member

@jiasli jiasli commented Sep 1, 2021

@jamesrapadmi,

Is there a --resource value that will allow access to the https://dev.azure.com/_apis?

This is out of the scope of Azure CLI, but I will try my best to answer.


Using AAD Bearer token to call Azure DevOps API seems to be undocumented. I am not able to find the resource ID of Azure DevOps API https://dev.azure.com/ from the doc https://docs.microsoft.com/en-us/azure/devops/integrate/get-started/authentication/authentication-guidance

Then I followed the sample for Interactive client-side (REST):

image

The scopes (for MSAL) is exactly the resource (for ADAL, which Azure CLI still uses) we want:

https://github.com/microsoft/azure-devops-auth-samples/blob/30ad4e1ed9cf343c3b001bb07a9a967370661b6b/ManagedClientConsoleAppSample/Program.cs#L28

        internal static string[] scopes = new string[] { "499b84ac-1321-427f-aa17-267ca6975798/user_impersonation" }; //Constant value to target Azure DevOps. Do not change  

So we can use 499b84ac-1321-427f-aa17-267ca6975798 as the value of --resource to call az rest:

> az rest --url 'https://dev.azure.com/azure-sdk/_apis/projects?stateFilter=All&api-version=2.2' --resource 499b84ac-1321-427f-aa17-267ca6975798
{
  "count": 4,
  "value": [
    {
      ...
      "name": "Release",
      ...
    },
    {
      ...
      "name": "public",
      ...
    },
    ...
  ]
}

If you use PowerShell, pay attention to quoting rules: https://github.com/Azure/azure-cli/blob/dev/doc/quoting-issues-with-powershell.md

Too long; don't read

Actually I did search for 499b84ac-1321-427f-aa17-267ca6975798 in the doc:

image

The only place it appears is https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/manage-personal-access-tokens-via-api. The doc only mentions 499b84ac-1321-427f-aa17-267ca6975798 is for https://vssps.dev.azure.com/, but it doesn't mention 499b84ac-1321-427f-aa17-267ca6975798 can also be used for https://dev.azure.com/.

Also I found an error in the sample: microsoft/azure-devops-auth-samples#79

Loading

@jamesrapadmi
Copy link

@jamesrapadmi jamesrapadmi commented Sep 1, 2021

@jiasli that works! Thank you so much! I thought there would be a magic value to make this all work and you've found it for me . Much appreciated.

Loading

@BhargaviAnnadevara-MSFT
Copy link

@BhargaviAnnadevara-MSFT BhargaviAnnadevara-MSFT commented Sep 1, 2021

Awesomesauce! :D Thank you very much @jiasli!

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.