Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resource IDs containing spaces can't be consumed by other az commands #12885

Open
mattcuk opened this issue Apr 6, 2020 · 20 comments
Open

Resource IDs containing spaces can't be consumed by other az commands #12885

mattcuk opened this issue Apr 6, 2020 · 20 comments
Assignees
Labels
ARM customer-reported feature-request needs-team-attention Service Attention
Milestone

Comments

@mattcuk
Copy link

mattcuk commented Apr 6, 2020

I've been trying to use a command to list all the Logic App IDs which have an Alert set up for them. To do this, I'm running 2 commands; az resource list, and az resource show. However, since az resource list returns IDs that contain spaces, az resource show will then fail.

Command Name
az resource show --ids $(az resource list --resource-type Microsoft.Insights/metricAlerts --query [].id --output tsv) --query properties.scopes

Errors:
az resource: error: argument --ids: invalid ResourceId value: 'failed'

To Reproduce:

Create an alert against any Logic App.. make sure the Alert Name has a space in it, like 'logic-app-test-la failed'.

Run the command above.

Expected Behavior

It should really be able to handle the encoding of the spaces so that az resource show can ingest the result of az resource list without having to resort to a Bash script to work round the problem.

Environment Summary

Linux-4.15.0-1075-azure-x86_64-with-debian-stretch-sid
Python 3.6.5

azure-cli 2.2.0 *

Extensions:
interactive 0.4.4

Additional Context

@yonzhan yonzhan added the ARM label Apr 6, 2020
@triage-new-issues triage-new-issues bot removed the triage label Apr 6, 2020
@yonzhan yonzhan added this to the S169 - For Build milestone Apr 6, 2020
@yonzhan
Copy link
Collaborator

yonzhan commented Apr 6, 2020

add to S169

@yonzhan
Copy link
Collaborator

yonzhan commented Apr 10, 2020

resource

@yonzhan yonzhan added the OKR3.4 Candidate label Apr 18, 2020
@zhoxing-ms
Copy link
Contributor

zhoxing-ms commented Apr 20, 2020

Hi @mattcuk, I tried to use the resource name logic-app-test-la failed, but I didn't reproduce your problem.
Could you please print out the result of the execution of command az resource list --resource-type Microsoft.Insights/metricAlerts --query [].id --output tsv, and show it to me?

@e14mattc
Copy link

e14mattc commented Apr 20, 2020

Sure.. here's a sample of some of the results.. I've removed the subscription ID + changed some of the names a bit..

/subscriptions/eec/resourceGroups/eg-app-rg-009/providers/microsoft.insights/metricalerts/eg-ecomm-azuresphererouting-failures
/subscriptions/eec/resourceGroups/eg-app-rg-009/providers/microsoft.insights/metricalerts/eg-ecomm-azuresphereupdateexcel-failure-alerts
/subscriptions/eec/resourceGroups/eg-app-transkeyvalue-rg-019/providers/microsoft.insights/metricalerts/eg-extendedrange-init-la failed
/subscriptions/eec/resourceGroups/eg-app-transkeyvalue-rg-019/providers/microsoft.insights/metricalerts/eg-persistent-complete-la failed
/subscriptions/eec/resourceGroups/eg-app-transkeyvalue-rg-019/providers/microsoft.insights/metricalerts/eg-persistent-init-la failed
/subscriptions/eec/resourceGroups/eg-app-transkeyvalue-rg-019/providers/microsoft.insights/metricalerts/eg-persistentlog-la failed
/subscriptions/eec/resourceGroups/eg-app-transkeyvalue-rg-019/providers/microsoft.insights/metricalerts/expiredocs-la failed
/subscriptions/eec/resourceGroups/eg-app-transkeyvalue-rg-019/providers/microsoft.insights/metricalerts/persistentkeyval-datafactory
/subscriptions/eec/resourceGroups/eg-app-transkeyvalue-rg-019/providers/microsoft.insights/metricalerts/wuk-pf-persistent-imports-la failed
/subscriptions/eec/resourceGroups/eg-int-cmtyautomation-rg-012/providers/microsoft.insights/metricalerts/eg-ecomm-cmty-page-speed-logger-la failed
/subscriptions/eec/resourceGroups/eg-int-cmtyautomation-rg-012/providers/microsoft.insights/metricalerts/eg-ecomm-funcapp-restart-to-msteams-la failed

Hope this helps.

@zhoxing-ms
Copy link
Contributor

zhoxing-ms commented Apr 20, 2020

@e14mattc Hi, I created some new alerts based on these examples, and the problem still can't be reproduced.
Could you please add --debug to the original command? And then please show me the information returned, thanks~

@e14mattc
Copy link

e14mattc commented Apr 20, 2020

I'm going to email you direct @zhoxing-ms .. since I'm loathed to paste in production output here. Hope that's ok.

@zhoxing-ms
Copy link
Contributor

zhoxing-ms commented Apr 21, 2020

@e14mattc OK, thanks~ Has the email been sent? I don't seem to have received it

@e14mattc
Copy link

e14mattc commented Apr 21, 2020

@zhoxing-ms yes, I sent it at 10am BST yesterday. I wonder if it got stuck in a spam filter due it having a .txt file attachment? The email address I used was your MS account which you have on your GitHub profile. I'll send another with the output in the email itself, rather than an attachment.

@zhoxing-ms
Copy link
Contributor

zhoxing-ms commented Apr 24, 2020

@e14mattc The email has been received, but I still can't reproduce your question.
Could you please execute az resource show --ids $(az resource list --resource-type Microsoft.Insights/metricAlerts --query [].id --output tsv) --debug and show me the error message returned?

@e14mattc
Copy link

e14mattc commented Apr 24, 2020

Sure.. an email is winging its way to you.

For anyone reading along, the output I just emailed shows that the alert ID is being split up on the space.. the word 'failed' should be part of the alert ID;

image

@zhoxing-ms
Copy link
Contributor

zhoxing-ms commented Apr 26, 2020

I can reproduce the problem currently: there is no problem with this command executing in PowerShell of Windows, it only occurs in the Shell of Linux.
It seems to be a usage problem of $() in the Shell of Linux, the result of $(az resource list XXX) obtained in Shell is separated by the space in ResourceID. This situation is like creating a file with a filename contains spaces, and then executing echo $(ls), the filename containing spaces in the printed result is split into multiple names due to spaces.
So it's not a problem related to CLI, we can't deal with it. @e14mattc Do you have any questions about it?

@e14mattc
Copy link

e14mattc commented Apr 26, 2020

Couldn't the az resource list XXX command encode spaces in any resource IDs? That would solve the problem, wouldn't it?

@zhoxing-ms
Copy link
Contributor

zhoxing-ms commented Apr 28, 2020

Hi @e14mattc, In fact, this issue is not caused by encoding, but by the subexpression operator $() in Bash handling the result as a string.
For example:

When executing $(az resource list XXX) in PowerShell returns results as an array, so --ids is passed as an array to the CLI and can be parsed normally.
image

But executing $(az resource list XXX) in Bash returns results as a string, and the elements between arrays in the results are also separated by spaces. CLI parses the string value of --ids to convert the parameter to an array with a space as the separator, so the spaces in the resourceId will affect the parsing of the resourceId.
image

If you have a good idea, welcome to communicate with us~

@e14mattc
Copy link

e14mattc commented Apr 28, 2020

How many Azure resources allow a space in the ID, I wonder? Alerts are the only one I've come across so far. Feels like they should maybe restrict this to stop the user entering spaces & potentially running across this issue.

If the az resource list XXX command encoded any IDs that contained spaces with a %20 (maybe?) wouldn't this fix the problem & not break any existing Bash/Powershell scripts?

The code below is how I've worked around the issue, but it's not elegant & it's pretty slow. Is there a better way of dealing with this problem?

alerts=$(az resource list --output tsv --resource-type "Microsoft.Insights/metricAlerts" --query [].id)
alerts=${alerts// /SPACEFIX}

for alert in $alerts
do
        az resource show --ids "${alert//SPACEFIX/ }" --query properties.scopes[0] --output tsv
done

@zhoxing-ms
Copy link
Contributor

zhoxing-ms commented Apr 29, 2020

How many Azure resources allow a space in the ID, I wonder? Alerts are the only one I've come
across so far. Feels like they should maybe restrict this to stop the user entering spaces & potentially running across this issue.

This requires the service team to see why the resource id allows spaces in some resources.

If the az resource list XXX command encoded any IDs that contained spaces with a %20 (maybe?) wouldn't this fix the problem & not break any existing Bash/Powershell scripts?

Yes, this is a solution, and the service team needs to evaluate whether it is breaking change and whether it can be modified~
By the way, if this problem is not easy to solve at the rest API, CLI can consider adding a parameter for customers to encode the space.

@zhoxing-ms zhoxing-ms added the Service Attention label Apr 29, 2020
@msftbot
Copy link

msftbot bot commented Apr 29, 2020

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @armleads-azure.

@yonzhan yonzhan added feature-request and removed OKR3.4 Candidate labels May 16, 2020
@yonzhan yonzhan modified the milestones: S169 - For Build, Backlog May 16, 2020
@azure-sdk azure-sdk added the customer-reported label Sep 24, 2020
@navba-MSFT
Copy link
Contributor

navba-MSFT commented Mar 2, 2022

@mattcuk @e14mattc Apologies for the delayed response. Could you please let me know if you are still facing this issue ? Awaiting your reply.

@navba-MSFT navba-MSFT added the needs-author-feedback label Mar 2, 2022
@e14mattc
Copy link

e14mattc commented Mar 2, 2022

Apologies for the delayed response. Could you please let me know if you are still facing this issue ? Awaiting your reply.

Yes @navba-MSFT if I run the command in my OP it still fails with the same error.

@msftbot msftbot bot added the no-recent-activity label Mar 9, 2022
@msftbot
Copy link

msftbot bot commented Mar 9, 2022

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

@mattcuk
Copy link
Author

mattcuk commented Mar 9, 2022

Yes @navba-MSFT if I run the command in my OP it still fails with the same error.

@msftbot msftbot bot added needs-team-attention and removed needs-author-feedback no-recent-activity labels Mar 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ARM customer-reported feature-request needs-team-attention Service Attention
Projects
None yet
Development

No branches or pull requests

7 participants