New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

properties are shown as null when query microsoft.insights resources #3457

Closed
SergeyKanzhelev opened this Issue May 25, 2017 · 13 comments

Comments

Projects
None yet
3 participants
@SergeyKanzhelev
Member

SergeyKanzhelev commented May 25, 2017

Description

When I run az resource list --namespace microsoft.insights --resource-type components I get the list of Application Insights components. Json field properties is shown as null for them:

  {
    "id": "/subscriptions/01f5dafd-52e6-4115-b012-803e14712153/resourceGroups/metricstest/providers/microsoft.insights/components/metricstest",
    "identity": null,
    "kind": "web",
    "location": "eastus",
    "managedBy": null,
    "name": "metricstest",
    "plan": null,
    "properties": null,
    "resourceGroup": "metricstest",
    "sku": null,
    "tags": {},
    "type": "microsoft.insights/components"
  },

Actual properties looks like this:

{
  "id": "/subscriptions/01f5dafd-52e6-4115-b012-803e14712153/resourceGroups/MetricsTest/providers/microsoft.insights/components/metricstest",
  "name": "metricstest",
  "type": "microsoft.insights/components",
  "location": "East US",
  "tags": {},
  "kind": "web",
  "etag": "\"02001b81-0000-0000-0000-582e73720000\"",
  "properties": {
    "Ver": "v2",
    "ApplicationId": "metricstest",
    "AppId": "0c7e8188-1694-46ed-a642-6256073c177f",
    "Application_Type": "web",
    "Flow_Type": "Redfield",
    "Request_Source": "IbizaAIExtension",
    "InstrumentationKey": "6f54ba65-5b29-423c-a696-6e93c7fa0c58",
    "Name": "metricstest",
    "CreationDate": "2016-06-26T17:51:47.3274152+00:00",
    "PackageId": null,
    "TenantId": "94e37bed-e625-4771-ac40-fd27b8c9f940",
    "HockeyAppId": null,
    "HockeyAppToken": null,
    "provisioningState": "Succeeded",
    "SamplingPercentage": null
  }
}

Environment summary

Install Method: Just used it from Azure portal:

image

CLI Version: What version of the CLI and modules are installed? (Use az --version)
Answer here:

sergey@Azure:~$ az --version
azure-cli (2.0.6)

acr (2.0.4)
acs (2.0.6)
appservice (0.1.6)
batch (2.0.4)
cdn (0.0.2)
cloud (2.0.2)
cognitiveservices (0.1.2)
command-modules-nspkg (2.0.0)
component (2.0.4)
configure (2.0.6)
core (2.0.6)
cosmosdb (0.1.6)
dla (0.0.6)
dls (0.0.6)
feedback (2.0.2)
find (0.2.2)
interactive (0.3.2)
iot (0.1.5)
keyvault (2.0.4)
lab (0.0.4)
monitor (0.0.4)
network (2.0.6)
nspkg (3.0.0)
profile (2.0.4)
rdbms (0.0.1)
redis (0.2.3)
resource (2.0.6)
role (2.0.4)
sf (1.0.1)
sql (2.0.3)
storage (2.0.6)
vm (2.0.6)

Python (Linux) 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609]

Python location '/usr/bin/python'

OS Version: What OS and version are you using?
Answer here: In-browser on Azure portal.

Shell Type: What shell are you using? (e.g. bash, cmd.exe, Bash on Windows)
Answer here: In-browser on Azure portal

@tjprescott

This comment has been minimized.

Show comment
Hide comment
@tjprescott

tjprescott May 25, 2017

Member

Were the actual properties supplied by the resource show command? If so, this may simply reflect a difference in what the generic ARM APIs return for list and show.

Member

tjprescott commented May 25, 2017

Were the actual properties supplied by the resource show command? If so, this may simply reflect a difference in what the generic ARM APIs return for list and show.

@SergeyKanzhelev

This comment has been minimized.

Show comment
Hide comment
@SergeyKanzhelev
Member

SergeyKanzhelev commented May 25, 2017

@frankguodongchen do you know?

@frankguodongchen

This comment has been minimized.

Show comment
Hide comment
@frankguodongchen

frankguodongchen May 25, 2017

ARM have a cache copy for the resources (it does not cache properties of resources), and for the list api, it was handled by ARM directly and does not pass to resource provider, so it will return back properties as null. When ask individual resource, ARM pass through the call to resource provider, so it will be able to get the full properties information.

frankguodongchen commented May 25, 2017

ARM have a cache copy for the resources (it does not cache properties of resources), and for the list api, it was handled by ARM directly and does not pass to resource provider, so it will return back properties as null. When ask individual resource, ARM pass through the call to resource provider, so it will be able to get the full properties information.

@SergeyKanzhelev

This comment has been minimized.

Show comment
Hide comment
@SergeyKanzhelev

SergeyKanzhelev May 25, 2017

Member

@tjprescott I'd really like to be able to get the list of components and their instrumentation key (which is in properties) in a single query to azure cli. Do you know who can we ask from ARM to return properties when cache is being used?

Member

SergeyKanzhelev commented May 25, 2017

@tjprescott I'd really like to be able to get the list of components and their instrumentation key (which is in properties) in a single query to azure cli. Do you know who can we ask from ARM to return properties when cache is being used?

@frankguodongchen

This comment has been minimized.

Show comment
Hide comment
@frankguodongchen

frankguodongchen May 25, 2017

I think a work around can try out is that give a "$filter=" parameter (accept OData format) in to filter by a property, it will force it go to resource provider, and then I think it possible get back properties.

frankguodongchen commented May 25, 2017

I think a work around can try out is that give a "$filter=" parameter (accept OData format) in to filter by a property, it will force it go to resource provider, and then I think it possible get back properties.

@tjprescott tjprescott added the Question label May 25, 2017

@tjprescott

This comment has been minimized.

Show comment
Hide comment
@tjprescott

tjprescott May 25, 2017

Member

My first thought was you do something like:

az resource list --type Microsoft.insights/component --query "[].id" -o tsv | az resource show --ids @-

However, that doesn't work because the resource show command doesn't accept --ids!

I've reached out to the ARM team to see if they have any recommendations on this.

Member

tjprescott commented May 25, 2017

My first thought was you do something like:

az resource list --type Microsoft.insights/component --query "[].id" -o tsv | az resource show --ids @-

However, that doesn't work because the resource show command doesn't accept --ids!

I've reached out to the ARM team to see if they have any recommendations on this.

@SergeyKanzhelev

This comment has been minimized.

Show comment
Hide comment
@SergeyKanzhelev

SergeyKanzhelev May 25, 2017

Member

@frankguodongchen I do not know how to pass filter to the az resource list command. Can you please advice

@tjprescott using pipes I can write something like

az resource list --namespace microsoft.insights --resource-type components --query [*].[id] --o tsv 
| while read ID; do az resource show --id "$ID"; done

However now I struggle with az resource show that do not work well even with the "take all" query like --query [*]. I need --query [*].[Properties].[InstrumentationKey]. Not talking about performance as querying component by component is quite slow

The scenario I'm trying to implement is to find the component with the specific instrumentation key. So far from what I see it requires quite a lot of scripting.

Member

SergeyKanzhelev commented May 25, 2017

@frankguodongchen I do not know how to pass filter to the az resource list command. Can you please advice

@tjprescott using pipes I can write something like

az resource list --namespace microsoft.insights --resource-type components --query [*].[id] --o tsv 
| while read ID; do az resource show --id "$ID"; done

However now I struggle with az resource show that do not work well even with the "take all" query like --query [*]. I need --query [*].[Properties].[InstrumentationKey]. Not talking about performance as querying component by component is quite slow

The scenario I'm trying to implement is to find the component with the specific instrumentation key. So far from what I see it requires quite a lot of scripting.

@tjprescott

This comment has been minimized.

Show comment
Hide comment
@tjprescott

tjprescott May 26, 2017

Member

@SergeyKanzhelev as written, you cannot write a custom filter. Instead the command provides convenience arguments for common scenarios and we build the OData filter behind the scenes. However, we can (and probably should) offer this for expert users.

I'm glad you found a way to get the information you wanted. I experimented with some ways that essentially do what you did in a slightly more convenient way, but pulled back for the same performance issues you noted (as well as REST API quota implications) in favor of engaging the service team.

However, ultimately, using the ARM commands is a stopgap for the fact that we don't have strongly-typed appinsights commands in the CLI. I see that they do have a Swagger spec, so I'll engage with the necessary teams to get this on our roadmap.

Member

tjprescott commented May 26, 2017

@SergeyKanzhelev as written, you cannot write a custom filter. Instead the command provides convenience arguments for common scenarios and we build the OData filter behind the scenes. However, we can (and probably should) offer this for expert users.

I'm glad you found a way to get the information you wanted. I experimented with some ways that essentially do what you did in a slightly more convenient way, but pulled back for the same performance issues you noted (as well as REST API quota implications) in favor of engaging the service team.

However, ultimately, using the ARM commands is a stopgap for the fact that we don't have strongly-typed appinsights commands in the CLI. I see that they do have a Swagger spec, so I'll engage with the necessary teams to get this on our roadmap.

@tjprescott

This comment has been minimized.

Show comment
Hide comment
@tjprescott

tjprescott May 26, 2017

Member

Created issue #3478 to track the on-boarding of the AppInsights service. Issue #3477 tracks adding support for raw --filter.

Member

tjprescott commented May 26, 2017

Created issue #3478 to track the on-boarding of the AppInsights service. Issue #3477 tracks adding support for raw --filter.

@SergeyKanzhelev

This comment has been minimized.

Show comment
Hide comment
@SergeyKanzhelev

SergeyKanzhelev May 26, 2017

Member

@tjprescott thanks for the follow up!

However, ultimately, using the ARM commands is a stopgap for the fact that we don't have strongly-typed appinsights commands in the CLI. I see that they do have a Swagger spec, so I'll engage with the necessary teams to get this on our roadmap.

@tjprescott my hope was that az resource list is a generic command that have to allow access and query by any json property visible in https://resources.azure.com/ The entire experience of cached vs. non-cached results with the missing fields looks like a bug. The fact that az resource show --id "<id>" --query [*].[properties] doesn't return result and forces to parse json is a bug.

Forcing to learn RP-specific commands set to compensate for the shortcomings of a generic az resource list does not sound like a good approach.

What did ARM team said regarding cached results that doesn't return properties?

Member

SergeyKanzhelev commented May 26, 2017

@tjprescott thanks for the follow up!

However, ultimately, using the ARM commands is a stopgap for the fact that we don't have strongly-typed appinsights commands in the CLI. I see that they do have a Swagger spec, so I'll engage with the necessary teams to get this on our roadmap.

@tjprescott my hope was that az resource list is a generic command that have to allow access and query by any json property visible in https://resources.azure.com/ The entire experience of cached vs. non-cached results with the missing fields looks like a bug. The fact that az resource show --id "<id>" --query [*].[properties] doesn't return result and forces to parse json is a bug.

Forcing to learn RP-specific commands set to compensate for the shortcomings of a generic az resource list does not sound like a good approach.

What did ARM team said regarding cached results that doesn't return properties?

@tjprescott

This comment has been minimized.

Show comment
Hide comment
@tjprescott

tjprescott May 26, 2017

Member

For resource show, I think your query syntax is incorrect. It should look like --query properties.instrumentationKey. The show command returns only a single object, so the [*] in the beginning of your syntax will cause it to output nothing.

Member

tjprescott commented May 26, 2017

For resource show, I think your query syntax is incorrect. It should look like --query properties.instrumentationKey. The show command returns only a single object, so the [*] in the beginning of your syntax will cause it to output nothing.

@SergeyKanzhelev

This comment has been minimized.

Show comment
Hide comment
@SergeyKanzhelev

SergeyKanzhelev May 26, 2017

Member

@tjprescott oh, I see. I haven't noticed that az resource show returns a single element. When you mentioned that it doesn't support --ids - did you mean that the plan is to make it work similarly to az resource list?

Thank you again for the help understanding cli.

Member

SergeyKanzhelev commented May 26, 2017

@tjprescott oh, I see. I haven't noticed that az resource show returns a single element. When you mentioned that it doesn't support --ids - did you mean that the plan is to make it work similarly to az resource list?

Thank you again for the help understanding cli.

@tjprescott tjprescott referenced this issue May 26, 2017

Closed

[ARM] Resource commands should support --ids #3479

2 of 3 tasks complete
@tjprescott

This comment has been minimized.

Show comment
Hide comment
@tjprescott

tjprescott May 26, 2017

Member

@SergeyKanzhelev I mean that most of the strongly typed show/update/delete commands support a parameter called --ids. It essentially performs that operation once for each ID in the list and is implemented in the CLI's infrastructure so that it can be common across command modules.

Resource show doesn't support this same mechanism (it only accepts a single ID) so we should see if it can be made to work within the same infrastructure. It wouldn't really improve your scenario as it stands today--it would just make that command more consistent with the rest of the CLI. I've created a separate issue for this (#3479).

Member

tjprescott commented May 26, 2017

@SergeyKanzhelev I mean that most of the strongly typed show/update/delete commands support a parameter called --ids. It essentially performs that operation once for each ID in the list and is implemented in the CLI's infrastructure so that it can be common across command modules.

Resource show doesn't support this same mechanism (it only accepts a single ID) so we should see if it can be made to work within the same infrastructure. It wouldn't really improve your scenario as it stands today--it would just make that command more consistent with the rest of the CLI. I've created a separate issue for this (#3479).

@tjprescott tjprescott closed this Sep 7, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment