properties are shown as null when query microsoft.insights resources

[SergeyKanzhelev]

Description

When I run az resource list --namespace microsoft.insights --resource-type components I get the list of Application Insights components. Json field properties is shown as null for them:

  {
    "id": "/subscriptions/01f5dafd-52e6-4115-b012-803e14712153/resourceGroups/metricstest/providers/microsoft.insights/components/metricstest",
    "identity": null,
    "kind": "web",
    "location": "eastus",
    "managedBy": null,
    "name": "metricstest",
    "plan": null,
    "properties": null,
    "resourceGroup": "metricstest",
    "sku": null,
    "tags": {},
    "type": "microsoft.insights/components"
  },

Actual properties looks like this:

{
  "id": "/subscriptions/01f5dafd-52e6-4115-b012-803e14712153/resourceGroups/MetricsTest/providers/microsoft.insights/components/metricstest",
  "name": "metricstest",
  "type": "microsoft.insights/components",
  "location": "East US",
  "tags": {},
  "kind": "web",
  "etag": "\"02001b81-0000-0000-0000-582e73720000\"",
  "properties": {
    "Ver": "v2",
    "ApplicationId": "metricstest",
    "AppId": "0c7e8188-1694-46ed-a642-6256073c177f",
    "Application_Type": "web",
    "Flow_Type": "Redfield",
    "Request_Source": "IbizaAIExtension",
    "InstrumentationKey": "6f54ba65-5b29-423c-a696-6e93c7fa0c58",
    "Name": "metricstest",
    "CreationDate": "2016-06-26T17:51:47.3274152+00:00",
    "PackageId": null,
    "TenantId": "94e37bed-e625-4771-ac40-fd27b8c9f940",
    "HockeyAppId": null,
    "HockeyAppToken": null,
    "provisioningState": "Succeeded",
    "SamplingPercentage": null
  }
}

---

Environment summary

Install Method: Just used it from Azure portal:

CLI Version: What version of the CLI and modules are installed? (Use az --version)
Answer here:

sergey@Azure:~$ az --version
azure-cli (2.0.6)

acr (2.0.4)
acs (2.0.6)
appservice (0.1.6)
batch (2.0.4)
cdn (0.0.2)
cloud (2.0.2)
cognitiveservices (0.1.2)
command-modules-nspkg (2.0.0)
component (2.0.4)
configure (2.0.6)
core (2.0.6)
cosmosdb (0.1.6)
dla (0.0.6)
dls (0.0.6)
feedback (2.0.2)
find (0.2.2)
interactive (0.3.2)
iot (0.1.5)
keyvault (2.0.4)
lab (0.0.4)
monitor (0.0.4)
network (2.0.6)
nspkg (3.0.0)
profile (2.0.4)
rdbms (0.0.1)
redis (0.2.3)
resource (2.0.6)
role (2.0.4)
sf (1.0.1)
sql (2.0.3)
storage (2.0.6)
vm (2.0.6)

Python (Linux) 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609]

Python location '/usr/bin/python'

OS Version: What OS and version are you using?
Answer here: In-browser on Azure portal.

Shell Type: What shell are you using? (e.g. bash, cmd.exe, Bash on Windows)
Answer here: In-browser on Azure portal

[tjprescott]

Were the actual properties supplied by the resource show command? If so, this may simply reflect a difference in what the generic ARM APIs return for list and show.

[tjprescott]

Were the actual properties supplied by the resource show command? If so, this may simply reflect a difference in what the generic ARM APIs return for list and show.

[SergeyKanzhelev]

@frankguodongchen do you know?

[SergeyKanzhelev]

@frankguodongchen do you know?

[frankguodongchen]

ARM have a cache copy for the resources (it does not cache properties of resources), and for the list api, it was handled by ARM directly and does not pass to resource provider, so it will return back properties as null. When ask individual resource, ARM pass through the call to resource provider, so it will be able to get the full properties information.

[frankguodongchen]

ARM have a cache copy for the resources (it does not cache properties of resources), and for the list api, it was handled by ARM directly and does not pass to resource provider, so it will return back properties as null. When ask individual resource, ARM pass through the call to resource provider, so it will be able to get the full properties information.

[SergeyKanzhelev]

@tjprescott I'd really like to be able to get the list of components and their instrumentation key (which is in properties) in a single query to azure cli. Do you know who can we ask from ARM to return properties when cache is being used?

[SergeyKanzhelev]

@tjprescott I'd really like to be able to get the list of components and their instrumentation key (which is in properties) in a single query to azure cli. Do you know who can we ask from ARM to return properties when cache is being used?

[frankguodongchen]

I think a work around can try out is that give a "$filter=" parameter (accept OData format) in to filter by a property, it will force it go to resource provider, and then I think it possible get back properties.

[frankguodongchen]

I think a work around can try out is that give a "$filter=" parameter (accept OData format) in to filter by a property, it will force it go to resource provider, and then I think it possible get back properties.

[tjprescott]

My first thought was you do something like:

az resource list --type Microsoft.insights/component --query "[].id" -o tsv | az resource show --ids @-

However, that doesn't work because the resource show command doesn't accept --ids!

I've reached out to the ARM team to see if they have any recommendations on this.

[tjprescott]

My first thought was you do something like:

az resource list --type Microsoft.insights/component --query "[].id" -o tsv | az resource show --ids @-

However, that doesn't work because the resource show command doesn't accept --ids!

I've reached out to the ARM team to see if they have any recommendations on this.

[SergeyKanzhelev]

@frankguodongchen I do not know how to pass filter to the az resource list command. Can you please advice

@tjprescott using pipes I can write something like

az resource list --namespace microsoft.insights --resource-type components --query [*].[id] --o tsv 
| while read ID; do az resource show --id "$ID"; done

However now I struggle with az resource show that do not work well even with the "take all" query like --query [*]. I need --query [*].[Properties].[InstrumentationKey]. Not talking about performance as querying component by component is quite slow

The scenario I'm trying to implement is to find the component with the specific instrumentation key. So far from what I see it requires quite a lot of scripting.

[SergeyKanzhelev]

@frankguodongchen I do not know how to pass filter to the az resource list command. Can you please advice

@tjprescott using pipes I can write something like

az resource list --namespace microsoft.insights --resource-type components --query [*].[id] --o tsv 
| while read ID; do az resource show --id "$ID"; done

However now I struggle with az resource show that do not work well even with the "take all" query like --query [*]. I need --query [*].[Properties].[InstrumentationKey]. Not talking about performance as querying component by component is quite slow

The scenario I'm trying to implement is to find the component with the specific instrumentation key. So far from what I see it requires quite a lot of scripting.

[tjprescott]

@SergeyKanzhelev as written, you cannot write a custom filter. Instead the command provides convenience arguments for common scenarios and we build the OData filter behind the scenes. However, we can (and probably should) offer this for expert users.

I'm glad you found a way to get the information you wanted. I experimented with some ways that essentially do what you did in a slightly more convenient way, but pulled back for the same performance issues you noted (as well as REST API quota implications) in favor of engaging the service team.

However, ultimately, using the ARM commands is a stopgap for the fact that we don't have strongly-typed appinsights commands in the CLI. I see that they do have a Swagger spec, so I'll engage with the necessary teams to get this on our roadmap.

[tjprescott]

@SergeyKanzhelev as written, you cannot write a custom filter. Instead the command provides convenience arguments for common scenarios and we build the OData filter behind the scenes. However, we can (and probably should) offer this for expert users.

I'm glad you found a way to get the information you wanted. I experimented with some ways that essentially do what you did in a slightly more convenient way, but pulled back for the same performance issues you noted (as well as REST API quota implications) in favor of engaging the service team.

However, ultimately, using the ARM commands is a stopgap for the fact that we don't have strongly-typed appinsights commands in the CLI. I see that they do have a Swagger spec, so I'll engage with the necessary teams to get this on our roadmap.

[tjprescott]

Created issue #3478 to track the on-boarding of the AppInsights service. Issue #3477 tracks adding support for raw --filter.

[tjprescott]

Created issue #3478 to track the on-boarding of the AppInsights service. Issue #3477 tracks adding support for raw --filter.

[SergeyKanzhelev]

@tjprescott thanks for the follow up!

However, ultimately, using the ARM commands is a stopgap for the fact that we don't have strongly-typed appinsights commands in the CLI. I see that they do have a Swagger spec, so I'll engage with the necessary teams to get this on our roadmap.

@tjprescott my hope was that az resource list is a generic command that have to allow access and query by any json property visible in https://resources.azure.com/ The entire experience of cached vs. non-cached results with the missing fields looks like a bug. The fact that az resource show --id "<id>" --query [*].[properties] doesn't return result and forces to parse json is a bug.

Forcing to learn RP-specific commands set to compensate for the shortcomings of a generic az resource list does not sound like a good approach.

What did ARM team said regarding cached results that doesn't return properties?

[SergeyKanzhelev]

@tjprescott thanks for the follow up!

However, ultimately, using the ARM commands is a stopgap for the fact that we don't have strongly-typed appinsights commands in the CLI. I see that they do have a Swagger spec, so I'll engage with the necessary teams to get this on our roadmap.

@tjprescott my hope was that az resource list is a generic command that have to allow access and query by any json property visible in https://resources.azure.com/ The entire experience of cached vs. non-cached results with the missing fields looks like a bug. The fact that az resource show --id "<id>" --query [*].[properties] doesn't return result and forces to parse json is a bug.

Forcing to learn RP-specific commands set to compensate for the shortcomings of a generic az resource list does not sound like a good approach.

What did ARM team said regarding cached results that doesn't return properties?

[tjprescott]

For resource show, I think your query syntax is incorrect. It should look like --query properties.instrumentationKey. The show command returns only a single object, so the [*] in the beginning of your syntax will cause it to output nothing.

[tjprescott]

For resource show, I think your query syntax is incorrect. It should look like --query properties.instrumentationKey. The show command returns only a single object, so the [*] in the beginning of your syntax will cause it to output nothing.

[SergeyKanzhelev]

@tjprescott oh, I see. I haven't noticed that az resource show returns a single element. When you mentioned that it doesn't support --ids - did you mean that the plan is to make it work similarly to az resource list?

Thank you again for the help understanding cli.

[SergeyKanzhelev]

@tjprescott oh, I see. I haven't noticed that az resource show returns a single element. When you mentioned that it doesn't support --ids - did you mean that the plan is to make it work similarly to az resource list?

Thank you again for the help understanding cli.

[tjprescott]

@SergeyKanzhelev I mean that most of the strongly typed show/update/delete commands support a parameter called --ids. It essentially performs that operation once for each ID in the list and is implemented in the CLI's infrastructure so that it can be common across command modules.

Resource show doesn't support this same mechanism (it only accepts a single ID) so we should see if it can be made to work within the same infrastructure. It wouldn't really improve your scenario as it stands today--it would just make that command more consistent with the rest of the CLI. I've created a separate issue for this (#3479).

[tjprescott]

@SergeyKanzhelev I mean that most of the strongly typed show/update/delete commands support a parameter called --ids. It essentially performs that operation once for each ID in the list and is implemented in the CLI's infrastructure so that it can be common across command modules.

Resource show doesn't support this same mechanism (it only accepts a single ID) so we should see if it can be made to work within the same infrastructure. It wouldn't really improve your scenario as it stands today--it would just make that command more consistent with the rest of the CLI. I've created a separate issue for this (#3479).