[bug] az iot product test does not validate command responses with Map

[rido-min]

I'm validating a PnP device using a command with the next response

  "response": {
        "@type": "CommandPayload",
        "name": "diagnosticResults",
        "schema": {
          "@type": "Map",
          "mapKey": {
            "name": "key",
            "schema": "string"
          },
          "mapValue": {
            "name": "val",
            "schema": "string"
          }
        }
      }

The device returns a proper JSON object,

When running the az iot product test I got

              "message": "{\"commandName\":\"getRuntimeStats\",\"expectedSchemaType\":\"Map\",\"actualSchemaType\":\"Unknown\",\"message\":\"Inconsistent value type. Expected type: [String], actual type: [JObject], Error code:1004, Error code:1024\",\"passed\":false,\"time\":\"2022-01-27T19:18:30.5515017+00:00\"}",
              "time": "2022-01-27T19:18:30.5515017+00:00"
            },
            {
              "message": "PnP interface commands validation failed",
              "time": "2022-01-27T19:18:30.5519899+00:00"
            }

[digimaun]

The response is from the certification service, the az iot product test command group is not doing any device verification client side.

If you can provide more details on the arguments passed into the commands and with use of --debug, we can determine if the service calls are constructed correctly, but if everything checks out the next step is to engage the certification service team.

[rido-min]

for completness here is the response payload from the CLI

C:\code\temp\aics_cli> az iot hub invoke-device-method -n ridox --device-id memmon23 --method-name getRuntimeStats --method-payload '2'
{
  "payload": {
    "diagnosticResults": {
      "command: ": "1",
      "machine name": "RIDOSBOOK3",
      "os version": "Microsoft Windows NT 10.0.22000.0",
      "reconnects: ": "0",
      "telemetry: ": "11",
      "twin receive: ": "0",
      "twin sends: ": "4"
    }
  },
  "status": 200
}

full command invocation and debug traces below

az iot product test task create --type QueueTestRun --test-id 35346649-1627-4e57-b701-c45e7231a0fa --wait --debug

cli.knack.cli: Command arguments: ['iot', 'product', 'test', 'task', 'create', '--type', 'QueueTestRun', '--test-id', '35346649-1627-4e57-b701-c45e7231a0fa', '--wait', '--debug'] cli.knack.cli: __init__ debug log: Enable color in terminal. cli.knack.cli: Event: Cli.PreExecute [] cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [, , ] cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate [] cli.azure.cli.core: Modules found from index for 'iot': ['azure.cli.command_modules.iot', 'azext_iot'] cli.azure.cli.core: Loading command modules: cli.azure.cli.core: Name Load Time Groups Commands cli.azure.cli.core: iot 0.042 16 71 cli.azure.cli.core: Total (1) 0.042 16 71 cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next'] cli.azure.cli.core: Loading extensions: cli.azure.cli.core: Name Load Time Groups Commands Directory cli.azure.cli.core: azure-iot 0.026 60 203 C:\Users\rmpablos\.azure\cliextensions\azure-iot cli.azure.cli.core: Total (1) 0.026 60 203 cli.azure.cli.core: Loaded 72 groups, 274 commands. cli.azure.cli.core: Found a match in the command table. cli.azure.cli.core: Raw command : iot product test task create cli.azure.cli.core: Command table: iot product test task create cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [] cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to 'C:\Users\rmpablos\.azure\commands\2022-01-27.14-02-18.iot_product_test_task_create.34216.log'. az_command_data_logger: command args: iot product test task create --type {} --test-id {} --wait --debug cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [.add_subscription_parameter at 0x043522B0>, .register_query_examples at 0x043609B8>] cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad [] cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [.add_ids_arguments at 0x04360A00>, .add_cache_arguments at 0x04360A90>] cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded [] cli.knack.cli: Event: CommandInvoker.OnPreParseArgs [] cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [, , .handle_example_parameter at 0x04352388>, .parse_ids_arguments at 0x04360A48>, ] az_command_data_logger: extension name: azure-iot az_command_data_logger: extension version: 0.13.0 Command group 'iot product' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus msrest.universal_http.requests: Configuring retry: max_retries=4, backoff_factor=0.8, max_backoff=90 cli.azure.cli.core.auth.persistence: build_persistence: location='C:\\Users\\rmpablos\\.azure\\msal_token_cache.bin', encrypt=True cli.azure.cli.core.auth.identity: _load_msal_http_cache: C:\Users\rmpablos\.azure\msal_http_cache.bin urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None) msal.authority: openid_config = {'token_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/kerberos', 'tenant_region_scope': 'WW', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.microsoft.com', 'rbac_url': 'https://pas.windows.net'} cli.azure.cli.core.auth.msal_authentication: UserCredential.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={} msal.application: Cache hit an AT msal.telemetry: Generate or reuse correlation_id: 07842287-933b-4885-8416-94575e73a8d4 msrest.universal_http: Configuring redirects: allow=True, max=30 msrest.universal_http: Configuring request: timeout=100, verify=True, cert=None msrest.universal_http: Configuring proxies: '' msrest.universal_http: Evaluate proxies against ENV settings: True urllib3.connectionpool: Starting new HTTPS connection (1): certify.azureiotsolutions.com:443 urllib3.connectionpool: https://certify.azureiotsolutions.com:443 "POST /deviceTests/35346649-1627-4e57-b701-c45e7231a0fa/tasks?api-version=2020-05-01-preview HTTP/1.1" 202 170 urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None) msal.authority: openid_config = {'token_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/kerberos', 'tenant_region_scope': 'WW', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.microsoft.com', 'rbac_url': 'https://pas.windows.net'} cli.azure.cli.core.auth.msal_authentication: UserCredential.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={} msal.application: Cache hit an AT msal.telemetry: Generate or reuse correlation_id: 25c9abc8-bb99-491d-8f76-86db0d7c6c5e msrest.universal_http: Configuring redirects: allow=True, max=30 msrest.universal_http: Configuring request: timeout=100, verify=True, cert=None msrest.universal_http: Configuring proxies: '' msrest.universal_http: Evaluate proxies against ENV settings: True urllib3.connectionpool: Starting new HTTPS connection (1): certify.azureiotsolutions.com:443 urllib3.connectionpool: https://certify.azureiotsolutions.com:443 "GET /deviceTests/35346649-1627-4e57-b701-c45e7231a0fa/tasks/47ba688c-7678-41b5-abf7-66a97822bef2?api-version=2020-05-01-preview HTTP/1.1" 200 317 urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None) msal.authority: openid_config = {'token_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/kerberos', 'tenant_region_scope': 'WW', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.microsoft.com', 'rbac_url': 'https://pas.windows.net'} cli.azure.cli.core.auth.msal_authentication: UserCredential.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={} msal.application: Cache hit an AT msal.telemetry: Generate or reuse correlation_id: f9a316f1-ccf0-449c-861b-4042c57fdc63 msrest.universal_http: Configuring redirects: allow=True, max=30 msrest.universal_http: Configuring request: timeout=100, verify=True, cert=None msrest.universal_http: Configuring proxies: '' msrest.universal_http: Evaluate proxies against ENV settings: True urllib3.connectionpool: Starting new HTTPS connection (1): certify.azureiotsolutions.com:443 urllib3.connectionpool: https://certify.azureiotsolutions.com:443 "GET /deviceTests/35346649-1627-4e57-b701-c45e7231a0fa/tasks/47ba688c-7678-41b5-abf7-66a97822bef2?api-version=2020-05-01-preview HTTP/1.1" 200 317 urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None) msal.authority: openid_config = {'token_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/kerberos', 'tenant_region_scope': 'WW', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.microsoft.com', 'rbac_url': 'https://pas.windows.net'} cli.azure.cli.core.auth.msal_authentication: UserCredential.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={} msal.application: Cache hit an AT msal.telemetry: Generate or reuse correlation_id: 83b135f7-b4cb-49cb-884b-4781b021c402 msrest.universal_http: Configuring redirects: allow=True, max=30 msrest.universal_http: Configuring request: timeout=100, verify=True, cert=None msrest.universal_http: Configuring proxies: '' msrest.universal_http: Evaluate proxies against ENV settings: True urllib3.connectionpool: Starting new HTTPS connection (1): certify.azureiotsolutions.com:443 urllib3.connectionpool: https://certify.azureiotsolutions.com:443 "GET /deviceTests/35346649-1627-4e57-b701-c45e7231a0fa/tasks/47ba688c-7678-41b5-abf7-66a97822bef2?api-version=2020-05-01-preview HTTP/1.1" 200 317 urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None) msal.authority: openid_config = {'token_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/kerberos', 'tenant_region_scope': 'WW', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.microsoft.com', 'rbac_url': 'https://pas.windows.net'} cli.azure.cli.core.auth.msal_authentication: UserCredential.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={} msal.application: Cache hit an AT msal.telemetry: Generate or reuse correlation_id: 2d99a96e-a2f7-4f20-bc3d-d4d8195910a4 msrest.universal_http: Configuring redirects: allow=True, max=30 msrest.universal_http: Configuring request: timeout=100, verify=True, cert=None msrest.universal_http: Configuring proxies: '' msrest.universal_http: Evaluate proxies against ENV settings: True urllib3.connectionpool: Starting new HTTPS connection (1): certify.azureiotsolutions.com:443 urllib3.connectionpool: https://certify.azureiotsolutions.com:443 "GET /deviceTests/35346649-1627-4e57-b701-c45e7231a0fa/tasks/47ba688c-7678-41b5-abf7-66a97822bef2?api-version=2020-05-01-preview HTTP/1.1" 200 319 urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None) msal.authority: openid_config = {'token_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/kerberos', 'tenant_region_scope': 'WW', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.microsoft.com', 'rbac_url': 'https://pas.windows.net'} cli.azure.cli.core.auth.msal_authentication: UserCredential.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={} msal.application: Cache hit an AT msal.telemetry: Generate or reuse correlation_id: 302fc890-1cb8-4226-b0d1-39c739817e22 msrest.universal_http: Configuring redirects: allow=True, max=30 msrest.universal_http: Configuring request: timeout=100, verify=True, cert=None msrest.universal_http: Configuring proxies: '' msrest.universal_http: Evaluate proxies against ENV settings: True urllib3.connectionpool: Starting new HTTPS connection (1): certify.azureiotsolutions.com:443 urllib3.connectionpool: https://certify.azureiotsolutions.com:443 "GET /deviceTests/35346649-1627-4e57-b701-c45e7231a0fa/testRuns/61ae0d5d-48c5-4d5a-a876-a8e6178908b1?api-version=2020-05-01-preview HTTP/1.1" 200 1468 cli.knack.cli: Event: CommandInvoker.OnTransformResult [, ] cli.knack.cli: Event: CommandInvoker.OnFilterResult [] { "certificationBadgeResults": [ { "preValidationTasks": [ { "endTime": "2022-01-27T22:02:21.1146082+00:00", "logs": [ { "message": "DPS registration data blob is verified.", "time": "2022-01-27T22:02:21.1143429+00:00" } ], "name": "Validate DigitalTwin DPS registration", "rawData": [ "{\"payload\":{\"modelId\":\"dtmi:rido:pnp:memmon;1\"}}" ], "startTime": "2022-01-27T22:02:21.0573295+00:00", "status": "Passed" }, { "endTime": "2022-01-27T22:02:21.3904073+00:00", "logs": [ { "message": "Testcases are verified.", "time": "2022-01-27T22:02:21.3901891+00:00" } ], "name": "Validate DigitalTwin testcases", "rawData": [], "startTime": "2022-01-27T22:02:21.1154128+00:00", "status": "Passed" } ], "resolutionSource": 0, "type": "Pnp", "validationTasks": [ { "componentName": "Default component", "endTime": "2022-01-27T22:02:33.5324887+00:00", "interfaceId": "dtmi:rido:pnp:memmon;1", "logs": [ { "message": "Waiting for telemetry from the device", "time": "2022-01-27T22:02:21.3911621+00:00" }, { "message": "Validating PnP properties", "time": "2022-01-27T22:02:21.3984302+00:00" }, { "message": "Validating PnP commands", "time": "2022-01-27T22:02:21.4006695+00:00" }, { "message": "{\"propertyName\":\"started\",\"expectedSchemaType\":null,\"actualSchemaType\":null,\"message\":\"Property is successfully validated\",\"passed\":true,\"time\":\"2022-01-27T22:02:21.5543334+00:00\"}", "time": "2022-01-27T22:02:21.5543334+00:00" }, { "message": "{\"commandName\":\"getRuntimeStats\",\"expectedSchemaType\":\"Map\",\"actualSchemaType\":\"Unknown\",\"message\":\"Inconsistent value type. Expected type: [String], actual type: [JObject], Error code:1004, Error code:1024\",\"passed\":false,\"time\":\"2022-01-27T22:02:21.6477856+00:00\"}", "time": "2022-01-27T22:02:21.6477856+00:00" }, { "message": "PnP interface commands validation failed", "time": "2022-01-27T22:02:21.6480103+00:00" }, { "message": "{\"propertyName\":\"enabled\",\"expectedSchemaType\":null,\"actualSchemaType\":null,\"message\":\"Property is successfully validated\",\"passed\":true,\"time\":\"2022-01-27T22:02:21.6562344+00:00\"}", "time": "2022-01-27T22:02:21.6562344+00:00" }, { "message": "{\"propertyName\":\"interval\",\"expectedSchemaType\":null,\"actualSchemaType\":null,\"message\":\"Property is successfully validated\",\"passed\":true,\"time\":\"2022-01-27T22:02:21.6562399+00:00\"}", "time": "2022-01-27T22:02:21.6562399+00:00" }, { "message": "PnP interface properties validation passed", "time": "2022-01-27T22:02:21.6563608+00:00" }, { "message": "{\"telemetryName\":\"workingSet\",\"expectedSchemaType\":null,\"actualSchemaType\":null,\"message\":\"Telemetry is successfully validated\",\"passed\":true,\"time\":\"2022-01-27T22:02:33.5317486+00:00\"}", "time": "2022-01-27T22:02:33.5317486+00:00" }, { "message": "All the telemetries are successfully validated", "time": "2022-01-27T22:02:33.5319191+00:00" } ], "message": "All the telemetries are successfully validated", "name": null, "progressData": null, "rawData": [ "{\"telemetryName\":\"workingSet\",\"reportedValues\":[{\"time\":\"2022-01-27T22:02:33.5317323+00:00\",\"data\":\"50929664\"}],\"receivedTime\":\"2022-01-27T22:02:33.5317343+00:00\"}", "{\"propertyName\":\"started\",\"desiredValue\":null,\"reportedValue\":\"2022-01-27T14:02:13.9788355-08:00\",\"isWritable\":false,\"receivedTime\":\"2022-01-27T22:02:21.5543348+00:00\"}", "{\"propertyName\":\"enabled\",\"desiredValue\":false,\"reportedValue\":\"True\",\"isWritable\":true,\"receivedTime\":\"2022-01-27T22:02:21.5545085+00:00\"}", "{\"propertyName\":\"interval\",\"desiredValue\":1,\"reportedValue\":\"8\",\"isWritable\":true,\"receivedTime\":\"2022-01-27T22:02:21.5545111+00:00\"}", "{\"commandName\":\"getRuntimeStats\",\"requestValue\":\"0\",\"responseValue\":{\"diagnosticResults\":{\"machine name\":\"RIDOSBOOK3\",\"os version\":\"Microsoft Windows NT 10.0.22000.0\"}},\"responseStatus\":\"200\",\"receivedTime\":\"2022-01-27T22:02:21.6472217+00:00\"}" ], "startTime": "2022-01-27T22:02:21.390615+00:00", "status": "Failed", "timeout": 0 } ] } ], "endTime": "2022-01-27T22:02:33.532736+00:00", "id": "61ae0d5d-48c5-4d5a-a876-a8e6178908b1", "startTime": "2022-01-27T22:02:20.883797+00:00", "status": "Failed" } cli.knack.cli: Event: Cli.SuccessfulExecute [] cli.knack.cli: Event: Cli.PostExecute [] az_command_data_logger: exit code: 0 cli.__main__: Command ran in 17.468 seconds (init: 0.320, invoke: 17.148) telemetry.save: Save telemetry record of length 3028 in cache