[metricsCollector] Custom Prometheus metrics in InsightsMetrics do not have Computer/Device name set

[dave-read]

Expected Behavior

Custom Prometheus metrics posted directly to Log Analytics InsightsMetrics table should populate the device name (or user configurable source name) in the Computer (or some other) field of the InsightsMetrics table.

Current Behavior

Custom Prometheus metrics are populated, but the Computer field is empty and value of edge_device is not visible in any other fields.

Steps to Reproduce

Provide a detailed set of steps to reproduce the bug.

1. Configure the metrics-collector module as documented to send metrics directly to Log Analytics Workspace
2. Enable custom metrics in a custom module. In this case python using client_python
3. Generate metrics
4. Query the InsightsMetrics table and note that Computer field is not populated.

Note: the system level metrics from IOT Edge system modules do include within the Tags field the edge_device property that includes the device name.

Context (Environment)

IOT Edge deployed on Jetson Nano

Output of iotedge check

Configuration checks (aziot-identity-service)
---------------------------------------------
√ keyd configuration is well-formed - OK
√ certd configuration is well-formed - OK
√ tpmd configuration is well-formed - OK
√ identityd configuration is well-formed - OK
√ daemon configurations up-to-date with config.toml - OK
√ identityd config toml file specifies a valid hostname - OK
√ aziot-identity-service package is up-to-date - OK
√ host time is close to reference time - OK
√ preloaded certificates are valid - OK
√ keyd is running - OK
√ certd is running - OK
√ identityd is running - OK
√ read all preloaded certificates from the Certificates Service - OK
√ read all preloaded key pairs from the Keys Service - OK
√ ensure all preloaded certificates match preloaded private keys with the same ID - OK

Connectivity checks (aziot-identity-service)
--------------------------------------------
√ host can connect to and perform TLS handshake with iothub AMQP port - OK
√ host can connect to and perform TLS handshake with iothub HTTPS / WebSockets port - OK
√ host can connect to and perform TLS handshake with iothub MQTT port - OK

Configuration checks
--------------------
√ aziot-edged configuration is well-formed - OK
√ configuration up-to-date with config.toml - OK
√ container engine is installed and functional - OK
√ configuration has correct URIs for daemon mgmt endpoint - OK
√ aziot-edge package is up-to-date - OK
√ container time is close to host time - OK
‼ DNS server - Warning
    Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub.
    Please see https://aka.ms/iotedge-prod-checklist-dns for best practices.
    You can ignore this warning if you are setting DNS server per module in the Edge deployment.
‼ production readiness: logs policy - Warning
    Container engine is not configured to rotate module logs which may cause it run out of disk space.
    Please see https://aka.ms/iotedge-prod-checklist-logs for best practices.
    You can ignore this warning if you are setting log policy per module in the Edge deployment.
‼ production readiness: Edge Agent's storage directory is persisted on the host filesystem - Warning
    The edgeAgent module is not configured to persist its /tmp/edgeAgent directory on the host filesystem.
    Data might be lost if the module is deleted or updated.
    Please see https://aka.ms/iotedge-storage-host for best practices.
‼ production readiness: Edge Hub's storage directory is persisted on the host filesystem - Warning
    The edgeHub module is not configured to persist its /tmp/edgeHub directory on the host filesystem.
    Data might be lost if the module is deleted or updated.
    Please see https://aka.ms/iotedge-storage-host for best practices.
√ Agent image is valid and can be pulled from upstream - OK
√ proxy settings are consistent in aziot-edged, aziot-identityd, moby daemon and config.toml - OK

Connectivity checks
-------------------
√ container on the default network can connect to upstream  AMQP port - OK
√ container on the default network can connect to upstream HTTPS / WebSockets port - OK
√ container on the default network can connect to upstream MQTT port - OK
√ container on the IoT Edge module network can connect to upstream AMQP port - OK
√ container on the IoT Edge module network can connect to upstream HTTPS / WebSockets port - OK
√ container on the IoT Edge module network can connect to upstream MQTT port - OK
32 check(s) succeeded.
4 check(s) raised warnings. Re-run with --verbose for more details.

Device Information

• Host OS: Ubuntu 18.04 (JetPack 4.6)
• Architecture: arm64
• Container OS: Linux containers - Docker

Runtime Versions

• aziot-edged [run iotedge version]: iotedge 1.2.7
• Edge Agent [image tag (e.g. 1.0.0)]: mcr.microsoft.com/azureiotedge-agent:1.2
• Edge Hub [image tag (e.g. 1.0.0)]: mcr.microsoft.com/azureiotedge-hub:1.2
• Docker/Moby [run docker version]: 20.10.7

Logs

N/A

[nlcamp]

@dave-read Thanks for reporting this. Your suggestion makes sense. We've reached out to the Azure Monitor team to discuss the feasibility of making this change given the current REST API. We'll get back to you soon.

[nlcamp]

Noting here that this issue is closely related to #6081

[dave-read]

@nlcamp Thanks, as a workaround, we've been pulling the device ID from the IOTEDGE_DEVICEID container environment variable and adding it as an extra label in our metrics. Not ideal, but it's working.

[nlcamp]

@dave-read - Glad to hear you've found a workaround. We're still discussing this internally and hope to populate the 'computer' field in a future release. It would help us prioritize this request If you could submit it as a suggestion at https://aka.ms/iotedge-suggestion. Please tag it with the "Monitoring" label.

With that said I'll close this issue since there's a workaround and this is a feature request that we're considering for a future release.

[dave-read]

@nlcamp Thanks for following up. I created the suggestion tagged with "Monitoring". https://feedback.azure.com/d365community/idea/7c40e8ca-1aa0-ec11-a81c-0022484ee92d