additional_label's are missing in custom metrics of EXTERNAL probe

[skuethe]

First off, thank you for providing this very useful tool! :)

Describe the bug
It seems that any additional_label defined inside an EXTERNAL probe only applies to the build-in metrics (success, total and latency). Any custom metrics returned by the external probe does not have these labels set.

Cloudprober Version
Tested both v0.11.2 and latest master (as of today: 2021-08-25)

To Reproduce
I have an external probe which calls Azure API and returns additional data about the remaining API rate limits (which are send by Azure in the HTTP Header). For example Microsoft_Compute_LowCostGet30Min.
These custom metrics are exposed and scraped successfully, but the labels are missing. My probe configuration looks like this:

probe {
  name: "azure_vms"
  type: EXTERNAL

  [...]

  external_probe {
    mode: ONCE
    command: "azureapiprobe --vmname @target@ --rgname @target.label.rgname@"
  }
  additional_label {
    key: "rgname"
    value: "@target.label.rgname@"
  }
  additional_label {
    key: "os"
    value: "@target.label.os@"
  }
}

Example output:

## These are fine:

#TYPE cloudprober_success counter
cloudprober_success{ptype="external",probe="azure_vms",dst="TARGETNAME",rgname=RGNAME,os="linux"} 2 1629904383080
#TYPE cloudprober_total counter
cloudprober_total{ptype="external",probe="azure_vms",dst="TARGETNAME",rgname=RGNAME,os="linux"} 2 1629904383080
#TYPE cloudprober_latency counter
cloudprober_latency{ptype="external",probe="azure_vms",dst="TARGETNAME",rgname=RGNAME,os="linux"} 896401.042 1629904383080

## This is missing the `rgname` and `os` labels:

#TYPE cloudprober_ratelimit_Microsoft_Compute_LowCostGet30Min gauge
cloudprober_ratelimit_Microsoft_Compute_LowCostGet30Min{ptype="external",probe="azure_vms",dst="TARGETNAME"} 31779.000 1629904383080

[manugarg]

Above change should fix this issue. You can test it with the "master" tag docker image. It will be released with the v0.11.3 release.

[manugarg]

Closing this issue now. Please feel free to reopen if it doesn't work. You can track the release here:
https://github.com/google/cloudprober/milestone/11

[skuethe]

Works like a charm. Thank you @manugarg!