feat: explicit gce interface ipv4 address metadata

[jfreeland]

Always add GCE interface IPv4 address metadata for all interfaces. In cases where you have multiple interfaces, this can then be used, for instance, to rewrite the address label to target metrics on a separate interface.

This utilizes the Google Compute API's instances metadata, https://cloud.google.com/compute/docs/reference/rest/v1/instances, using the networkInterfaces[].networkIP field which according to the documentation will return an IPv4 address.

networkInterfaces[].networkIP | string | An IPv4 internal IP address to assign to the instance for this network interface. If not specified by the user, an unused internal IP is assigned by the system.
networkInterfaces[].ipv6Address | string | [Output Only] An IPv6 internal network address for this network interface.

In GCP a TCP load balancer (and probably others) can only forward traffic to nic0. A snippet from https://cloud.google.com/load-balancing/docs/network/setting-up-network-backend-service#configuring_the_load_balancer:

• "If you are using an image provided by Compute Engine, your instances are automatically configured to handle this IP address. If you are using any other image, you must configure this address as an alias on nic0 or as a loopback on each instance."

In some cases this may mean that you need to move functions that historically ran on nic0 to another interface. For instance, some network software vendors typically run their management interface, where one scrapes metrics and manages the machine on nic0, however in GCP this requires moving the management interface to nic1 or some other interface. Take for instance https://github.com/memes/terraform-google-f5-bigip/blob/main/modules/metadata/files/multiNicMgmtSwap.sh.

Testing this requires that you set up 2 instances in GCE:

• a node named <node_prefix>-N that has 2 interfaces and is exposing metrics on the relevant IP and port, e.g. node_exporter --web.listen-address="<management_ip>:9090"
  • the primary interface, e.g. nic0, should be in some network that is not your default network
  • the second interface, e.g. nic1, should be in some 'management' subnet
• a prometheus node with an interface in the management subnet where you can download, build, and run prometheus from this branch, that (ideally) is running with a service account that has access to the GCP metadata service.

I used the following prometheus.conf and this worked as intended.

global:
  scrape_interval:     15s
  evaluation_interval: 30s

scrape_configs:
- job_name: prometheus

  honor_labels: true

  gce_sd_configs:
  - project: <my project>
    zone: us-central1-a
    filter: "name:<node_prefix>-*"

  relabel_configs:
    - source_labels: [__meta_gce_interface_ipv4_nic1]
      target_label: __address__
      replacement: "${1}:9090"

Related to #7406.

[SuperQ]

The GCE networkInterfaces JSON includes a name attribute, we could possibly us that instead of assuming ethX

[jfreeland]

The GCE networkInterfaces JSON includes a name attribute, we could possibly us that instead of assuming ethX

🤦 . updated.

[roidelapluie]

Thanks for your PR. For legal reasons, we require that all commits are signed with a DCO before we can merge them. See this blog post for considerations around this.

This means that the last line of your commit message should read like:

Signed-Off-By: Your Name <your@email.address>

If you are using GitHub through the web interface, it's quickest to close this PR and open a new one with the appropriate line.

If you are using Git on the command line, it is probably quickest to amend and force push. You can do that with

git commit --amend --signoff
git push -f $remote $remote_branch_for_pr

As always, be careful when force-pushing.

[jfreeland]

Thanks for your PR. For legal reasons, we require that all commits are signed with a DCO before we can merge them. See this blog post for considerations around this.

This means that the last line of your commit message should read like:

Signed-Off-By: Your Name <your@email.address>

If you are using GitHub through the web interface, it's quickest to close this PR and open a new one with the appropriate line.

If you are using Git on the command line, it is probably quickest to amend and force push. You can do that with

git commit --amend --signoff
git push -f $remote $remote_branch_for_pr

As always, be careful when force-pushing.

Sorry. Updated.

[roidelapluie]

I like this change. Does the api guarantees we would only get ipv4?

[jfreeland]

I like this change. Does the api guarantees we would only get ipv4?

We are using the networkIP field from https://cloud.google.com/compute/docs/reference/rest/v1/instances. There is a separate field for IPv6 address.

networkInterfaces[].networkIP | string | An IPv4 internal IP address to assign to the instance for this network interface. If not specified by the user, an unused internal IP is assigned by the system.
networkInterfaces[].ipv6Address | string | [Output Only] An IPv6 internal network address for this network interface.

{
  "id": string,
  "creationTimestamp": string,
  "name": string,
...
  },
...
  "networkInterfaces": [
    {
...
      "networkIP": string,
      "ipv6Address": string,

[roidelapluie]

I will let a few more hours for @SuperQ to review, otherwise I will merge this tomorrow.

Many thanks.

[jfreeland]

I like this change. Does the api guarantees we would only get ipv4?

We are using the networkIP field from https://cloud.google.com/compute/docs/reference/rest/v1/instances. There is a separate field for IPv6 address.

Description updated for completeness.

[SuperQ]

LGTM, nice

[roidelapluie]

Thanks! One last nit.

[roidelapluie]

(I missed it in my last review, sorry)

[roidelapluie]

Thanks!