Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

openstack_sd chosing random network-interface #4339

Closed
alfarme opened this Issue Jul 3, 2018 · 0 comments

Comments

Projects
None yet
2 participants
@alfarme
Copy link

alfarme commented Jul 3, 2018

Bug Report

What did you do?

I set up openstack_sd with the below config.

What did you expect to see?
openstack_sd should discover nodes and use consistently use the same network-interface (10.x.x.x) on all servers.

What did you see instead? Under which circumstances?
openstack_sd successfully added all available nodes, but the chose network interface seems arbitrary. it also changes on each discovery run (every 60 seconds)

out of 44 nodes, between 6 and 10 get "re-discovered" with a new private_ip every 60 seconds.

** examples **

before:

before

address="10.150.x.x:9100"
__meta_openstack_instance_flavor="1310a6bc-3b91-47d0-ad50-9a50b9cf743f"
__meta_openstack_instance_id="1cda4a6c-fc4a-4de5-9cc5-13bba908333f"
__meta_openstack_instance_name="xxx"
__meta_openstack_instance_status="ACTIVE"
__meta_openstack_private_ip="10.150.x.x"
metrics_path="/metrics"
scheme="http"
job="openstack_dc1"

after:

after

(marked in yellow are the two servers from the 'before' picture)

address="100.100.x.x:9100"
__meta_openstack_instance_flavor="1310a6bc-3b91-47d0-ad50-9a50b9cf743f"
__meta_openstack_instance_id="1cda4a6c-fc4a-4de5-9cc5-13bba908333f"
__meta_openstack_instance_name="xxx"
__meta_openstack_instance_status="ACTIVE"
__meta_openstack_private_ip="100.100.x.x"
metrics_path="/metrics"
scheme="http"
job="openstack_dc1"

Environment

  • System information:

Linux 4.4.70 x86_64`

  • Prometheus version:

Version 1.8.1
Revision 3a7c51a
Branch HEAD
BuildUser root@ab78fb101775
BuildDate 20171023-15:50:57
GoVersion go1.9.1

  • Prometheus configuration file: (relevant part)
- job_name: 'openstack_dc1'
    scrape_interval: 1m
    scrape_timeout: 10s
    metrics_path: /metrics
    scheme: http
    tls_config:
     insecure_skip_verify: true
    openstack_sd_configs:
    - identity_endpoint: 'https://xxx:5000/v2.0'
      username: <secret>
      userid: <secret>
      password: '<secret>'
      project_name: xxx
      project_id: xxx
      domain_name: default
      domain_id: default
      role: instance
      region: RegionOne
      refresh_interval: 1m
      port: 9100
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.