serverset_sd_configs is only scraping the first configured path

[StephanErb]

I am running Prometheus 0.16.0rc2 and have the following config:

  - job_name: 'aurora'
    metrics_path: /probe
    serverset_sd_configs:
    - servers:
      - 'one.zk.server:2181'
      - 'two.zk.server:2181'
      - 'three.zk.server:2181'
      paths:
      - '/aurora/myrole/test/service-1'
      - '/aurora/myrole/test/service-2'
    relabel_configs:
      ... 

• Current behaviour: Prometheus is only scraping one entry in the paths list. Whatever is listed first is scraped, all others are ignored. Multiple serversets are only scraped if one is specifying a root entry such as /aurora.
• Expected behaviour: All listed paths are scraped.

I believe this code here might be the culprint

[fabxc]

@brian-brazil serversets are your area of expertise I guess :)

[juliusv]

I have zero knowledge about Zookeeper or serversets, but this line stood out to me very obviously:

https://github.com/prometheus/prometheus/blob/master/retrieval/discovery/serverset.go#L90

sd.treeCache = newZookeeperTreeCache(conn, conf.Paths[0], updates)

I guess that's the bug and we just need one tree cache per watched path?

[juliusv]

Without reading the code much further, does this branch work?

https://github.com/prometheus/prometheus/commits/fix-serverset-multiple-paths

See commit 62a6871

[StephanErb]

Looks valid. I will give it a test run and report back here.

[juliusv]

Thanks! It's now out in a PR as well: #1148

[juliusv]

@StephanErb Any news on this one?

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.