service discovery by exec

[genofire]

Proposal

It would be nice that prometheus runs itself an application and use the output from stdout.
Instead of running a cronjob or something else to write contains files and fetch them again by custom service discovery like on file_sd_config.

Idea of configuration

It could be very similar to: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#file_sd_config

scrape_configs:
- job_name: prometheus

  honor_labels: true

  execs_sd_configs:
    - execs:
      - "/opt/sd-mysql.py"
      - "curl https://example.org/prometheus-services.json"
      - "cat single/file.yml"
      refresh_interval: 10m

Transformated example of Blogpost

https://prometheus.io/blog/2015/06/01/advanced-service-discovery/#custom-service-discovery
(untested)

import json

from itertools import groupby
from MySQLdb import connect

cur = connect('localhost', 'root', '', 'test')

# Fetch all rows.
cur.execute("SELECT address, job, zone FROM instances")

tgs = []
# Group all instances by their job and zone values.
for key, vals in groupby(cur.fetchall(), key=lambda r: (r[1], r[2])):
  tgs.append({
    'labels': dict(zip(['job', 'zone'], key)),
    'targets': [t[0] for t in vals],
  })

print(json.dump(tgs))

[eloo]

@genofire
i really like your idea because atm i'm exactly hitting this issue.
for now my plan is to have a sidecar which does this.. but this doesn't feel good so far..

so it would be really cool if this could be considered as one of the next features

[brian-brazil]

There's no plans at all to add something like this, if you need this then cron and file_sd is the way to go.

[eloo]

@brian-brazil okay.. thats sad.. and what about an sd feature for promregator? like consul_sd or the others?

[brian-brazil]

promregator does far more than SD, so even if we begin accepting SDs again in the future I don't see that being one of them.