Sanity when working with relative paths in config

[msiebuhr]

What did you do?
Inserted a relative path in prometheus.yml (see #2246 for some context).

scrape_configs:
  - job_name: 'mdns-sd'
    file_sd_configs:
      - files:
        - "./mdns-sd.json"

What did you expect to see?
That it resolved relative to the path of prometheus.yml (element of least surprise and all), imported the service-definitions of the file and so on.

What did you see instead? Under which circumstances?
Nothing. No errors, no internal metrics to indicate no files were found/imported.

Given I was working in an unfamiliar environment (FreeBSD jail; I'm a linux-guy), I couldn't readily find a way to find out what the working directory of the running prometheus server was (/proc/$pid/cwd doesn't exist, nor strace. Some poking at ktrace and dtrace didn't look promising without investing a lot of time in learning them, ...)

Though I did expect the menu status → Runtime & Build information to show the CWD, it didn't.

Environment

• System information:

  FreeBSD 10.3-STABLE amd64

• Prometheus version:

  build user:
  build date:
  go version: go1.7.3

[msiebuhr]

For the record, I ended up just banging in the absolute path in the config. Ugly, but working.

[fabxc]

Nothing. No errors, no internal metrics to indicate no files were found/imported.

That's intended behavior. We expect rule and target files to not necessarily exist at startup time.

Generally, just using absolute paths in a production config is generally more sane but might indeed be ugly for longer paths.

For rule files an API endpoint (and corresponding UI view) showing what files were discovered were could help. But exposing equally detailed runtime information on file SD etc. would be quite the maintenance burden.

[msiebuhr]

Right. I can see how it'll be difficult to expose meaningful info in such corner-cases.

Wrt. the metrics, I did expect something to show up on how many targets each service-discovery mechanism coughs up with (might also be relevant in comparison w. number of scraped services).

Another brainfart - expand relative paths to absolute ones when rendering the config.file in the UI. I have a hunch that'd also be a lot of work for very little gain, but not much more.

[brian-brazil]

I'm not sure we should be munging paths in the config. I can imagine that catching users out where the base of the path is dynamic, and they want to do a straight config copy elsewhere.

[powerman]

Guys, do not go crazy, just resolve relative paths in config as relative to config file location - like everybody else does. Anything else is counter-intuitive and doesn't makes much sense.

Another option is ban relative paths at all. But resolving relative paths in config as relative to directory where prometheus binary was executed - is just plain wrong because may result in undefined behaviour depending on subtle environment change.

But, if you insist on current behaviour, then lifehack is:

cd /etc/prometheus && prometheus -config.file ./prometheus.yml …

[fabxc]

So what's missing to resolve this is to apply the resolution of files paths relative to the config to file SD? (https://github.com/prometheus/prometheus/blob/dev-2.0/config/config.go#L248)
Seems like merely an oversight in the past but by now probably a breaking change.

[brian-brazil]

That sounds about right to me. I think this should go in 2.0.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.