Make promtool better for CI with syntax-only config check

[glightfoot]

Proposal

Use case. Why is this important?

We validate our rules with promtool in CI before deploying, it would be nice to be able to check for config file syntax validity without having to deploy all secrets, certs, etc.

Bug Report

What did you do?
promtool check config prometheus.yaml

What did you expect to see?

./promtool check config --syntax-only prometheus.yaml                                                                                                                                                                                
Checking prometheus.yaml
  SUCCESS: prometheus.yaml is valid prometheus config file syntax
  SUCCESS: 0 rule files found

What did you see instead? Under which circumstances?

promtool check config prometheus.yaml
Checking prometheus.yaml
  FAILED: error checking bearer token file "/var/run/secrets/kubernetes.io/serviceaccount/token": stat /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory

Environment
CI (explicitly does not have the token and certs present in the deployment)

• System information:

  Linux 3.10.0-514.26.2.el7.x86_64 x86_64

• Prometheus version:

  2.7.1

[roidelapluie]

I do not agree with this. Those files are part of the configuration and I want promtool to check that it can actually read them.

[glightfoot]

@roidelapluie - this will not change that behavior. I agree that that is an important check to have available. This will only add the ability to do the tests without those files as a new feature.

promtool check config prometheus.yaml - still works as it does now
promtool check config --syntax-only prometheus.yaml - only checks syntax

[geekodour]

@codesome can I add this with #21 ?

EDIT: oh there's already a PR. I missed it somehow.

[glightfoot]

Is there any way we can move this along? I had syntax bug get checked in that we could have prevented with this...