Move consoles and consoles_libraries dirs out of /etc/prometheus in Docker image

[anioool]

Hi,

It seems that image created by docker hub does not contain consoles and consoles_libraries under /etc/prometheus/ path.

[jimmidyson]

Seems OK to me:

$ docker run -it --rm --entrypoint=ls prom/prometheus:v1.0.0 -l --color=never /etc/prometheus/consoles /etc/prometheus/console_libraries

/etc/prometheus/console_libraries:
total 16
-rw-rw-r--    1 root     root          4683 Jul 18 15:09 menu.lib
-rw-rw-r--    1 root     root          5660 Jul 18 15:09 prom.lib

/etc/prometheus/consoles:
total 92
-rw-rw-r--    1 root     root          1795 Jul 18 15:09 aws_elasticache.html
-rw-rw-r--    1 root     root          2140 Jul 18 15:09 aws_elb.html
-rw-rw-r--    1 root     root          4065 Jul 18 15:09 aws_redshift-cluster.html
-rw-rw-r--    1 root     root          1509 Jul 18 15:09 aws_redshift.html
-rw-rw-r--    1 root     root          1443 Jul 18 15:09 blackbox.html
-rw-rw-r--    1 root     root          3227 Jul 18 15:09 cassandra.html
-rw-rw-r--    1 root     root           616 Jul 18 15:09 cloudwatch.html
-rw-rw-r--    1 root     root          2921 Jul 18 15:09 haproxy-backend.html
-rw-rw-r--    1 root     root          1491 Jul 18 15:09 haproxy-backends.html
-rw-rw-r--    1 root     root          1928 Jul 18 15:09 haproxy-frontend.html
-rw-rw-r--    1 root     root           995 Jul 18 15:09 haproxy-frontends.html
-rw-rw-r--    1 root     root          2975 Jul 18 15:09 haproxy.html
-rw-rw-r--    1 root     root          1024 Jul 18 15:09 index.html.example
-rw-rw-r--    1 root     root          2587 Jul 18 15:09 node-cpu.html
-rw-rw-r--    1 root     root          3445 Jul 18 15:09 node-disk.html
-rw-rw-r--    1 root     root          5597 Jul 18 15:09 node-overview.html
-rw-rw-r--    1 root     root          1412 Jul 18 15:09 node.html
-rw-rw-r--    1 root     root          5511 Jul 18 15:09 prometheus-overview.html
-rw-rw-r--    1 root     root          1547 Jul 18 15:09 prometheus.html
-rw-rw-r--    1 root     root          2457 Jul 18 15:09 snmp-overview.html
-rw-rw-r--    1 root     root          2288 Jul 18 15:09 snmp.html

[jimmidyson]

If you're mounting a dir with config to /etc/prometheus then this will look like these console dirs don't exist (/etc/prometheus will be bind mounted over the top). I did this on Kubernetes, using a configmap for configuration mounted to /etc/prometheus which bind mounted over the top of the existing /etc/prometheus dir.

Solution is to either mount the individual config files to /etc/prometheus/config.yml, etc. or to mount the config dir to a different dir (e.g. /etc/prometheus/config) & update the startup flags to point to the new files (e.g. -config.file=/etc/prometheus/config/prometheus.yml).

I think it might be better if the console* dirs were actually added to /usr/share/prometheus/consoles or something like that instead to ease use of this when running on Docker?

[anioool]

Hey,

You are right. In case when I am using the kubernetes and configMaps for alerts files, configuration the /etc/prometheus path is cleaned by volume attaching mechanism.

I will try to move the configuration to the different folder first as you suggested.

[jimmidyson]

Going to reopen & change this to consider moving console dirs out of /etc/prometheus in Docker image so we don't get other reports like this.

[jimmidyson]

@sdurrheimer @juliusv What do you think?

[sdurrheimer]

I think it is too late for this kind of change. Changing it now will bother people already used to this path, which would lead to another kind of report "why my consoles are now working ?" or "why changing this now ?".

I like the fact that all those conf files are below a single folder /etc/prometheus, like others packages/software would do.

We have enough config flags to change this behavior if users really want to. If they need to only share a config file with a volume, they can either use a totally different path or directly share the config file as a volume target path.

[jimmidyson]

I think it is too late for this kind of change.

Yeah thought it might be, it's just really annoying when you want to mount config files dir in Docker container and have console templates working. It means most users (especially on Kubernetes if they follow @fabxc's guide using configmap) will have to change the defaiult flags in the Docker image.

Thinking about it, if users are using the default flags from the image, then moving the console files to different dir & updating flags in the next version of the image should be transparent.

I like the fact that all those conf files are below a single folder /etc/prometheus

But these aren't really conf files are they? Not what I would consider conf files anyway.

directly share the config file as a volume target path.

Unless you have multiple rules files as I would expect most "real" users to have.

Anyway, understand if this can't be changed for fear of breaking existing users so will leave it with you.

[fabxc]

I think our 1.0 guarantees about deployment stability prohibits these changes right now. We can and should reconsider if we ever talk about 2.0.

Creating a 2.x milestone to collect all similar issues.

[brian-brazil]

I think our 1.0 guarantees permit this. We say the example consoles are not covered, and nor are flags. There's no mention of our Docker container, so I'd say it's not included.

[fabxc]

I don't want to analyze what we wrote there like a legal contract, looking for loop holes. From a user perspective, my intuitive understanding would be that this is covered by the stable deployment guarantees.

Then again, consoles are really not my area. So your call.

[brian-brazil]

We promise that parts of Prometheus itself is stable, that says nothing about our tarballs or other distribution mechanisms. Would you consider it okay to change our tarball layout within 1.0?

[fabxc]

No, I wanted to do a change to the tarball about 1 year ago and back then you and Julius already said it's too breaking to people.

[brian-brazil]

Thinking on it, can we drop in some symlinks?

[jimmidyson]

That sounds good to me.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.