Added init container which applies config file once by setting --watch-interval equals to 0

[zsuzhengdu]

Fix #3061

Depends on a new Thanos release that includes thanos-io/thanos#3994

[coderanger]

At the point that the init container runs, the primary containers have not started. So prometheus won't actually be up yet for this to trigger a reload on.

[zsuzhengdu]

Good call. Updated to skip making reload request to :locahost:9090/-/reload when --watch-interval==0; please see thanos-io/thanos#4032
Also skip starting metrics server in cmd/prometheus-config-reloader/main.go when --watch-interval==0.

[zsuzhengdu]

@coderanger @brancz

Would you please help to review the changes? Thanks!

[simonpasquier]

can you update the description of the --watch-interval flag to explain that when 0, the program runs only once and exits.

[simonpasquier]

A few suggestions but great work!

[fpetkovski]

Thanks for the contribution, the PR looks quite good so far.

As one last improvement, could you also add tests to the prometheus and alertmanager test suites to make sure the init container is added to the respective statefulsets?

[zsuzhengdu]

@fpetkovski Are these changes good enough and would you please give another review?

[fpetkovski]

Hey @zsuzhengdu, this looks good to me now. I also tested the changes locally and they look solid. Thanks for the contribution!

[zsuzhengdu]

Thank you for the review and local try out, @fpetkovski!

[paulfantom]

lgtm from my side, leaving final say to @simonpasquier

[simonpasquier]

Thanks a lot for your perseverance and sorry about the delayed review from my side! Also thanks to @fpetkovski for taking over :)

I have 2 last comments:

• I don't think we need to add the init container to the Alertmanager statefulset because the config reloader doesn't substitute environment variables in the Alertmanager config file.
• It would be good to mention the new init-config-reloader container in the API documentation. In practice, this comment should be expanded to match what's already documented for regular containers generated by the operator.

[simonpasquier]

👍 thanks a lot! 🎉

[simonpasquier]

@zsuzhengdu it took longer than expected but finally it's merged! Thanks a lot for your patience 🙇