Skip to content

Prometheus configuration auto-reloading#355

Merged
jchristgit merged 6 commits into
mainfrom
jb3/kubernetes/266-prometheus-config-reload
Jun 10, 2024
Merged

Prometheus configuration auto-reloading#355
jchristgit merged 6 commits into
mainfrom
jb3/kubernetes/266-prometheus-config-reload

Conversation

@jb3
Copy link
Copy Markdown
Member

@jb3 jb3 commented Jun 9, 2024

I have created a repository, owl-corp/inotify-base, which is a
Debian slim image with some added niceities to trigger a script run on changes
to a given path.

We then have a script in this repository to reload Prometheus using the
management API and notify in the Discord channel if this has succeeded or not.

I have also enabled scraping of the Prometheus server and also added an alert
for failed config reloads to integrate it into our existing alerting pipeline.

Makes progress on #266, there is still no auto-deployment of content from this repository.

@jb3
Copy link
Copy Markdown
Member Author

jb3 commented Jun 9, 2024

image

@jb3 jb3 added component: networking An issue relating to a host networking (e.g. DNS, WireGuard, SSH) component: monitoring An issue relating to a monitoring component (e.g. Prometheus, Grafana) webscale group: kubernetes Issues and pull requests related to the Kubernetes setup labels Jun 9, 2024
Comment thread kubernetes/namespaces/monitoring/alerts/alerts.d/prometheus.yaml Outdated
Comment thread kubernetes/namespaces/monitoring/alerts/alerts.d/prometheus.yaml Outdated
Comment thread kubernetes/namespaces/monitoring/prometheus/deployment.yaml
Comment thread kubernetes/namespaces/monitoring/prometheus/reloader-script.yaml Outdated
Comment thread kubernetes/namespaces/monitoring/prometheus/reloader-script.yaml Outdated
Comment thread kubernetes/namespaces/monitoring/prometheus/reloader-script.yaml Outdated
Comment thread kubernetes/namespaces/monitoring/prometheus/reloader-script.yaml Outdated
@jchristgit
Copy link
Copy Markdown
Contributor

jchristgit commented Jun 10, 2024 via email

@jb3 jb3 force-pushed the jb3/kubernetes/266-prometheus-config-reload branch from 3f2e65d to 990cefd Compare June 10, 2024 13:14
@jb3
Copy link
Copy Markdown
Member Author

jb3 commented Jun 10, 2024

All review comments have been addressed in the force push.

@jb3 jb3 requested a review from jchristgit June 10, 2024 13:15
@jb3 jb3 force-pushed the jb3/kubernetes/266-prometheus-config-reload branch from 990cefd to 499bb71 Compare June 10, 2024 13:24
@jb3 jb3 force-pushed the jb3/kubernetes/266-prometheus-config-reload branch from 499bb71 to 7e76e2a Compare June 10, 2024 13:33
@jchristgit jchristgit merged commit 3b4613e into main Jun 10, 2024
@jchristgit jchristgit deleted the jb3/kubernetes/266-prometheus-config-reload branch June 10, 2024 15:36
@jchristgit
Copy link
Copy Markdown
Contributor

Thanks!

1 similar comment
@github-actions
Copy link
Copy Markdown

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component: monitoring An issue relating to a monitoring component (e.g. Prometheus, Grafana) component: networking An issue relating to a host networking (e.g. DNS, WireGuard, SSH) group: kubernetes Issues and pull requests related to the Kubernetes setup webscale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants