Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[prometheus][grafana][prometheus-crd][flant-integration] Add grafana alerts notification channels and migrate to proxy (server) data sources from direct (browser). #402

Merged
merged 1 commit into from Dec 24, 2021
Merged

[prometheus][grafana][prometheus-crd][flant-integration] Add grafana alerts notification channels and migrate to proxy (server) data sources from direct (browser). #402

merged 1 commit into from Dec 24, 2021

Conversation

name212
Copy link
Member

@name212 name212 commented Dec 14, 2021 •
edited

Description

Add grafana notification channel support.

Why we need it and what problem does it solve?

Users need to add alerts in Grafana for non-Prometheus data sources like Clickhouse. They can save dashboards with notifications to the cluster as GrafanaDashboardDefinitions, but without notification channels, it will not work.

Changelog entries

module: prometheus-crd
type: feature
description: Add GrafanaAlertsChannel CRD.
note: Support only prometheus alert manager notification channel
---
module: prometheus
type: feature
description: Provisioning alerts channels from CRD's to grafana via new secret. Migrate to direct datasources.
note: |
  Grafana will be restarted.
  Now grafana using direct (proxy) type for deckhouse datasources (main, longterm, uncached), because direct(browse) datasources type is depreated now. And alerts don't work with direct data sources.
  Provisioning datasources from secret instead configmap. Deckhouse datasources need client certificates to connect to  prometheus or trickter. Old cm leave to prevent mount error while terminating.
---
module: flant-integration
type: feature
description: Add madison-proxy notification channel to send alert from grafana to madison via proxy and show them in Polk 
note: |
  Add rewrite rule to madison-proxy from /api/v1/alerts url to madison url, because grafana always send notification to this url.

@name212 name212 force-pushed the 245-nm-graphana-notification-channels branch from cf05cc8 to 2034774 Compare December 14, 2021 10:04
@nabokihms nabokihms added the area/monitoring Pull requests that update monitoring modules label Dec 14, 2021
@EvgenySamoylov EvgenySamoylov added this to the v1.30.0 milestone Dec 16, 2021
@EvgenySamoylov
Copy link
Contributor

Fixes #245

@name212 name212 force-pushed the 245-nm-graphana-notification-channels branch 2 times, most recently from f241560 to d0e0309 Compare December 20, 2021 06:19
@name212 name212 changed the title 245 nm graphana notification channels [prometheus][grafana][prometheus-crd][flant-integration] Add grafana alerts notification channels and migrate to proxy (server) data sources from direct (browser). Dec 20, 2021
@name212 name212 linked an issue Dec 20, 2021 that may be closed by this pull request
[prometheus] Grafana notification channels provisioning #245
Closed
2 tasks
@name212 name212 marked this pull request as ready for review December 20, 2021 06:42
@name212 name212 force-pushed the 245-nm-graphana-notification-channels branch from 8d2ce1b to 63f64d2 Compare December 22, 2021 01:07
@name212 @nabokihms
[prometheus][grafana][prometheus-crd] Add grafana alerts notification…
0e31aa5 
… channels and migrate to proxy (server) data sources from direct (browser)

[flant-integration] Add default madison proxy grafana notification channel

Co-authored-by: Maksim Nabokikh <maksim.nabokikh@flant.com>
@name212 name212 force-pushed the 245-nm-graphana-notification-channels branch from 63f64d2 to 0e31aa5 Compare December 22, 2021 11:25
@nabokihms nabokihms merged commit 5998944 into main Dec 24, 2021
@nabokihms nabokihms deleted the 245-nm-graphana-notification-channels branch December 24, 2021 10:19
@deckhouse-BOaTswain deckhouse-BOaTswain mentioned this pull request Dec 24, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nabokihms nabokihms nabokihms approved these changes

Assignees

@name212 name212

Labels
area/monitoring Pull requests that update monitoring modules
Projects
None yet
Milestone
v1.30.0
Development

Successfully merging this pull request may close these issues.

[prometheus] Grafana notification channels provisioning
3 participants
@name212 @EvgenySamoylov @nabokihms