mgr/dashboard: Prometheus integration #25309

Devp00l · 2018-11-28T17:56:44Z

The backend is now capable of receiving alert notifications from
the Prometheus alertmanager and it can get all alerts with all kinds of
parameters from the API of the same.

Now Prometheus alerts can be found in "Cluster > Alerts". Incoming
notifications can be seen as usual in the notifications popover.

To clarify:
Prometheus alerts are received from the alertmanager API.
Prometheus alert notification are send from the alertmanager to the
backend receiver. An alert notification can have multiple alerts, but
these alerts differ from the prometheus alerts.

To clarify that, I've added some models and services.

If one of the methods to get alerts contains changes the user will be
notified.

In addition to that the following new things were implemented:

It's possible to style values inside the key value table based on the
values.
Notifications and alerts show an application icon now, that gives a hint
about their origin.

Fixes: https://tracker.ceph.com/issues/37950
Fixes: https://tracker.ceph.com/issues/37951
Fixes: https://tracker.ceph.com/issues/36721
Signed-off-by: Stephan Müller smueller@suse.com

References tracker ticket
Updates documentation if necessary
Includes tests for new functionality or reproducer for bug

Some impressions:

doc/mgr/dashboard.rst

s0nea · 2018-11-29T09:04:13Z

src/pybind/mgr/dashboard/HACKING.rst

+How to use a custom API endpoint in a RESTController?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you have no access restriction you can use ``@Endpoint`` but if you have you


Suggested change

If you have no access restriction you can use ``@Endpoint`` but if you have you

If you don't have any access restriction you can use ``@Endpoint``. But if you do

s0nea · 2018-11-29T09:07:39Z

src/pybind/mgr/dashboard/HACKING.rst

+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you have no access restriction you can use ``@Endpoint`` but if you have you
+use instead ``@RESTController.Collection`` or ``@RESTController.Resource``, if


Suggested change

use instead ``@RESTController.Collection`` or ``@RESTController.Resource``, if

and ``RESOURCE_ID`` is set you'll have to use ``@RESTController.Collection`` or ``@RESTController.Resource`` instead.

You only can use @RESTController.Resource if RESOURCE_ID is set, but you can use @RESTController.Collection without having the RESOURCE_ID set.

s0nea · 2018-11-29T09:07:59Z

src/pybind/mgr/dashboard/HACKING.rst

+
+If you have no access restriction you can use ``@Endpoint`` but if you have you
+use instead ``@RESTController.Collection`` or ``@RESTController.Resource``, if
+``RESOURCE_ID`` is set. ``@Endpoint`` will fail here, as it does not know which


Suggested change

``RESOURCE_ID`` is set. ``@Endpoint`` will fail here, as it does not know which

Otherwise ``@Endpoint`` will fail here, as it does not know which

s0nea · 2018-11-29T09:15:50Z

src/pybind/mgr/dashboard/frontend/src/app/shared/models/cd-notification.ts

 import { NotificationType } from '../enum/notification-type.enum';

+export class CdNotificationConfig {


Would it possibly make sense to move all the changes related to CdNotificationConfig to a separate (pre-)PR? Because this one is quite long already.

LenzGr · 2018-12-06T13:13:22Z

doc/mgr/dashboard.rst

+#. Use the Prometheus Alertmanager API.
+
+#. Use both sources simultaneously.
+


What are the benefits/differences between these methods here?
When would one want to use one or the other?
Please elaborate on the rationale for having these two different methods.

LenzGr · 2018-12-06T13:14:27Z

doc/mgr/dashboard.rst

@@ -366,6 +366,63 @@ To enable SSO::

  $ ceph dashboard sso enable saml2

+Enabling Prometheus alerting
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+


Please add a short note about what the Prometheus Alert manager is all about, including an URL that points to more information, especially how to install and configure it.

p-se · 2018-12-06T17:03:40Z

src/pybind/mgr/dashboard/controllers/prometheus.py

+from ..settings import Settings
+
+
+# The receiver is needed in order to receive alert notifications (reports)


Can you please move this documentation to the PrometheusReceiver class?

src/pybind/mgr/dashboard/controllers/prometheus.py

Devp00l · 2018-12-07T15:19:32Z

Addressed all comments

...nd/mgr/dashboard/frontend/src/app/core/navigation/notifications/notifications.component.html

...mgr/dashboard/frontend/src/app/core/navigation/notifications/notifications.component.spec.ts

src/pybind/mgr/dashboard/frontend/src/app/shared/api/prometheus.service.spec.ts

src/pybind/mgr/dashboard/frontend/src/app/shared/api/prometheus.service.ts

src/pybind/mgr/dashboard/frontend/src/app/shared/services/prometheus-alert-formatter.ts

src/pybind/mgr/dashboard/frontend/src/app/shared/services/prometheus-alert.service.spec.ts

...ybind/mgr/dashboard/frontend/src/app/shared/services/prometheus-notification.service.spec.ts

alfonsomthd

Tested locally: it works as expected.

Devp00l · 2019-01-25T12:40:28Z

@p-na Sorry I miss understood you I thought you mend squashing all commits into one. Will do that.

Devp00l · 2019-01-25T13:24:11Z

Addressed all comments. Put interval outside of angular so that the e2e test will pass again.

Requested changes have been resolved.

doc/mgr/dashboard.rst

src/pybind/mgr/dashboard/controllers/prometheus.py

src/pybind/mgr/dashboard/frontend/src/app/shared/services/notification.service.ts

votdev · 2019-01-30T12:07:56Z

...ashboard/frontend/src/app/shared/datatable/table-key-value/table-key-value.component.spec.ts

+
+    it('should have the following column set up if customCss is defined', () => {
+      component.customCss = {
+        'answer-of-everything': 42


src/pybind/mgr/dashboard/frontend/src/app/shared/services/notification.service.ts

src/pybind/mgr/dashboard/frontend/src/app/shared/services/prometheus-alert-formatter.ts

Devp00l · 2019-01-30T14:49:54Z

@votdev Addressed all your comments 👍

The backend is now capable of receiving alert notifications from the Prometheus alertmanager and it can get all alerts with all kinds of parameters from the API of the same. In the frontend Prometheus alerts can be found in "Cluster > Alerts". Incoming notifications can be seen as usual in the notifications popover. To clarify: Prometheus alerts are received from the alertmanager API. Prometheus alert notification are send from the alertmanager to the backend receiver. An alert notification can have multiple alerts, but these alerts differ from the prometheus alerts. To clarify that, I've added some models and services. If one of the methods to get alerts contains changes the user will be notified. The documentation explains how to configure the alertmanager to use the dashboard receiver and how to connect the use of the alertmanager API. Further it explains where to find the alerts and what happens if they are configured and something is happening. Fixes: https://tracker.ceph.com/issues/36721 Signed-off-by: Stephan Müller <smueller@suse.com>

Now it's possible to style values inside the KV-table based on the values. Fixes: https://tracker.ceph.com/issues/37951 Signed-off-by: Stephan Müller <smueller@suse.com>

Now notifications and alerts show an application icon, that gives a hint about their origin. Fixes: https://tracker.ceph.com/issues/37950 Signed-off-by: Stephan Müller <smueller@suse.com>

Signed-off-by: Stephan Müller <smueller@suse.com>

Devp00l · 2019-01-31T09:47:02Z

jenkins test dashboard

Devp00l · 2019-01-31T11:57:27Z

jenkins test dashboard

s0nea

LGTM

src/pybind/mgr/dashboard/controllers/prometheus.py

callithea · 2019-02-04T07:51:37Z

http://pulpito.ceph.com/laura-2019-02-01_11:52:18-rados:mgr-wip-lpaduano-testing-prometheus-alerts-distro-basic-smithi/

There's one test failure (which does not seem to be related to this PR): Test failure: setUpClass (tasks.mgr.dashboard.test_rgw.RgwUserKeyTest)

Devp00l added the dashboard label Nov 28, 2018

Devp00l requested review from LenzGr, tspmelo, votdev, p-se and ricardoasmarques November 28, 2018 17:56

s0nea reviewed Nov 29, 2018

View reviewed changes

s0nea added the feature label Nov 29, 2018

s0nea reviewed Nov 29, 2018

View reviewed changes

Devp00l force-pushed the wip-prometheus-alerting branch 4 times, most recently from f46b72b to bc78ee0 Compare November 29, 2018 12:13

Devp00l changed the title ~~mgr/dashboard: Prometheus integration~~ [After #25327, #25325] mgr/dashboard: Prometheus integration Nov 29, 2018

Devp00l force-pushed the wip-prometheus-alerting branch 3 times, most recently from 6974d1c to 5701d56 Compare December 4, 2018 14:46

LenzGr added the needs-review label Dec 5, 2018

LenzGr reviewed Dec 6, 2018

View reviewed changes

p-se reviewed Dec 6, 2018

View reviewed changes

Devp00l force-pushed the wip-prometheus-alerting branch from 5701d56 to 5402a3c Compare December 7, 2018 15:15

Devp00l changed the title ~~[After #25327, #25325] mgr/dashboard: Prometheus integration~~ [After #25325] mgr/dashboard: Prometheus integration Dec 11, 2018

callithea added the needs-rebase label Dec 12, 2018

Devp00l force-pushed the wip-prometheus-alerting branch from 5402a3c to ac1fdcd Compare December 13, 2018 12:30

Devp00l changed the title ~~[After #25325] mgr/dashboard: Prometheus integration~~ mgr/dashboard: Prometheus integration Dec 13, 2018

Devp00l removed the needs-rebase label Dec 13, 2018

tspmelo previously requested changes Dec 13, 2018

View reviewed changes

alfonsomthd approved these changes Jan 25, 2019

View reviewed changes

Devp00l force-pushed the wip-prometheus-alerting branch from 06c4114 to ef43b81 Compare January 25, 2019 13:22

epuertat requested review from b-ranto and zmc January 29, 2019 09:22

votdev requested changes Jan 30, 2019

View reviewed changes

Devp00l force-pushed the wip-prometheus-alerting branch from ef43b81 to 31597d5 Compare January 30, 2019 14:45

votdev approved these changes Jan 30, 2019

View reviewed changes

p-se added the needs-rebase label Jan 30, 2019

Stephan Müller added 4 commits January 30, 2019 16:42

mgr/dashboard: Provide values with different style in KV-table

b5e0f99

Now it's possible to style values inside the KV-table based on the values. Fixes: https://tracker.ceph.com/issues/37951 Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Application icons in notifications

454f1ca

Now notifications and alerts show an application icon, that gives a hint about their origin. Fixes: https://tracker.ceph.com/issues/37950 Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Updated messages.xlf

a768571

Signed-off-by: Stephan Müller <smueller@suse.com>

Devp00l force-pushed the wip-prometheus-alerting branch from 31597d5 to a768571 Compare January 30, 2019 15:43

b-ranto approved these changes Jan 30, 2019

View reviewed changes

callithea removed the needs-rebase label Jan 31, 2019

LenzGr added the needs-qa label Jan 31, 2019

callithea added the wip-laura-testing label Jan 31, 2019

s0nea approved these changes Jan 31, 2019

View reviewed changes

p-se reviewed Jan 31, 2019

View reviewed changes

src/pybind/mgr/dashboard/controllers/prometheus.py Show resolved Hide resolved

callithea removed needs-qa wip-laura-testing labels Feb 4, 2019

Devp00l removed the needs-review label Feb 4, 2019

s0nea merged commit 6116c96 into ceph:master Feb 4, 2019

mgr/dashboard: Prometheus integration #25309

mgr/dashboard: Prometheus integration #25309

Devp00l commented Nov 28, 2018 •

edited

s0nea Nov 29, 2018

s0nea Nov 29, 2018

Devp00l Nov 29, 2018

s0nea Nov 29, 2018

s0nea Nov 29, 2018

Devp00l Nov 29, 2018

LenzGr Dec 6, 2018

LenzGr Dec 6, 2018

p-se Dec 6, 2018

Devp00l commented Dec 7, 2018

alfonsomthd left a comment

Devp00l commented Jan 25, 2019

Devp00l commented Jan 25, 2019

votdev Jan 30, 2019

Devp00l commented Jan 30, 2019

Devp00l commented Jan 31, 2019

Devp00l commented Jan 31, 2019

s0nea left a comment

callithea commented Feb 4, 2019

	If you have no access restriction you can use ``@Endpoint`` but if you have you
	If you don't have any access restriction you can use ``@Endpoint``. But if you do

	use instead ``@RESTController.Collection`` or ``@RESTController.Resource``, if
	and ``RESOURCE_ID`` is set you'll have to use ``@RESTController.Collection`` or ``@RESTController.Resource`` instead.

	``RESOURCE_ID`` is set. ``@Endpoint`` will fail here, as it does not know which
	Otherwise ``@Endpoint`` will fail here, as it does not know which

		import { NotificationType } from '../enum/notification-type.enum';

		export class CdNotificationConfig {

		#. Use the Prometheus Alertmanager API.

		#. Use both sources simultaneously.

		from ..settings import Settings


		# The receiver is needed in order to receive alert notifications (reports)

mgr/dashboard: Prometheus integration #25309

mgr/dashboard: Prometheus integration #25309

Conversation

Devp00l commented Nov 28, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Devp00l commented Dec 7, 2018

alfonsomthd left a comment

Choose a reason for hiding this comment

Devp00l commented Jan 25, 2019

Devp00l commented Jan 25, 2019

Choose a reason for hiding this comment

Devp00l commented Jan 30, 2019

Devp00l commented Jan 31, 2019

Devp00l commented Jan 31, 2019

s0nea left a comment

Choose a reason for hiding this comment

callithea commented Feb 4, 2019

Devp00l commented Nov 28, 2018 •

edited