Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pacific: backport of monitoring related PRs #45980

Merged
merged 13 commits into from
Apr 26, 2022
Merged

pacific: backport of monitoring related PRs #45980

merged 13 commits into from
Apr 26, 2022

Conversation

aaSharma14
Copy link
Contributor

Backport of #45583
Backport of #45335
Backport of #45254

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows

@aaSharma14 aaSharma14 requested review from a team as code owners April 21, 2022 05:14
@aaSharma14 aaSharma14 requested review from pereman2 and Sarthak0702 and removed request for a team April 21, 2022 05:14
@github-actions github-actions bot added this to the pacific milestone Apr 21, 2022
@aaSharma14
Copy link
Contributor Author

jenkins test make check

@pereman2 pereman2 force-pushed the backport-monitoring-pacific branch from 86355f4 to 6299953 Compare April 21, 2022 09:59
@epuertat epuertat changed the title Backport of monitoring related PRs pacific: backport of monitoring related PRs Apr 22, 2022
@github-actions
Copy link

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

MrFreezeex and others added 7 commits April 26, 2022 10:03
@MrFreezeex @aaSharma14
spec: debian: add golang as build dependency
c35512a 
Add golang as a build dependency to build golang project in the test
for monitoring/ceph-mixin.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
(cherry picked from commit e102620)
@MrFreezeex @aaSharma14
mgr/dashboard: monitoring: refactor into ceph-mixin
fd5ec3b 
Mixin is a way to bundle dashboards, prometheus rules and alerts into
jsonnet package. Shifting to mixin will allow easier integration with
monitoring automation that some users may use.

This commit moves `/monitoring/grafana/dashboards` and
`/monitoring/prometheus` to `/monitoring/ceph-mixin`. Prometheus alerts
was also converted to Jsonnet using an automated way (from yaml to json
to jsonnet). This commit minimises any change made to the generated files
and should not change neithers the dashboards nor the Prometheus alerts.

In the future some configuration will also be added to jsonnet to add
more functionalities to the dashboards or alerts (i.e.: multi cluster).

Fixes: https://tracker.ceph.com/issues/53374
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
(cherry picked from commit 98236e3)

Conflicts:
      monitoring/grafana/dashboards/CMakeLists.txt
      monitoring/grafana/dashboards/jsonnet/grafana_dashboards.jsonnet
      monitoring/grafana/dashboards/tox.ini
      monitoring/prometheus/README.md
      src/test/CMakeLists.txt

Trivial fixes

      monitoring/ceph-mixin/dashboards_out/osds-overview.json
      monitoring/ceph-mixin/dashboards_out/pool-overview.json
      monitoring/ceph-mixin/dashboards_out/rbd-details.json

Include small fixes not brought in pacific yet
@MrFreezeex @aaSharma14
spec: debian: monitoring: build jsonnet from source to use 0.18.0
7186921 
As this new version is recently released it's still not in every distro
we use. We now build jsonnet from source so that we can use this new
version of jsonnet. This commit could be reverted later on when the new
version would be available everywhere.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
(cherry picked from commit ecaf907)
@MrFreezeex @aaSharma14
monitoring: build jsonnet/jb only for testing
1ba34e6 
Build jsonnet and jb in the testso that we can build ceph without
internet access and still be able to run the test needed for monitoring
using jsonnet tools.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
(cherry picked from commit 8ff1e6b)
@nizamial09 @aaSharma14
cephadm: change shared_folder directory for prometheus and grafana
4df3e18 
After ceph#44059 the monitoring/prometheus
and monitoring/grafana/dashboards directories are changed to
monitoring/ceph-mixins. That broke the shared_folders in the cephadm
bootstrap script.

Changed all the instances of monitoring/prometheus and
monitoring/grafana/dashboards to monitoring/ceph-mixins

Also, renaming all the instances of prometheus_alerts.yaml to
prometheus_alerts.yml.

Fixes: https://tracker.ceph.com/issues/54176
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 27592b7)
@aaSharma14
mgr/dashboard: Pool overall performance shows multiple entries of sam…
e44f9e4 
…e pool in pool overview

This PR intends to fix this issue

Fixes:https://tracker.ceph.com/issues/54513
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 9719cc7)
@travisn @aaSharma14
prometheus: spell check the alert descriptions
f08cc45 
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 9cca95b)
@aaSharma14
Copy link
Contributor Author

jenkins test dashboard cephadm

@aaSharma14
Copy link
Contributor Author

jenkins test dashboard

@aaSharma14
Copy link
Contributor Author

jenkins test dashboard

@pereman2
monitoring/grafana: doctest util regex
68a3d0f 
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
@pereman2
monitoring/grafana: Replace missing legendFormat warning with error
5684e73 
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
@pereman2
monitoring/grafana: ValueError instead of RuntimeError
f5a3a8a 
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
@pereman2
monitoring/grafana: remove explicit str casting
45452b9 
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
@pereman2
monitoring/grafana: use Path class instead of split
e7a739f 
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
@pereman2
monitoring/grafana: tox doctests tests_dashboards/util.py
6525df9 
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
@pereman2 pereman2 force-pushed the backport-monitoring-pacific branch from 8fc7e83 to 6525df9 Compare April 26, 2022 08:50
@epuertat epuertat mentioned this pull request Apr 26, 2022
Closed
@epuertat
Copy link
Member

epuertat commented Apr 26, 2022 •
edited
Loading

@yuriw once the Dashboard QA run passes, we may merge this:

@epuertat epuertat mentioned this pull request Apr 26, 2022
Merged
@epuertat epuertat merged commit 2a4a95f into ceph:pacific Apr 26, 2022
@epuertat epuertat deleted the backport-monitoring-pacific branch April 26, 2022 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pereman2 pereman2 pereman2 approved these changes

@epuertat epuertat epuertat approved these changes

@nizamial09 nizamial09 nizamial09 approved these changes

@Sarthak0702 Sarthak0702 Awaiting requested review from Sarthak0702 Sarthak0702 is a code owner automatically assigned from ceph/dashboard

Assignees
No one assigned
Labels
build/ops cephadm dashboard documentation mgr monitoring needs-qa pacific-batch-1 pybind tests wip-epuertat-testing
Projects
Ceph-Dashboard
Archived in project
Milestone
pacific
Development

Successfully merging this pull request may close these issues.
6 participants
@aaSharma14 @epuertat @pereman2 @nizamial09 @MrFreezeex @travisn