Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mgr/dashboard: monitoring: refactor into ceph-mixin #44059

Merged
merged 4 commits into from Feb 4, 2022
Merged

mgr/dashboard: monitoring: refactor into ceph-mixin #44059

merged 4 commits into from Feb 4, 2022

Conversation

MrFreezeex
Copy link
Member

@MrFreezeex MrFreezeex commented Nov 23, 2021 •
edited

AFTER #43685

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
  • Teuthology
    • Completed teuthology run
    • No teuthology test necessary (e.g., documentation)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox

@MrFreezeex MrFreezeex requested a review from a team as a code owner November 23, 2021 08:54
@MrFreezeex MrFreezeex requested review from alfonsomthd and nizamial09 and removed request for a team November 23, 2021 08:54
@github-actions github-actions bot added this to In progress in Dashboard Nov 23, 2021
@alfonsomthd
Copy link
Contributor

@MrFreezeex Please add the tracker issue related to this PR.

@alfonsomthd
Copy link
Contributor

alfonsomthd commented Nov 23, 2021 •
edited

Hi @MrFreezeex ,
@epuertat and me are inviting you to join the next Ceph Dashboard community Daily Standup so you can introduce yourself 😃 and give us more insight on this PR.
The calendar meeting:

Ceph Dashboard Daily Standup
10:30 – 11:00 CET (Mondays, Tuesdays, Wednesdays, Fridays)
Google Meet Video call link: https://meet.google.com/bix-nypb-jma

@MrFreezeex
Copy link
Member Author

Hi @MrFreezeex , @epuertat and me are inviting you to join the next Ceph Dashboard community Daily Standup so you can introduce yourself smiley and give us more insight on this PR.

Hi, sure I will try to join tomorrow!

@github-actions github-actions bot added the tests label Nov 23, 2021
@epuertat epuertat moved this from Reviewer approved to Ready-to-merge in Dashboard Feb 4, 2022
@epuertat epuertat merged commit 84287bd into ceph:master Feb 4, 2022
Dashboard automation moved this from Ready-to-merge to Done Feb 4, 2022
@epuertat
Copy link
Member

epuertat commented Feb 4, 2022

Merged 🥳 Congrats, @MrFreezeex ! And thanks a lot for your hard work and patience here!

@epuertat epuertat mentioned this pull request Feb 4, 2022
Merged
@epuertat epuertat removed the needs-quincy-backport backport required for quincy label Feb 4, 2022
guits added a commit to guits/ceph that referenced this pull request Feb 7, 2022
@guits
cephadm: update mounts for shared_ceph_folder
0837561 
ceph#44059 introduced a refactor that breaks `--shared_ceph_folder` option
in cephadm.
Let's update cephadm and mount the new path.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
@guits guits mentioned this pull request Feb 7, 2022
Closed
nizamial09 added a commit to rhcs-dashboard/ceph that referenced this pull request Feb 7, 2022
@nizamial09
cephadm: change shared_folder directory for prometheus and grafana
27592b7 
After ceph#44059 the monitoring/prometheus
and monitoring/grafana/dashboards directories are changed to
monitoring/ceph-mixins. That broke the shared_folders in the cephadm
bootstrap script.

Changed all the instances of monitoring/prometheus and
monitoring/grafana/dashboards to monitoring/ceph-mixins

Also, renaming all the instances of prometheus_alerts.yaml to
prometheus_alerts.yml.

Fixes: https://tracker.ceph.com/issues/54176
Signed-off-by: Nizamudeen A <nia@redhat.com>
@nizamial09 nizamial09 mentioned this pull request Feb 7, 2022
Merged
14 tasks
epuertat pushed a commit to rhcs-dashboard/ceph that referenced this pull request Feb 7, 2022
@nizamial09 @epuertat
cephadm: change shared_folder directory for prometheus and grafana
5a0a14c 
After ceph#44059 the monitoring/prometheus
and monitoring/grafana/dashboards directories are changed to
monitoring/ceph-mixins. That broke the shared_folders in the cephadm
bootstrap script.

Changed all the instances of monitoring/prometheus and
monitoring/grafana/dashboards to monitoring/ceph-mixins

Also, renaming all the instances of prometheus_alerts.yaml to
prometheus_alerts.yml.

Fixes: https://tracker.ceph.com/issues/54176
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 27592b7)
epuertat pushed a commit to rhcs-dashboard/ceph that referenced this pull request Feb 9, 2022
@nizamial09 @epuertat
cephadm: change shared_folder directory for prometheus and grafana
27c2994 
After ceph#44059 the monitoring/prometheus
and monitoring/grafana/dashboards directories are changed to
monitoring/ceph-mixins. That broke the shared_folders in the cephadm
bootstrap script.

Changed all the instances of monitoring/prometheus and
monitoring/grafana/dashboards to monitoring/ceph-mixins

Also, renaming all the instances of prometheus_alerts.yaml to
prometheus_alerts.yml.

Fixes: https://tracker.ceph.com/issues/54176
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 27592b7)
@MrFreezeex MrFreezeex mentioned this pull request Mar 23, 2022
Closed
MrFreezeex pushed a commit to MrFreezeex/ceph that referenced this pull request Mar 23, 2022
@nizamial09 @MrFreezeex
cephadm: change shared_folder directory for prometheus and grafana
b2f0d5d 
After ceph#44059 the monitoring/prometheus
and monitoring/grafana/dashboards directories are changed to
monitoring/ceph-mixins. That broke the shared_folders in the cephadm
bootstrap script.

Changed all the instances of monitoring/prometheus and
monitoring/grafana/dashboards to monitoring/ceph-mixins

Also, renaming all the instances of prometheus_alerts.yaml to
prometheus_alerts.yml.

Fixes: https://tracker.ceph.com/issues/54176
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 27592b7)
MrFreezeex pushed a commit to MrFreezeex/ceph that referenced this pull request Mar 23, 2022
@nizamial09 @MrFreezeex
cephadm: change shared_folder directory for prometheus and grafana
b3f1b4e 
After ceph#44059 the monitoring/prometheus
and monitoring/grafana/dashboards directories are changed to
monitoring/ceph-mixins. That broke the shared_folders in the cephadm
bootstrap script.

Changed all the instances of monitoring/prometheus and
monitoring/grafana/dashboards to monitoring/ceph-mixins

Also, renaming all the instances of prometheus_alerts.yaml to
prometheus_alerts.yml.

Fixes: https://tracker.ceph.com/issues/54176
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 27592b7)
MrFreezeex pushed a commit to MrFreezeex/ceph that referenced this pull request Mar 23, 2022
@nizamial09 @MrFreezeex
cephadm: change shared_folder directory for prometheus and grafana
65e9005 
After ceph#44059 the monitoring/prometheus
and monitoring/grafana/dashboards directories are changed to
monitoring/ceph-mixins. That broke the shared_folders in the cephadm
bootstrap script.

Changed all the instances of monitoring/prometheus and
monitoring/grafana/dashboards to monitoring/ceph-mixins

Also, renaming all the instances of prometheus_alerts.yaml to
prometheus_alerts.yml.

Fixes: https://tracker.ceph.com/issues/54176
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 27592b7)
@MrFreezeex MrFreezeex mentioned this pull request Mar 23, 2022
Merged
14 tasks
zalsader pushed a commit to zalsader/ceph that referenced this pull request Apr 11, 2022
@nizamial09 @zalsader
cephadm: change shared_folder directory for prometheus and grafana
0d27d39 
After ceph#44059 the monitoring/prometheus
and monitoring/grafana/dashboards directories are changed to
monitoring/ceph-mixins. That broke the shared_folders in the cephadm
bootstrap script.

Changed all the instances of monitoring/prometheus and
monitoring/grafana/dashboards to monitoring/ceph-mixins

Also, renaming all the instances of prometheus_alerts.yaml to
prometheus_alerts.yml.

Fixes: https://tracker.ceph.com/issues/54176
Signed-off-by: Nizamudeen A <nia@redhat.com>
@djgalloway
Copy link

I think maybe this isn't working? https://jenkins.ceph.com/job/ceph-pull-requests/93945/

214/277 Test #264: run-tox-jsonnet-lint ......................***Failed    4.78 sec
Requirement already satisfied: tox in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (3.25.0)
Requirement already satisfied: pluggy>=0.12.0 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from tox) (1.0.0)
Requirement already satisfied: py>=1.4.17 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from tox) (1.10.0)
Requirement already satisfied: virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from tox) (20.14.1)
Requirement already satisfied: filelock>=3.0.0 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from tox) (3.6.0)
Requirement already satisfied: toml>=0.9.4 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from tox) (0.10.2)
Requirement already satisfied: six>=1.14.0 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from tox) (1.16.0)
Requirement already satisfied: packaging>=14 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from tox) (21.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from packaging>=14->tox) (2.4.7)
Requirement already satisfied: platformdirs<3,>=2 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0->tox) (2.5.2)
Requirement already satisfied: distlib<1,>=0.3.1 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0->tox) (0.3.4)
jsonnet-lint create: /home/jenkins-build/build/workspace/ceph-pull-requests/monitoring/ceph-mixin/.tox/jsonnet-lint
jsonnet-lint installdeps: -rrequirements-grafonnet.txt
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
jsonnet-lint installed: jsondiff==2.0.0
jsonnet-lint run-test-pre: PYTHONHASHSEED='3895844870'
jsonnet-lint run-test: commands[0] | ./lint-jsonnet.sh --test
/home/jenkins-build/build/workspace/ceph-pull-requests/monitoring/ceph-mixin/lint-jsonnet.sh: 5: jsonnetfmt: not found
ERROR: InvocationError for command /home/jenkins-build/build/workspace/ceph-pull-requests/monitoring/ceph-mixin/lint-jsonnet.sh --test (exited with code 127)

@MrFreezeex
Copy link
Member Author

I think maybe this isn't working? https://jenkins.ceph.com/job/ceph-pull-requests/93945/

Hmmm well the jsonnet-build tasks should install jsonnetfmt (https://github.com/ceph/ceph/blob/master/monitoring/ceph-mixin/CMakeLists.txt#L14-L15 and https://github.com/ceph/ceph/blob/master/monitoring/ceph-mixin/jsonnet-build.sh), and I can see the jsonnet-build task being executed, so not sure why it doesn't find the binary... It's weird because the jsonnet binary seems to be installed as the jsonnet-check passed...

@djgalloway
Copy link

Could WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration. have anything to do with it?

@MrFreezeex
Copy link
Member Author

Hmmm I think this line of log is normal... Is there any sign to consistently reproduce the failure? Like perhaps using worker installed recently?

@epuertat
Copy link
Member

@djgalloway we should definitely try to move to containerized builds sooner than later... Having reproducible builds that don't mess with the system's deps is going to save us lots of environment-related pains.

@djgalloway
Copy link

@djgalloway we should definitely try to move to containerized builds sooner than later... Having reproducible builds that don't mess with the system's deps is going to save us lots of environment-related pains.

Do you mean building RPMs inside a container? Let's discuss in the next CLT call.

aaSharma14 pushed a commit to rhcs-dashboard/ceph that referenced this pull request Apr 26, 2022
@nizamial09 @aaSharma14
cephadm: change shared_folder directory for prometheus and grafana
4df3e18 
After ceph#44059 the monitoring/prometheus
and monitoring/grafana/dashboards directories are changed to
monitoring/ceph-mixins. That broke the shared_folders in the cephadm
bootstrap script.

Changed all the instances of monitoring/prometheus and
monitoring/grafana/dashboards to monitoring/ceph-mixins

Also, renaming all the instances of prometheus_alerts.yaml to
prometheus_alerts.yml.

Fixes: https://tracker.ceph.com/issues/54176
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 27592b7)
@epuertat
Copy link
Member

@djgalloway we should definitely try to move to containerized builds sooner than later... Having reproducible builds that don't mess with the system's deps is going to save us lots of environment-related pains.

Do you mean building RPMs inside a container? Let's discuss in the next CLT call.

Sure (not necessarily RPMs, just the binaries).

@ktdreyer
Copy link
Member

Yeah, ideally mock would do this for us. It builds RPMs in isolated chroots. We do this on Debian-based nodes using pbuilder, but for various reasons we never did it for RPMs. It would help with reproducibility a lot.

dpaganel pushed a commit to dpaganel/ceph that referenced this pull request May 17, 2022
@nizamial09 @dpaganel
cephadm: change shared_folder directory for prometheus and grafana
8684d34 
After ceph#44059 the monitoring/prometheus
and monitoring/grafana/dashboards directories are changed to
monitoring/ceph-mixins. That broke the shared_folders in the cephadm
bootstrap script.

Changed all the instances of monitoring/prometheus and
monitoring/grafana/dashboards to monitoring/ceph-mixins

Also, renaming all the instances of prometheus_alerts.yaml to
prometheus_alerts.yml.

Fixes: https://tracker.ceph.com/issues/54176
Signed-off-by: Nizamudeen A <nia@redhat.com>
@tchaikov
Copy link
Contributor

I think maybe this isn't working? https://jenkins.ceph.com/job/ceph-pull-requests/93945/

214/277 Test #264: run-tox-jsonnet-lint ......................***Failed    4.78 sec
Requirement already satisfied: tox in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (3.25.0)
Requirement already satisfied: pluggy>=0.12.0 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from tox) (1.0.0)
Requirement already satisfied: py>=1.4.17 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from tox) (1.10.0)
Requirement already satisfied: virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from tox) (20.14.1)
Requirement already satisfied: filelock>=3.0.0 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from tox) (3.6.0)
Requirement already satisfied: toml>=0.9.4 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from tox) (0.10.2)
Requirement already satisfied: six>=1.14.0 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from tox) (1.16.0)
Requirement already satisfied: packaging>=14 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from tox) (21.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from packaging>=14->tox) (2.4.7)
Requirement already satisfied: platformdirs<3,>=2 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0->tox) (2.5.2)
Requirement already satisfied: distlib<1,>=0.3.1 in /home/jenkins-build/build/workspace/ceph-pull-requests/build/jsonnet-lint-virtualenv/lib/python3.8/site-packages (from virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0->tox) (0.3.4)
jsonnet-lint create: /home/jenkins-build/build/workspace/ceph-pull-requests/monitoring/ceph-mixin/.tox/jsonnet-lint
jsonnet-lint installdeps: -rrequirements-grafonnet.txt
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
jsonnet-lint installed: jsondiff==2.0.0
jsonnet-lint run-test-pre: PYTHONHASHSEED='3895844870'
jsonnet-lint run-test: commands[0] | ./lint-jsonnet.sh --test
/home/jenkins-build/build/workspace/ceph-pull-requests/monitoring/ceph-mixin/lint-jsonnet.sh: 5: jsonnetfmt: not found
ERROR: InvocationError for command /home/jenkins-build/build/workspace/ceph-pull-requests/monitoring/ceph-mixin/lint-jsonnet.sh --test (exited with code 127)

this indeed does not work. another occurrence is https://jenkins.ceph.com/job/ceph-pull-requests/101536/console

i created #47617 to address the test failure.

NitzanMordhai pushed a commit to NitzanMordhai/ceph that referenced this pull request Nov 19, 2023
@nizamial09 @NitzanMordhai
cephadm: change shared_folder directory for prometheus and grafana
42916cd 
After ceph#44059 the monitoring/prometheus
and monitoring/grafana/dashboards directories are changed to
monitoring/ceph-mixins. That broke the shared_folders in the cephadm
bootstrap script.

Changed all the instances of monitoring/prometheus and
monitoring/grafana/dashboards to monitoring/ceph-mixins

Also, renaming all the instances of prometheus_alerts.yaml to
prometheus_alerts.yml.

Fixes: https://tracker.ceph.com/issues/54176
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 27592b7)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@epuertat epuertat epuertat approved these changes

@nizamial09 nizamial09 Awaiting requested review from nizamial09 nizamial09 was automatically assigned from ceph/dashboard

@tchaikov tchaikov Awaiting requested review from tchaikov

@pcuzner pcuzner Awaiting requested review from pcuzner

@djgalloway djgalloway Awaiting requested review from djgalloway

@p-se p-se Awaiting requested review from p-se

@aaSharma14 aaSharma14 Awaiting requested review from aaSharma14

@ktdreyer ktdreyer Awaiting requested review from ktdreyer

@votdev votdev Awaiting requested review from votdev

@smithfarm smithfarm Awaiting requested review from smithfarm

@pereman2 pereman2 Awaiting requested review from pereman2

Assignees
No one assigned
Labels
build/ops dashboard DNM documentation monitoring pybind tests
Projects
Ceph-Dashboard
Archived in project
Dashboard
  
Done
Milestone
No milestone
7 participants
@MrFreezeex @alfonsomthd @epuertat @djgalloway @ktdreyer @tchaikov @smithfarm