Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pacific: osd/OSDMap: Add health warning if 'require-osd-release' != current release #44259

Merged
merged 8 commits into from Feb 2, 2022
Merged

pacific: osd/OSDMap: Add health warning if 'require-osd-release' != current release #44259

merged 8 commits into from Feb 2, 2022

Conversation

sseshasa
Copy link
Contributor

@sseshasa sseshasa commented Dec 9, 2021

backport tracker: https://tracker.ceph.com/issues/53551

backport of #44090
parent tracker: https://tracker.ceph.com/issues/51984

this backport was staged using ceph-backport.sh version 16.0.0.6848
find the latest version at https://github.com/ceph/ceph/blob/master/src/script/ceph-backport.sh

@sseshasa sseshasa added this to the pacific milestone Dec 9, 2021
@sseshasa sseshasa added the core label Dec 9, 2021
neha-ojha
neha-ojha reviewed Dec 9, 2021
View reviewed changes
PendingReleaseNotes Outdated Show resolved Hide resolved
@batrick
Copy link
Member

batrick commented Dec 15, 2021

#44090 (comment)

@github-actions github-actions bot added the cephfs Ceph File System label Jan 13, 2022
@sseshasa
Copy link
Contributor Author

@vshankar @batrick Please review the changes to the fs upgrade files. Thanks!

@vshankar
Copy link
Contributor

@vshankar @batrick Please review the changes to the fs upgrade files. Thanks!

Any reason why #44324 is not a separate commit (with s/quincy/pacific) in this PR?

@sseshasa
Copy link
Contributor Author

@vshankar @batrick Please review the changes to the fs upgrade files. Thanks!

Any reason why #44324 is not a separate commit (with s/quincy/pacific) in this PR?

Sure, I can cherry-pick the commit from #44324 and add one more change to "..volumes/import-legacy/tasks/2-upgrade.yaml". Would that work for you?

@vshankar
Copy link
Contributor

Sure, I can cherry-pick the commit from #44324 and add one more change to "..volumes/import-legacy/tasks/2-upgrade.yaml". Would that work for you?

Yep - that would make it clean.

@sseshasa
Copy link
Contributor Author

Sure, I can cherry-pick the commit from #44324 and add one more change to "..volumes/import-legacy/tasks/2-upgrade.yaml". Would that work for you?

Yep - that would make it clean.

The commit is cherry-picked but I changed the commit message to reflect 'pacific' for correctness. I know that commit messages should not be modified during backports, but in this case I changed it and left a comment in the 'Conflicts' section to be clear about the change. I can revert it if this is not agreeable.

@sseshasa
Copy link
Contributor Author

@neha-ojha Please review this again as I cherry-picked a subset of commits from #44161.

batrick
batrick approved these changes Jan 13, 2022
View reviewed changes
Copy link
Member

@batrick batrick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fs:upgrade changes LGTM

@vshankar
Copy link
Contributor

The commit is cherry-picked but I changed the commit message to reflect 'pacific' for correctness. I know that commit messages should not be modified during backports, but in this case I changed it and left a comment in the 'Conflicts' section to be clear about the change. I can revert it if this is not agreeable.

Thanks. Looks good!

neha-ojha
neha-ojha approved these changes Jan 18, 2022
View reviewed changes
Copy link
Member

@neha-ojha neha-ojha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: It will be good to include an explanation in 7ef6c85's commit message for why it is not cherry-picked from master.

Needs to go through the entire upgrade suite

@sseshasa
Copy link
Contributor Author

nit: It will be good to include an explanation in 7ef6c85's commit message for why it is not cherry-picked from master.

Needs to go through the entire upgrade suite

Updated the commit message and I initiated upgrade tests in teuthology for the ones that were modified.

@sseshasa
Copy link
Contributor Author

Pacific Upgrade Runs Summary:

All failures seen (2 Nos) are unrelated.

cephfs/nofs - PASSED

  1. https://pulpito.ceph.com/sseshasa-2022-01-19_10:12:04-upgrade-wip-53551-pacific-test-distro-default-smithi

cephfs/featureful_client - PASSED

  1. https://pulpito.ceph.com/sseshasa-2022-01-19_10:10:49-upgrade-wip-53551-pacific-test-distro-default-smithi

Nautilux-x - PASSED

  1. https://pulpito.ceph.com/sseshasa-2022-01-19_09:03:32-upgrade-wip-53551-pacific-test-distro-default-smithi
    • 17 dead jobs because of infra issue.
  2. https://pulpito.ceph.com/sseshasa-2022-01-20_06:56:00-upgrade-wip-53551-pacific-test-distro-default-smithi/
    • All dead jobs from first run passed

Octopus-x - 1 Test failed

  1. https://pulpito.ceph.com/sseshasa-2022-01-19_09:05:33-upgrade-wip-53551-pacific-test-distro-default-smithi/
    • 1 job passed.
    • 7/8 dead jobs due to infra issue.
  2. https://pulpito.ceph.com/sseshasa-2022-01-20_06:54:06-upgrade-wip-53551-pacific-test-distro-default-smithi/

cephfs/volumes - FAILED

  1. https://pulpito.ceph.com/sseshasa-2022-01-19_10:13:02-upgrade-wip-53551-pacific-test-distro-default-smithi
    • First attempt failed due to infra issue.
  2. https://pulpito.ceph.com/sseshasa-2022-01-20_06:55:03-upgrade-wip-53551-pacific-test-distro-default-smithi

@github-actions
Copy link

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

sseshasa and others added 8 commits February 2, 2022 10:36
@sseshasa
osd/OSDMap: Add health warning if 'require-osd-release' != current re…
655a530 
…lease

After all OSDs are upgraded to a new release, generate a health warning if
the 'require-osd-release' flag doesn't match the the new release version.
This will result in the cluster showing a warning in the health state until
the flag is set properly.

Fixes: https://tracker.ceph.com/issues/51984
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit bd815bd)

Conflicts:
    src/osd/OSDMap.cc
- Removed checks for non-existent ceph_release_t 'quincy' flag from
  OSDMap::pending_require_osd_release().
@sseshasa
qa/suites/upgrade: Fix/Modify upgrade tests to work with 'pacific' re…
a9182c4 
…lease.

This commit is not a cherry-pick and fixes the following issues unique to
the pacific release:
1. Fixes the nautilus-x and octopus-x upgrade tests to work with the
   pacific release by updating the post upgrade step to set the
   'require-osd-release' flag to 'pacific'. This is done by using
   '.qa/releases/pacific.yaml' for all the tests.
2. Fixed an issue in 'upgrade-mon-osd-mds.yaml' under both the
   nautilus-x/parallel, octopus-x/parallel-no-cephadm tests. The
   'wait-for-healthy' check should not be performed after all the osds
   are upgraded since the 'require-osd-release' warning comes into
   effect. This check is delayed until after the 'require-osd-release'
   flag is set to 'pacific'.

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
@neha-ojha @sseshasa
qa: remove leftovers from nautilus
e02049e 
pglog_hardlimit and msgr2

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit ed4bb05)
@neha-ojha @sseshasa
qa: rename octopus install correctly
e7574ec 
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 3b15a04)
@neha-ojha @sseshasa
qa: test upgrades with hybrid allocator
3f83024 
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit df67040)
@neha-ojha @sseshasa
qa/suites/upgrade/octopus-x/stress-split-no-cephadm: remove msgr2
89da1aa 
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 6ad7a8a)
@batrick @sseshasa
qa: set pacific require-osd-release to avoid health warning
a58ab16 
Fixes: https://tracker.ceph.com/issues/53615
Fixes: bd815bd
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit bc2eaba)

Conflicts:
- Changed release name in commit message from 'quincy' to 'pacific'.

  ../fs/upgrade/featureful_client/old_client/tasks/2-upgrade.yaml
  ../fs/upgrade/featureful_client/upgraded_client/tasks/2-upgrade.yaml
  ../fs/upgrade/nofs/tasks/1-upgrade.yaml
- Changed release name from 'quincy' to 'pacific' when setting the
  'require-osd-release' flag in the above files.

  ../fs/upgrade/volumes/import-legacy/tasks/2-upgrade.yaml
- Changed release name from 'octopus' to 'pacific' when setting the
  'require-osd-release' flag in the above file.
@sseshasa
PendingReleaseNotes: Release note 'require-osd-release' health warning
9203ed5 
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit 34f18fa)

Conflicts:
    PendingReleaseNotes
- Add the release note under the correct release heading.
@yuriw yuriw merged commit 1d1c377 into ceph:pacific Feb 2, 2022
8 checks passed
@sseshasa
Copy link
Contributor Author

sseshasa commented Feb 3, 2022

Teuthology Test Result

http://pulpito.front.sepia.ceph.com/yuriw-2022-01-27_00:01:03-upgrade-wip-yuri2-testing-2022-01-21-0949-pacific-distro-default-smithi/

Unrelated Failures:

  1. http://pulpito.front.sepia.ceph.com/yuriw-2022-01-27_00:01:03-upgrade-wip-yuri2-testing-2022-01-21-0949-pacific-distro-default-smithi/6642457
  • ceph-mon on one node hit the below assertion failure:
    src/mds/FSMap.cc: 868: FAILED ceph_assert(info.compat.writeable(fs->mds_map.compat))
  • Tracked by: https://tracker.ceph.com/issues/54064
  1. http://pulpito.front.sepia.ceph.com/yuriw-2022-01-27_00:01:03-upgrade-wip-yuri2-testing-2022-01-21-0949-pacific-distro-default-smithi/6642471

The dead jobs are infrastructure related.

@sseshasa sseshasa deleted the wip-53551-pacific branch February 3, 2022 05:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@batrick batrick batrick approved these changes

@neha-ojha neha-ojha neha-ojha approved these changes

@jdurgin jdurgin Awaiting requested review from jdurgin

@vshankar vshankar Awaiting requested review from vshankar

Assignees
No one assigned
Labels
cephfs Ceph File System core documentation needs-qa wip-yuri2-testing
Projects
None yet
Milestone
pacific
5 participants
@sseshasa @batrick @vshankar @neha-ojha @yuriw