mds: prevent scrubbing for standby-replay MDS #53301

neesingh-rh · 2023-09-06T06:29:16Z

Fixes: https://tracker.ceph.com/issues/62537

Signed-off-by: Neeraj Pratap Singh neesingh@redhat.com

Contribution Guidelines

To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

Checklist

Tracker (select at least one)
- References tracker ticket
- Very recent bug; references commit where it was introduced
- New feature (ticket optional)
- Doc update (no ticket needed)
- Code cleanup (no ticket needed)
Component impact
- Affects Dashboard, opened tracker ticket
- Affects Orchestrator, opened tracker ticket
- No impact that needs to be tracked
Documentation (select at least one)
- Updates relevant documentation
- No doc update is appropriate
Tests (select at least one)
- Includes unit test(s)
- Includes integration test(s)
- Includes bug reproducer
- No tests

Show available Jenkins commands

jenkins retest this please
jenkins test classic perf
jenkins test crimson perf
jenkins test signed
jenkins test make check
jenkins test make check arm64
jenkins test submodules
jenkins test dashboard
jenkins test dashboard cephadm
jenkins test api
jenkins test docs
jenkins render docs
jenkins test ceph-volume all
jenkins test ceph-volume tox
jenkins test windows

src/mds/MDSRank.cc

vshankar · 2023-09-26T15:14:34Z

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230926.081818

src/mds/MDSRank.cc

vshankar · 2023-10-04T06:25:56Z

@neesingh-rh please ping when this ready for review again.

neesingh-rh · 2023-10-05T06:33:02Z

@neesingh-rh please ping when this ready for review again.

I have updated the PR, ready for re-review.

mchangir

LGTM

* refs/pull/53301/head: mds: prevent scrub start for standby-replay MDS Reviewed-by: Venky Shankar <vshankar@redhat.com>

vshankar

@neesingh-rh LGTM -- please add a test.

neesingh-rh · 2023-10-10T10:38:09Z

@neesingh-rh LGTM -- please add a test.

Added the test. PTAL

src/mds/MDSRank.cc

qa/tasks/cephfs/test_scrub_checks.py

batrick · 2023-10-17T16:10:45Z

qa/tasks/cephfs/test_scrub_checks.py

+        # start the scrub and verify
+        with self.assertRaises(CommandFailedError) as ce:
+            self.fs.run_scrub(["start", abs_test_path, "recursive"])
+        self.assertEqual(ce.exception.exitstatus, errno.EINVAL)


Suggested change

self.assertEqual(ce.exception.exitstatus, errno.EINVAL)

self.assertEqual(ce.exception.exitstatus, errno.EINVAL)

and others below

We should be checking for the error status only after getting the command fail exit status. I guess it should be this way only. https://github.com/ceph/ceph/blob/quincy/qa/tasks/cephfs/test_scrub_checks.py#L354

ce would be available inside with .., isn't it?

with.. is an alternative for try-catch, how can the statement be executed after the exception is encountered at the run_scrub only. Pls correct me if you mean something else.

I guess this is fine too

batrick · 2023-10-17T16:12:52Z

qa/tasks/cephfs/test_scrub_checks.py

+
+        # start the scrub and verify
+        with self.assertRaises(CommandFailedError) as ce:
+            self.fs.run_scrub(["start", abs_test_path, "recursive"])


I'm puzzled. Is this test not racing with the MDS becoming active?

I also expected to see a scrub command directed at the standby-replay daemon.

Spoke to @neesingh-rh - this needs fixing. Fetch the s-r mds daemons id and do a ceph tell mds.<> scrub start to the s-r daemon.

Updated the PR with the proposed changes.

vshankar · 2023-11-28T09:45:53Z

@neesingh-rh ping?

github-actions · 2024-01-27T11:01:26Z

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

neesingh-rh · 2024-04-04T06:53:32Z

@neesingh-rh status of this?

In progress!!

neesingh-rh · 2024-04-25T10:41:45Z

@vshankar Here's the run link with the latest changes: https://pulpito.ceph.com/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/

neesingh-rh · 2024-05-07T07:28:43Z

jenkins test make check

neesingh-rh · 2024-05-22T10:18:59Z

@vshankar Here's the run link with the latest changes: https://pulpito.ceph.com/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/

@vshankar ping?

neesingh-rh · 2024-05-28T11:11:35Z

jenkins retest this please

vshankar · 2024-05-28T12:18:57Z

@vshankar ping?

In my queue for reviews.

vshankar

LGTM

vshankar · 2024-06-03T11:03:37Z

This PR is under test in https://tracker.ceph.com/issues/66327.

neesingh-rh · 2024-06-13T10:28:49Z

jenkins test make check

vshankar · 2024-06-17T09:44:02Z

This PR is under test in https://tracker.ceph.com/issues/66522.

vshankar

https://tracker.ceph.com/projects/cephfs/wiki/Main#wip-vshankar-testing-20240617094253-debug

(https://tracker.ceph.com/issues/66522)

vshankar · 2024-06-25T07:11:52Z

jenkins test make check

vshankar · 2024-06-27T14:29:38Z

jenkins test make check

vshankar · 2024-06-28T17:18:29Z

jenkins test make check

vshankar · 2024-07-01T06:31:46Z

@neesingh-rh could you rebase please? Jenkins tests aren't passing - maybe a rebase would kick it up.

Fixes: https://tracker.ceph.com/issues/62537 Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>

github-actions bot added the cephfs Ceph File System label Sep 6, 2023

batrick requested changes Sep 12, 2023

View reviewed changes

src/mds/MDSRank.cc Outdated Show resolved Hide resolved

neesingh-rh force-pushed the wip-62537 branch 2 times, most recently from 631bb8b to c181dd8 Compare September 13, 2023 09:46

vshankar approved these changes Sep 26, 2023

View reviewed changes

vshankar added the wip-vshankar-testing3 label Sep 26, 2023

neesingh-rh requested a review from batrick September 26, 2023 11:37

dparmar18 approved these changes Sep 26, 2023

View reviewed changes

mchangir reviewed Sep 27, 2023

View reviewed changes

src/mds/MDSRank.cc Show resolved Hide resolved

neesingh-rh force-pushed the wip-62537 branch from c181dd8 to 8982199 Compare September 28, 2023 13:37

dparmar18 reviewed Sep 28, 2023

View reviewed changes

src/mds/MDSRank.cc Show resolved Hide resolved

vshankar removed the wip-vshankar-testing3 label Oct 4, 2023

mchangir approved these changes Oct 5, 2023

View reviewed changes

vshankar added a commit to vshankar/ceph that referenced this pull request Oct 7, 2023

Merge PR ceph#53301 into wip-vshankar-testing-20230926.081818

7e9ae82

* refs/pull/53301/head: mds: prevent scrub start for standby-replay MDS Reviewed-by: Venky Shankar <vshankar@redhat.com>

vshankar approved these changes Oct 9, 2023

View reviewed changes

vshankar requested changes Oct 9, 2023

View reviewed changes

github-actions bot added the tests label Oct 10, 2023

dparmar18 reviewed Oct 11, 2023

View reviewed changes

src/mds/MDSRank.cc Show resolved Hide resolved

dparmar18 reviewed Oct 11, 2023

View reviewed changes

qa/tasks/cephfs/test_scrub_checks.py Outdated Show resolved Hide resolved

neesingh-rh force-pushed the wip-62537 branch from 807985f to 5f504d4 Compare October 11, 2023 13:56

vshankar approved these changes Oct 12, 2023

View reviewed changes

vshankar added the wip-vshankar-testing4 label Oct 12, 2023

batrick requested changes Oct 17, 2023

View reviewed changes

vshankar removed the wip-vshankar-testing4 label Oct 20, 2023

neesingh-rh force-pushed the wip-62537 branch from 5ab119f to e383c06 Compare April 4, 2024 09:58

neesingh-rh force-pushed the wip-62537 branch 3 times, most recently from 3d6217a to 28458ef Compare April 18, 2024 10:27

neesingh-rh force-pushed the wip-62537 branch 3 times, most recently from a3b7a18 to af6be74 Compare April 25, 2024 10:40

neesingh-rh requested review from batrick and vshankar May 22, 2024 10:19

vshankar reviewed Jun 3, 2024

View reviewed changes

vshankar added the wip-vshankar-testing3 label Jun 3, 2024

vshankar approved these changes Jun 24, 2024

View reviewed changes

neeraj pratap singh added 2 commits July 1, 2024 12:36

mds: prevent scrub start for standby-replay MDS

525a3be

Fixes: https://tracker.ceph.com/issues/62537 Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>

qa: adding test for preventing scrub when mds is inactive

6940d3a

Fixes: https://tracker.ceph.com/issues/62537 Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>

neesingh-rh force-pushed the wip-62537 branch from af6be74 to 6940d3a Compare July 1, 2024 07:07

vshankar added ready-to-merge and removed wip-vshankar-testing3 labels Jul 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mds: prevent scrubbing for standby-replay MDS #53301

mds: prevent scrubbing for standby-replay MDS #53301

neesingh-rh commented Sep 6, 2023 •

edited

Loading

vshankar commented Sep 26, 2023

vshankar commented Oct 4, 2023

neesingh-rh commented Oct 5, 2023

mchangir left a comment

vshankar left a comment

neesingh-rh commented Oct 10, 2023

batrick Oct 17, 2023

batrick Oct 17, 2023

neesingh-rh Apr 4, 2024

vshankar May 28, 2024

neesingh-rh May 29, 2024

vshankar Jun 3, 2024

batrick Oct 17, 2023

vshankar Apr 2, 2024 •

edited

Loading

neesingh-rh Apr 25, 2024

vshankar commented Nov 28, 2023

github-actions bot commented Jan 27, 2024

neesingh-rh commented Apr 4, 2024

neesingh-rh commented Apr 25, 2024

neesingh-rh commented May 7, 2024

neesingh-rh commented May 22, 2024

neesingh-rh commented May 28, 2024

vshankar commented May 28, 2024

vshankar left a comment

vshankar commented Jun 3, 2024

neesingh-rh commented Jun 13, 2024

vshankar commented Jun 17, 2024

vshankar left a comment

vshankar commented Jun 25, 2024

vshankar commented Jun 27, 2024

vshankar commented Jun 28, 2024

vshankar commented Jul 1, 2024

	self.assertEqual(ce.exception.exitstatus, errno.EINVAL)
	self.assertEqual(ce.exception.exitstatus, errno.EINVAL)

mds: prevent scrubbing for standby-replay MDS #53301

Are you sure you want to change the base?

mds: prevent scrubbing for standby-replay MDS #53301

Conversation

neesingh-rh commented Sep 6, 2023 • edited Loading

Contribution Guidelines

Checklist

vshankar commented Sep 26, 2023

vshankar commented Oct 4, 2023

neesingh-rh commented Oct 5, 2023

mchangir left a comment

Choose a reason for hiding this comment

vshankar left a comment

Choose a reason for hiding this comment

neesingh-rh commented Oct 10, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vshankar Apr 2, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vshankar commented Nov 28, 2023

github-actions bot commented Jan 27, 2024

neesingh-rh commented Apr 4, 2024

neesingh-rh commented Apr 25, 2024

neesingh-rh commented May 7, 2024

neesingh-rh commented May 22, 2024

neesingh-rh commented May 28, 2024

vshankar commented May 28, 2024

vshankar left a comment

Choose a reason for hiding this comment

vshankar commented Jun 3, 2024

neesingh-rh commented Jun 13, 2024

vshankar commented Jun 17, 2024

vshankar left a comment

Choose a reason for hiding this comment

vshankar commented Jun 25, 2024

vshankar commented Jun 27, 2024

vshankar commented Jun 28, 2024

vshankar commented Jul 1, 2024

neesingh-rh commented Sep 6, 2023 •

edited

Loading

vshankar Apr 2, 2024 •

edited

Loading