quincy: mds: wait unlink to finish to avoid conflict when creating same dentries #48452

lxbsz · 2022-10-12T02:52:00Z

backport tracker: https://tracker.ceph.com/issues/57837

backport of #46331
parent tracker: https://tracker.ceph.com/issues/55332

Contribution Guidelines

To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

Checklist

Tracker (select at least one)
- References tracker ticket
- Very recent bug; references commit where it was introduced
- New feature (ticket optional)
- Doc update (no ticket needed)
- Code cleanup (no ticket needed)
Component impact
- Affects Dashboard, opened tracker ticket
- Affects Orchestrator, opened tracker ticket
- No impact that needs to be tracked
Documentation (select at least one)
- Updates relevant documentation
- No doc update is appropriate
Tests (select at least one)
- Includes unit test(s)
- Includes integration test(s)
- Includes bug reproducer
- No tests

Show available Jenkins commands

jenkins retest this please
jenkins test classic perf
jenkins test crimson perf
jenkins test signed
jenkins test make check
jenkins test make check arm64
jenkins test submodules
jenkins test dashboard
jenkins test dashboard cephadm
jenkins test api
jenkins test docs
jenkins render docs
jenkins test ceph-volume all
jenkins test ceph-volume tox
jenkins test windows

If the previous unlink request has been delayed due to some reasons, and the new creating for the same dentry may fail or new open will succeeds but new contents wrote to it will be lost. The kernel client will make sure before the unlink getting the first reply it won't send the followed create requests for the same dentry. Here we need to make sure that before the first reply has been sent out the dentry must be marked as unlinking. Fixes: https://tracker.ceph.com/issues/55332 Signed-off-by: Xiubo Li <xiubli@redhat.com> (cherry picked from commit d4b9431)

Signed-off-by: Xiubo Li <xiubli@redhat.com> (cherry picked from commit 417f247)

joscollin · 2023-02-17T10:50:07Z

jenkins test api

ljflores · 2023-02-17T17:26:03Z

Rados suite review: https://pulpito.ceph.com/?branch=wip-yuri3-testing-2023-02-16-0752-quincy

Failures, unrelated:
1. https://tracker.ceph.com/issues/58146
2. https://tracker.ceph.com/issues/58585

Details:
1. test_cephadm.sh: Error: Error initializing source docker://quay.ceph.io/ceph-ci/ceph:master - Ceph - Orchestrator
2. rook: failed to pull kubelet image - Ceph - Orchestrator

ljflores · 2023-02-17T17:26:12Z

jenkins test api

yuriw · 2023-02-23T16:20:48Z

jenkins test api

yuriw · 2023-02-23T16:21:39Z

this is tested and ready for merge @lxbsz @joscollin
ref: https://trello.com/c/HXwVRMzB

yuriw · 2023-02-23T18:23:44Z

See API tests errors:

`

No such file or directoryNo such file or directoryIndication 1

| Python Assertion ErrorPython Assertion Error (probably from a Unit Test)Indication 2
| Python Unit-test FailureFAIL: test_issue_tracker_create_with_invalid_key (tasks.mgr.dashboard.test_feedback.FeedbackTest)⇒ Check for already reported issues on "FAIL: test_issue_tracker_create_with_invalid_key (tasks.mgr.dashboard.test_feedback.FeedbackTest)" in Ceph Tracker...... or open a new issue. 2023-02-23 17:28:40,784.784 INFO:main:---------------------------------------------------------------------- 2023-02-23 17:28:40,784.784 INFO:main:Traceback (most recent call last): 2023-02-23 17:28:40,784.784 INFO:main: File "/home/jenkins-build/build/workspace/ceph-api/qa/tasks/mgr/dashboard/test_feedback.py", line 31, in test_issue_tracker_create_with_invalid_key 2023-02-23 17:28:40,784.784 INFO:main: self.assertStatus(400) 2023-02-23 17:28:40,784.784 INFO:main: File "/home/jenkins-build/build/workspace/ceph-api/qa/tasks/mgr/dashboard/helper.py", line 495, in assertStatus 2023-02-23 17:28:40,784.784 INFO:main: self.assertEqual(self._resp.status_code, status) 2023-02-23 17:28:40,784.784 INFO:main:AssertionError: 500 != 400 2023-02-23 17:28:40,784.784 INFO:main: Cannot find device "ceph-brx" 2023-02-23 17:28:40,806.806 INFO:main: 2023-02-23 17:28:40,806.806 INFO:main:---------------------------------------------------------------------- 2023-02-23 17:28:40,806.806 INFO:main:Indication 3
| Error E{INVAL,PERM,ACCESS...}Found some Errors in the build logIndication 4
| Python Dependency Error (PIP)An error happened while pip was installing Python dependencies:paramiko 3.0.0 requires bcrypt>=3.2, but you have bcrypt 3.1.4 which is incompatible. Indication 5
| Failed testTest failedIndication 6
| Python Tracebackurllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachableTraceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 382, in _make_request self._validate_conn(conn) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1012, in _validate_conn conn.connect() File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 353, in connect conn = self._new_conn() File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in _new_conn raise NewConnectionError( urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachableIndication 7

No such file or directory

No such file or directory
Indication 1
Python Assertion Error

Python Assertion Error (probably from a Unit Test)
Indication 2
Python Unit-test Failure

FAIL: test_issue_tracker_create_with_invalid_key (tasks.mgr.dashboard.test_feedback.FeedbackTest)
⇒ Check for already reported issues on "FAIL: test_issue_tracker_create_with_invalid_key (tasks.mgr.dashboard.test_feedback.FeedbackTest)" in Ceph Tracker...
... or open a new issue.

2023-02-23 17:28:40,784.784 INFO:main:----------------------------------------------------------------------
2023-02-23 17:28:40,784.784 INFO:main:Traceback (most recent call last):
2023-02-23 17:28:40,784.784 INFO:main: File "/home/jenkins-build/build/workspace/ceph-api/qa/tasks/mgr/dashboard/test_feedback.py", line 31, in test_issue_tracker_create_with_invalid_key
2023-02-23 17:28:40,784.784 INFO:main: self.assertStatus(400)
2023-02-23 17:28:40,784.784 INFO:main: File "/home/jenkins-build/build/workspace/ceph-api/qa/tasks/mgr/dashboard/helper.py", line 495, in assertStatus
2023-02-23 17:28:40,784.784 INFO:main: self.assertEqual(self._resp.status_code, status)
2023-02-23 17:28:40,784.784 INFO:main:AssertionError: 500 != 400
2023-02-23 17:28:40,784.784 INFO:main:
Cannot find device "ceph-brx"
2023-02-23 17:28:40,806.806 INFO:main:
2023-02-23 17:28:40,806.806 INFO:main:----------------------------------------------------------------------
2023-02-23 17:28:40,806.806 INFO:main:

Indication 3
Error E{INVAL,PERM,ACCESS...}

Found some Errors in the build log
Indication 4
Python Dependency Error (PIP)

An error happened while pip was installing Python dependencies:
paramiko 3.0.0 requires bcrypt>=3.2, but you have bcrypt 3.1.4 which is incompatible.

Indication 5
Failed test

Test failed
Indication 6
Python Traceback

urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1012, in _validate_conn
conn.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 353, in connect
conn = self._new_conn()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable

Indication 7`

yuriw · 2023-02-23T18:24:00Z

jenkins test api

lxbsz added 2 commits October 12, 2022 10:47

mds: clear the STATE_UNLINKING state when the unlink fails

bd71972

Signed-off-by: Xiubo Li <xiubli@redhat.com> (cherry picked from commit 417f247)

lxbsz requested a review from a team October 12, 2022 02:52

lxbsz requested a review from a team as a code owner October 12, 2022 02:52

github-actions bot added cephfs Ceph File System core labels Oct 12, 2022

github-actions bot added this to the quincy milestone Oct 12, 2022

vshankar added needs-qa quincy-batch-1 labels Dec 14, 2022

yuriw added the wip-yuri3-testing label Feb 6, 2023

joscollin approved these changes Feb 17, 2023

View reviewed changes

yuriw added TESTED ready-to-merge and removed wip-yuri3-testing labels Feb 23, 2023

vshankar approved these changes Feb 24, 2023

View reviewed changes

yuriw merged commit 4819147 into ceph:quincy Feb 25, 2023
14 of 15 checks passed

quincy: mds: wait unlink to finish to avoid conflict when creating same dentries #48452

quincy: mds: wait unlink to finish to avoid conflict when creating same dentries #48452

lxbsz commented Oct 12, 2022 •

edited by joscollin

joscollin commented Feb 17, 2023

ljflores commented Feb 17, 2023

ljflores commented Feb 17, 2023

yuriw commented Feb 23, 2023

yuriw commented Feb 23, 2023

yuriw commented Feb 23, 2023

yuriw commented Feb 23, 2023

quincy: mds: wait unlink to finish to avoid conflict when creating same dentries #48452

quincy: mds: wait unlink to finish to avoid conflict when creating same dentries #48452

Conversation

lxbsz commented Oct 12, 2022 • edited by joscollin

Contribution Guidelines

Checklist

joscollin commented Feb 17, 2023

ljflores commented Feb 17, 2023

ljflores commented Feb 17, 2023

yuriw commented Feb 23, 2023

yuriw commented Feb 23, 2023

yuriw commented Feb 23, 2023

No such file or directoryNo such file or directoryIndication 1

yuriw commented Feb 23, 2023

lxbsz commented Oct 12, 2022 •

edited by joscollin