Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

librbd: ignore EDQUOT errors when releasing the lock #29375

Closed
wants to merge 4 commits into from

Conversation

mxdInspur
Copy link
Contributor

@mxdInspur mxdInspur commented Jul 29, 2019

rbd: Add a process when blocking writes or invalidate cache returns with EDQUOT

If the pool is full, the block_write that releases the exclusive lock fails with EDQUOT when we delete the lun, which causes the close object_map to terminate and the object_map object not to be released. Then in the rbd_close func the close() is called, an assert about whether object_map is empty will go wrong and a core error occurs

Signed-off-by: muxiangdong muxiangdong@inspur.com

@sebastian-philipp
Copy link
Contributor

Please sign your work by adding a Signed-off-by to your git commit message. This is mandatory.

@dillaman
Copy link

@mxdInspur I am not sure I am following your claim of "data inconsistency and data loss". How does this occur?

@dillaman dillaman added the DNM label Aug 1, 2019
@mxdInspur mxdInspur changed the title add the process when EDQUOT occurs RBD: add the process when EDQUOT occurs Aug 13, 2019
@mxdInspur
Copy link
Contributor Author

Please sign your work by adding a Signed-off-by to your git commit message. This is mandatory.

done

@dillaman
Copy link

I am not sure I am following your claim of "data inconsistency and data loss". How does this occur?

@mxdInspur ping

@mxdInspur
Copy link
Contributor Author

I am not sure I am following your claim of "data inconsistency and data loss". How does this occur?

@mxdInspur ping

@dillaman I restate the issue, check this plesae

Copy link

@dillaman dillaman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add test cases to src/test/librbd/exclusive_lock/test_mock_PreReleaseRequest.cc where you inject a EDQUOT error when attempting to block writes and invalidate the cache. You might also need to cover the case where the journal is in-use and it fails to flush the journal due to to those errors?

src/librbd/exclusive_lock/PreReleaseRequest.cc Outdated Show resolved Hide resolved
src/librbd/exclusive_lock/PreReleaseRequest.cc Outdated Show resolved Hide resolved
@dillaman
Copy link

@mxdInspur lgtm -- just needs the new test cases

@mxdInspur
Copy link
Contributor Author

@mxdInspur lgtm -- just needs the new test cases

actually i add three test cases, please review,thanks.

@mxdInspur mxdInspur force-pushed the master branch 7 times, most recently from f55b324 to 8f0a4b0 Compare August 22, 2019 09:34
@dillaman
Copy link

dillaman commented Aug 22, 2019

@mxdInspur You should be able to combine all three new test cases into a single test case just like the blacklist test case since the errors are ignored.

Also, please prefix your commit subject lines w/ "librbd: ...." (and squash the two commits that stomp on each other) and "test/librbd: ...."

@mxdInspur
Copy link
Contributor Author

@mxdInspur You should be able to combine all three new test cases into a single test case just like the blacklist test case since the errors are ignored.

Also, please prefix your commit subject lines w/ "librbd: ...." (and squash the two commits that stomp on each other) and "test/librbd: ...."

@dillaman I have updated according to your suggestion. Please review,thanks.

@dillaman
Copy link

dillaman commented Aug 22, 2019

lgtm (pending the results of make check) -- still need to update the commit subject line to add a "librbd: ..." prefix -- maybe just change it to "librbd: ignore EDQUOT errors when releasing the lock".

@dillaman dillaman changed the title RBD: add the process when EDQUOT occurs librbd: ignore EDQUOT errors when releasing the lock Aug 22, 2019
@mxdInspur mxdInspur force-pushed the master branch 5 times, most recently from 14dbc3d to df48b03 Compare August 26, 2019 02:14
@mxdInspur
Copy link
Contributor Author

fix the signed-off-by issue

@mxdInspur mxdInspur reopened this Aug 30, 2019
Also add a new test cases on BlockWrite and invalidateCache

Signed-off-by: mxdInspur <muxiangdong@inspur.com>
@dillaman
Copy link

dillaman commented Sep 3, 2019

@mxdInspur How do you recreate the case where a full quota is returning a -EDQUOT error from the OSDs? It should only be returned when the try full flag is set (which only happens when removing things), but I cannot recreate the original problem.

@mxdInspur
Copy link
Contributor Author

mxdInspur commented Sep 7, 2019

@mxdInspur How do you recreate the case where a full quota is returning a -EDQUOT error from the OSDs? It should only be returned when the try full flag is set (which only happens when removing things), but I cannot recreate the original problem.

@dillaman In our client implementation process, release_lock is first called to unlock before deletiing the lun. If the pool is full, an EDQUOT will occur in the release_lock. then an assert in the lun-deleting will go wrong.

@dillaman
Copy link

dillaman commented Sep 7, 2019

@dillaman In our client implementation process, release_lock is first called to unlock before deletiing the lun. If the pool is full, an EDQUOT will occur in the release_lock. then an assert in the lun-deleting will go wrong.

@mxdInspur Yes, but how did you get the OSDs to return an error instead of blocking? Is this a custom application that is also setting the "try_full" flag in librados?

@stale
Copy link

stale bot commented Dec 15, 2019

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

@stale stale bot added the stale label Dec 15, 2019
liwenpeng19 and others added 2 commits February 4, 2020 12:09
Signed-off-by: liwenpeng <liwenpeng@inspur.com>
@stale stale bot removed the stale label Feb 17, 2020
@stale
Copy link

stale bot commented Apr 17, 2020

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

@stale stale bot added the stale label Apr 17, 2020
@stale
Copy link

stale bot commented Jul 18, 2020

This pull request has been automatically closed because there has been no activity for 90 days. Please feel free to reopen this pull request (or open a new one) if the proposed change is still appropriate. Thank you for your contribution!

@stale stale bot closed this Jul 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants