Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

potential inconsistent data when map a clone volume by curve-nbd with 512 block-size #985

Closed
wu-hanqing opened this issue Jan 13, 2022 · 2 comments
Assignees
Labels
bug Something isn't working curvebs low low priority

Comments

@wu-hanqing
Copy link
Contributor

wu-hanqing commented Jan 13, 2022

Describe the bug (描述bug)

for a clone volume, each chunk in ChunkServer has a bitmap, and each bit represents whether a 4k range has been written after clone, which means clone chunk can only receive 4k aligned write requests.

in this case, when map a clone volume by curve-nbd with 512 block-size, and user issues a 512 bytes write request, curve-client firstly reads the full 4k range data, and then paste 512 bytes into full 4k data, and send a 4k write request to chunkserver.

so, when user issues two non-overlapping 512 bytes within the same 4k range concurrently, one of the two requests' data may lost.

To Reproduce (复现方法)

I haven't reproduce this problem, but in theory it's reproducible, later I will construct scenarios to verify this problem.

@wu-hanqing wu-hanqing added the bug Something isn't working label Jan 13, 2022
@wu-hanqing wu-hanqing self-assigned this Jan 13, 2022
@aspirer
Copy link
Contributor

aspirer commented Jan 13, 2022

may be add some sleep() in the IO flow can reproduce this issue. such as:
if (offset=0) { ## the first 512 bytes request
sleep 5s;
}
the send another 512 bytes request during the sleeping 5s but offset=1kb, and the second request will be overwritten the first one.

@wu-hanqing
Copy link
Contributor Author

wu-hanqing commented Jan 21, 2022

this code can reproduce current problem
https://github.com/wu-hanqing/curve/tree/issue-985

here is the output
image
image

@cw123 cw123 added this to the Curve-2.3.0-beta milestone Apr 18, 2022
@wuhongsong wuhongsong added the low low priority label May 9, 2022
@cw123 cw123 added the curvebs label Jul 4, 2022
@opencurveadmin opencurveadmin removed this from the Curve-2.3.0-beta milestone Jul 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working curvebs low low priority
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants