New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HDFS-16970. EC: client copy wrong buffer from decode output during pread #5526
base: trunk
Are you sure you want to change the base?
Conversation
💔 -1 overall
This message was automatically generated. |
@tasanuma please review for this pr, it's a critical bug can cause wrong data when do preads. |
@luomh1998 Thanks for reporting the issue and submitting PR. I think the change is valid. Is it possible to add the unit test? |
Any update here @luomh1998 @tasanuma? |
Sorry, I will submit the related unit tests later. If you face the same problem, you can use this patch first. @bbeaudreault |
the newest version StripeReader in readToBuffer method will clear readbuffer in readToBuffer ,so will not read the data before timeout |
Description of PR
When dfsStripedInputStream do pread from a striped block group and read internal block timeout, so will read parity block for decode and fill original chunk buffer with decoded data.
Here try to fill original chunk buffer with decoded data, but get wrong data.
The reason is that
1.original chunk buffer already read some bytes before timeout from blockReader
2.chunkBytebuffer's slice always fill begin 0 position of decodeByteBuffer
slice bytebuffer will fill from wrong decodeByteBuffer position, so will get wrong data from pread.