[query] Fix data race in FASTA reader #9427
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In readBlock, membership was checked in a LinkedHashMap, then if it was found, the value was retrieved from the cache. This lead to a data race where a value could be evicted between the check and retrival. The solution is to grab the block value out of the map, and if it is not null, return it, otherwise, grab the block string, put it in the map, and return it. Co-authored-by: Tim Poterba <tpoterba@broadinstitute.org>
|
Did you run on dataproc? Could you replicate either the NPE or ClosedChannelException? |
|
The NPE, no. ClosedChannel, again, no. |
|
I did run it on Dataproc. |
|
huh, weird. So we maybe don't expect this to fix Julia's issue? |
|
I'm worried it's some other weird IO bug, but this should still fix the data race. |
patrick-schultz
approved these changes
Sep 10, 2020
pwc2
pushed a commit
to pwc2/hail
that referenced
this pull request
Sep 21, 2020
In readBlock, membership was checked in a LinkedHashMap, then if it was found, the value was retrieved from the cache. This lead to a data race where a value could be evicted between the check and retrival. The solution is to grab the block value out of the map, and if it is not null, return it, otherwise, grab the block string, put it in the map, and return it. Co-authored-by: Tim Poterba <tpoterba@broadinstitute.org> Co-authored-by: Tim Poterba <tpoterba@broadinstitute.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In readBlock, membership was checked in a LinkedHashMap, then if it was
found, the value was retrieved from the cache. This lead to a data race
where a value could be evicted between the check and retrival. The
solution is to grab the block value out of the map, and if it is not
null, return it, otherwise, grab the block string, put it in the map,
and return it.
Co-authored-by: Tim Poterba tpoterba@broadinstitute.org