Add fuzzing of crash recovery and fix a bunch of bugs #573
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cberner
force-pushed
the
crash_fuzz
branch
8 times, most recently
from
672a07c
to
bf603da
Compare
cberner
changed the title
cberner
force-pushed
the
crash_fuzz
branch
2 times, most recently
from
05c936b
to
be7c0f7
Compare
If an I/O error occurred during write(), the dirty page tracking would leak causing a panic when the transaction was committed/aborted
This fixes a cache poisoning issue in which a page might be in the userspace cache after walking the btrees during crash recovery, but then be freed by the rollback process of recovery. This could then poison the cache, leading to a crash in the future
Fixes a page leak if an I/O error occurred during commit(), since drop() would still flush the recovery bit to "clean"
Fixes a corruption issue that could occur if a Durability::None commit was made, followed by a durable commit, and the durable commit crashed during the call to commit(). Non-durable commits pushed their pending free pages into the freed table. The next durable commit then processed these *before* finalizing its commit. If that durable commit crashed after processing the frees, but before finalizing, then all the non-durable commits are rolled back, but the freed pages have already been processed. This left the database in a corrupted state since those pages could get overwritten as part of finalizing the durable commit
An I/O error during writeback could cause the page being flushed to be lost
The root would end up set, but not the path, this lead to a panic in the drop() method
Every issue found has reproduced at < 10k, and the corpus only has items up to ~50k
If an I/O occurring during commit(), the transaction would then try to abort() which might panic since the data structures are left in an inconsistent state
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #547
Fixes #556
Fixes #564