Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Fix possible data race when committing block files #12696
It was recently pointed out to me that calling
In theory this means that the block index can race block persistence, as a poorly timed power outage could cause us to commit data to the block index that gets lost in the filesystem. The change here ensures that we call
I'm using a new autoconf macro as well,
Apparently Windows doesn't have an similar method of syncing filesystem metadata---I'm not an expert on that though.
Also not strictly related to this change, but I have been working on a lot of platform-specific PRs recently and want to refactor
referenced this pull request
Mar 15, 2018
While I was looking at this again (originally to commit the undo data as @donaloconnor suggested) I decided it would be nice if we could only sync the parent directory if we know there's actually a new block file on disk. The block files are 128 MB, so most of the time we're flushing we don't need to sync the parent directory.
The full logic for all of the paths that can cause a file to be created is really confusing. First I tried adding a global variable that tracks if a new file has been opened, so that we can only sync the directory when that's set. Then I was trying to understand the
Since this is pretty important logic I feel like some of these code paths at least need better comments. I am going to take another look through the code tomorrow and see if I can work it out. Maybe we end up always syncing the parent directory anyway since that's always safe, but I want to make sure I have a better grasp of all of the logic in this file.