New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow the user to skip verification of index integrity on startup #1485

Merged
merged 1 commit into from Nov 20, 2017

Conversation

3 participants
@shaan1337
Member

shaan1337 commented Nov 8, 2017

While loading midpoints from PTable index files during start up, the whole PTable file is read to also verify its integrity using an MD5 checksum. This can take a long time for very large databases/indexes.

This change allows the user to skip this verification by specifying a command line argument:
EventStore.ClusterNode.exe --skip-index-verify

Specifying that parameter will not read the whole file but will only read the midpoints by seeking directly to the midpoint locations in the file.

@gregoryyoung

This comment has been minimized.

Show comment
Hide comment
@gregoryyoung

gregoryyoung Nov 8, 2017

Member

It might be worth pairing this change with a lazy midpoint load which would prevent the scan entirely (but the reads would be slower until warmed up). In my testing just disabling the check had little to no effect on the overall time to load.

To do a lazy midpoint load just initialize all the midpoints to -1 when searching if -1 then do the read and populate.

Member

gregoryyoung commented Nov 8, 2017

It might be worth pairing this change with a lazy midpoint load which would prevent the scan entirely (but the reads would be slower until warmed up). In my testing just disabling the check had little to no effect on the overall time to load.

To do a lazy midpoint load just initialize all the midpoints to -1 when searching if -1 then do the read and populate.

@shaan1337 shaan1337 changed the title from Allow the user to disable verification of index integrity on startup to Allow the user to skip verification of index integrity on startup Nov 20, 2017

@hayley-jean hayley-jean merged commit 980bb6d into release-v4.0.4 Nov 20, 2017

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
wercker/build-mono4 Wercker pipeline passed
Details

@hayley-jean hayley-jean deleted the disable-index-verification branch Nov 20, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment