Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use redb's two-phase write strategy in production #660

Merged
merged 1 commit into from
Oct 17, 2022
Merged

Conversation

casey
Copy link
Collaborator

@casey casey commented Oct 16, 2022

I benchmarked redb's two write strategies, checksum, which uses a checksum to recover the database in case of a crash, and two-phase, which guarantees that the header on disk does not need to be recovered in case of a crash, but which requires an additional fsync per commit.

Two-phase narrowly beat out checksum, completing in 27 hours vs 28 hours. This is likely because sync time is dominated by very large commits, where checksum calculation is more costly than an additional commit. I left the tests using the checksum write strategy, because it's faster in tests, at 5 seconds vs 4 seconds for the two-phase commit.

@cberner Nothing too surprising here, although I thought you'd be interested anyways.

checksum:
  start:
    Oct 13 06:01:45 ordinals.com ord-dev[652816]: [2022-10-13T06:01:45Z INFO  ord::index] Block 0 at 2009-01-03 18:15:05 UTC with 1 transactions…
  end:
    Oct 14 10:12:01 ordinals.com ord-dev[652816]: [2022-10-14T10:12:01Z INFO  ord::index] Block 758607 at 2022-10-14 10:04:47 UTC with 2788 transactions…
  time:
    28 hours, 11 minutes
  tests:
    4.1s

two-phase:
  start:
    Oct 14 17:53:13 ordinals.com ord-dev[667585]: [2022-10-14T17:53:13Z INFO  ord::index] Block 0 at 2009-01-03 18:15:05 UTC with 1 transactions…
  end:
    Oct 15 20:56:32 ordinals.com ord-dev[667585]: [2022-10-15T20:56:32Z INFO  ord::index] Block 758607 at 2022-10-14 10:04:47 UTC with 2788 transactions…
  time:
    27 hours, 3 minutes
  tests:
    5.1s

@casey casey requested a review from raphjaph October 16, 2022 03:34
@casey casey merged commit ca2e74a into master Oct 17, 2022
@casey casey deleted the two-phase-in-prod branch October 17, 2022 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants