Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use a single wallet batch for UpgradeKeyMetadata #15433

Merged

Conversation

Projects
None yet
7 participants
@jonasschnelli
Copy link
Member

commented Feb 18, 2019

Opening wallets (the first time) after #14021 took on my end around 30 seconds due to the keymetadata migration (tested on regtest).

Using a single wallet batch reduces the required time for the migration down to <1s on my system for a default 2k keypool wallet.

@jonasschnelli jonasschnelli force-pushed the jonasschnelli:2019/02/wallet_key_upgrade_batch branch from 204ad5c to 0bedcba Feb 18, 2019

@meshcollider meshcollider added this to the 0.18.0 milestone Feb 18, 2019

@meshcollider

This comment has been minimized.

Copy link
Member

commented Feb 18, 2019

Tagged for 0.18 because I'd consider this a bugfix

@Sjors

This comment has been minimized.

Copy link
Member

commented Feb 18, 2019

Concept ACK.

We should also consider doing that for importmulti (at least per descriptor).

@laanwj

This comment has been minimized.

Copy link
Member

commented Feb 18, 2019

utACK and concept ACK 0bedcba

@meshcollider

This comment has been minimized.

Copy link
Member

commented Feb 18, 2019

utACK 0bedcba

@achow101 want to take a look?

@promag
Copy link
Member

left a comment

Concept ACK.

On line 387, before throwing, shouldn't it commit the batch?

@MarcoFalke

This comment has been minimized.

Copy link
Member

commented Feb 18, 2019

@promag Wouldn't it doe that, as the unique ptr gets destroyed (and flush on exit was set to true by default)?

@promag

This comment has been minimized.

Copy link
Member

commented Feb 18, 2019

@MarcoFalke yap, it wasn't clear. Maybe worth a comment.

@jonasschnelli is there a rationale for 1000?

@achow101

This comment has been minimized.

Copy link
Member

commented Feb 18, 2019

utACK 0bedcba

@jonasschnelli

This comment has been minimized.

Copy link
Member Author

commented Feb 18, 2019

On line 387, before throwing, shouldn't it commit the batch?

As @MarcoFalke said, if the unique pointer gets out of the scope, it "commits" (flushes) to the database.

@jonasschnelli is there a rationale for 1000?

Not really. I thought 1000 is a good compromise between cache memory requirements and performance. A new constant looked after an overkill. Initially I wanted to tie it to the keypool size constant but somehow I thought then that this has nothing to do with memory consumption.

meshcollider added a commit to meshcollider/bitcoin that referenced this pull request Feb 18, 2019

Merge bitcoin#15433: Use a single wallet batch for UpgradeKeyMetadata
0bedcba Use a single wallet batch for UpgradeKeyMetadata (Jonas Schnelli)

Pull request description:

  Opening wallets (the first time) after bitcoin#14021 took on my end around 30 seconds due to the keymetadata migration (tested on regtest).

  Using a single wallet batch reduces the required time for the migration down to <1s on my system for a default 2k keypool wallet.

Tree-SHA512: f68739e452d382f5294186f47511b94884a1a0868688dd3179034a7e091a67f93bc9dd45cdfc9fa6b1fe90362772b719278012f2f56b752b803c87db8597a7b0

@meshcollider meshcollider merged commit 0bedcba into bitcoin:master Feb 18, 2019

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@MarcoFalke

This comment has been minimized.

Copy link
Member

commented Feb 18, 2019

1000 seems like a good upper bound, since the speedup flattens out after this:

(time in us)

upgradekeymetadata time

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.