New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wallet Performance enhancements - cache commit + transaction fetching logic #2375

Merged
merged 7 commits into from Jan 14, 2019

Conversation

Projects
None yet
2 participants
@yeastplume
Copy link
Member

yeastplume commented Jan 14, 2019

Some performance enhancements that should greatly improve speed of larger wallet sets:

  • I'd removed the cached commit from the database as it was making the logic more complicated, however it also ends up slowing everything down if all commits have to be re-calculated all the time (such as during a grin wallet outputs. To add this 'cache' back in a more sensible way, I've simply added it to the stored WalletOutput struct as an optional field, that can be populated when an output is being created.

  • Because this leads to slightly worse privacy by having the commit in the database, (though no worse then when I removed the cache earlier,) I've added a flag to the wallet config to turn off commit caching (i.e. store None in the commit field of the WalletOutput'. You can elect to turn off commit caching at the expense of a performance hit.

  • Also simplified transaction retrieval logic to avoid looping through iterator many times and speeding up the check for whether an output needs to be updated from the server (via checking the TX log)

yeastplume added some commits Jan 14, 2019

@yeastplume yeastplume requested review from ignopeverell and antiochp Jan 14, 2019

@antiochp

This comment has been minimized.

Copy link
Member

antiochp commented Jan 14, 2019

Taking a look now.

@yeastplume how comfortable are you with these changes going in pre-mainnet vs. post-mainnet?
What risks are there in terms of introducing something unexpected here?
i.e. Is it worth the risk to do it now vs. in a few days?

@yeastplume

This comment has been minimized.

Copy link
Member Author

yeastplume commented Jan 14, 2019

I think it's best to get this one in before mainnet, given that the performance increase relies on the cached commit being available at output creation time. The wallet tests have grown to be fairly comprehensive, so I'd be very surprised if this introduced anything odd (though of course that can never be ruled out).

@@ -199,6 +199,17 @@ where
&mut self.w2n_client
}

/// return the commit, if enabled, None otherwise
fn cached_commit(&mut self, amount: u64, id: &Identifier) -> Result<Option<String>, Error> {

This comment has been minimized.

@antiochp

antiochp Jan 14, 2019

Member

Is this maybe mis-named or am I misunderstanding what is going on here?
Its not actually cached in here?

@antiochp
Copy link
Member

antiochp left a comment

I think this looks fine.
One minor comment.

👍

yeastplume added some commits Jan 14, 2019

@yeastplume

This comment has been minimized.

Copy link
Member Author

yeastplume commented Jan 14, 2019

Thanks for the review! renamed that function to try and make it a bit more clear

@yeastplume yeastplume merged commit 9a497f1 into mimblewimble:master Jan 14, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@yeastplume yeastplume deleted the yeastplume:cache_commit branch Jan 31, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment