New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[base-node] Optimise pruned UTXO sync streaming protocol #2857
[base-node] Optimise pruned UTXO sync streaming protocol #2857
Conversation
8397456
to
c40e532
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good.
I think the fn synchronize_outputs(
is a bit more complex.
I also don't like the use of the word block
in the logs for the pruned mode sync. This is technically incorrect and I think confusing as we never download blocks. I flag some but not all uses
...c/base_node/state_machine_service/states/horizon_state_sync/horizon_state_synchronization.rs
Outdated
Show resolved
Hide resolved
...c/base_node/state_machine_service/states/horizon_state_sync/horizon_state_synchronization.rs
Outdated
Show resolved
Hide resolved
...c/base_node/state_machine_service/states/horizon_state_sync/horizon_state_synchronization.rs
Outdated
Show resolved
Hide resolved
...c/base_node/state_machine_service/states/horizon_state_sync/horizon_state_synchronization.rs
Show resolved
Hide resolved
6d82210
to
9b7a5cd
Compare
Streams a pruned UTXO set to a client. Each block boundary (given by the header) is punctuated by a deleted MMR difference bitmap. Replaces the previous sync_rpc method, which has been removed and so, this PR is not backward-compatible.
9b7a5cd
to
fec72ad
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Happy with the approach, but I am concerned about the single DeletedBitmap and can't see how it maps to the MMR hash in the header in all cases.
...c/base_node/state_machine_service/states/horizon_state_sync/horizon_state_synchronization.rs
Show resolved
Hide resolved
@mikethetike The deleted difference bitmap is sent at the end fo each block boundary (as given by output mmr size). The only difference to what is sent from before is that instead of many difference bitmaps EDIT: EDIT2: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, happy
Changes since v0.8.8: Base Node --- - [#2870](#2870) [base-node] Fixes and tidies up prune mode cleanup - [#2857](#2857) [base-node] Optimise pruned UTXO sync streaming protocol - [#2868](#2868) [base-node] Test to reproduce target difficulty problem Wallet --- - [#2867](#2867) [wallet] Add option to write unblinded UTXOs to CSV file - [#2862](#2862) [wallet] Add mined height to display of mined transactions in Console Wallet Common --- - [#2876](#2876) [common] Fix incorrect boolean condition - [#2873](#2873) [tests] Fix case of cucumber - [#2837](#2837) [docs] Clarify one sided payment - [#2871](#2871) [tests] Update tests to use sha3 - [#2855](#2855) [tests] Fix re-org test
Description
Streams a pruned UTXO set to a client. Each block boundary (given by the
header) is punctuated by a deleted MMR difference bitmap.
Replaces the previous sync_rpc method, which has been removed and so,
this PR is not backward-compatible.
Motivation and Context
Continues from changes in #2847.
There are a few other performance optimisations still to be made for horizon sync.
How Has This Been Tested?
Manually tested. Sync took 15m.
Checklist:
development
branch.