Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(pruned mode)!: prune inputs, allow horizon sync resume and other fixes #3521

Merged
merged 22 commits into from Dec 3, 2021
Merged

fix(pruned mode)!: prune inputs, allow horizon sync resume and other fixes #3521

merged 22 commits into from Dec 3, 2021

Conversation

sdbondi
Copy link
Member

@sdbondi sdbondi commented Nov 1, 2021 •
edited

Description

  • delete inputs behind pruning horizon during pruned mode cleanup
  • keep cumulative utxo_sum and kernel_sum in BlockAccumulatedData
  • prune spent outputs that may have not been pruned in previous horizon sync
  • bugfix: check rangeproof validation result in horizon sync (failed validation was ignored)
  • parallelize rangeproof verification for horizon sync
  • prune blocks between new and old horizon height if necessary before syncing
  • fix off-by-one in pruning horizon (pruning horizon is consistently defined as the last pruned block inclusive)
  • minor optimisations in horizon sync
  • new blockchain db unit tests

Motivation and Context

Inputs behind the pruning horizon were not pruned as propagated blocks arrive.
Breaking change: Blockchain database will need to be resynced

Based on #3520

How Has This Been Tested?

Cucumber tests pass
Manually, horizon sync to tip and wallet recovery

@sdbondi sdbondi changed the title fix(pruned mode): prune inputs, keep track of kernel/utxo sum fix(pruned mode)!: prune inputs, keep track of kernel/utxo sum Nov 1, 2021
@sdbondi sdbondi force-pushed the core-prune-inputs branch 7 times, most recently from 5a2a8b3 to f476d4c Compare November 1, 2021 13:36
@sdbondi sdbondi marked this pull request as draft November 1, 2021 14:27
@sdbondi sdbondi added the P-do_not_merge Process - Not ready for merging label Nov 1, 2021
@sdbondi sdbondi marked this pull request as ready for review November 2, 2021 06:49
@sdbondi sdbondi removed the P-do_not_merge Process - Not ready for merging label Nov 2, 2021
@sdbondi sdbondi marked this pull request as draft November 2, 2021 07:28
@sdbondi sdbondi marked this pull request as ready for review November 2, 2021 09:30
@sdbondi
re-add the commiment sum calculation at end of horizon sync
25acb67
@sdbondi
Merge branch 'development' into core-prune-inputs
7ec1adf
@sdbondi
Merge branch 'development' into core-prune-inputs
96e40ae
@sdbondi
Merge branch 'development' into core-prune-inputs
2da4991 
* development: (32 commits)
  feat: add atomic swap refund transaction handling (tari-project#3573)
  feat: improve wallet connectivity status for console wallet (tari-project#3577)
  v0.21.1
  feat: add error codes to LibWallet for CipherSeed errors (tari-project#3578)
  ci: split cucumber job into two (tari-project#3583)
  feat(wallet): import utxo’s as EncumberedToBeReceived rather than Unspent (tari-project#3575)
  docs: rfc 0250_Covenants (tari-project#3574)
  feat: get fee for transactions for stratum transcoder (tari-project#3571)
  test: make monerod stagenet usage resilient (tari-project#3572)
  feat: add atomic swap htlc sending and claiming (tari-project#3552)
  feat: implement prometheus metrics for base node (tari-project#3563)
  feat: implement multiple read single write for sqlite (tari-project#3568)
  feat: trigger time lock balance update when block received (tari-project#3567)
  test: reduce cucumber ci to critical only (tari-project#3566)
  test: fix cucumber console wallet startup (tari-project#3564)
  chore: add node id/public key to log mdc (tari-project#3559)
  fix: avoid implicit using of the time crate (tari-project#3562)
  feat: one-click installer - cli edition (tari-project#3534)
  ci: add workflow dispatch to libwallet build action (tari-project#3556)
  fix: stop leak of value of recovered output (tari-project#3558)
  ...
@sdbondi
fix bad merge
c0e0717
@sdbondi
wip2
6e46b9e
@sdbondi sdbondi marked this pull request as draft November 19, 2021 09:41
@sdbondi
Merge branch 'development' into core-prune-inputs
461a1ec 
* development:
  feat: language detection for mnemonic seed words (tari-project#3590)
  chore: minor clippy fixes (tari-project#3576)
  fix: be more permissive of responses for the incorrect request_id (tari-project#3588)
  feat: track ping failures and disconnect (tari-project#3597)
  chore: upgrade tokio deps tari-project#3581 (tari-project#3595)
  feat: standardize output hash for unblinded output, transaction output and transaction input (tari-project#3592)
  fix: allow bullet proof value only rewinding off one-sided transaction (tari-project#3587)
  refactor: update miningcore repository links (tari-project#3593)
  refactor: clean up unwraps in wallet_ffi (tari-project#3585)
  fix: update daily test start times and seed phrase (tari-project#3584)
  fix: allow bullet proof value only rewinding in atomic swaps (tari-project#3586)
  v0.21.2
  fix: remove delay from last request latency call
@sdbondi
Merge branch 'development' into core-prune-inputs
4292b38 
* development:
  refactor: remove tari_common dependency from tari_comms (tari-project#3580)
@sdbondi
refactor: sync_utxos, sync_kernels and horizon sync
31a5b5b
@sdbondi sdbondi changed the title fix(pruned mode)!: prune inputs, keep track of kernel/utxo sum fix(pruned mode)!: prune inputs, sync_utxos fix Nov 25, 2021
@sdbondi
Merge branch 'development' into core-prune-inputs
88eeed0 
* development:
  feat: implement dht pooled db connection (tari-project#3596)
  feat: add page for detailed mempool in explorer (tari-project#3613)
  chore: add pub key in the dailes notify (tari-project#3612)
  feat: display network for console wallet (tari-project#3611)
  docs: update covenants links (tari-project#3614)
@sdbondi sdbondi marked this pull request as ready for review November 25, 2021 07:38
@sdbondi
prune outputs at end of horizon sync
4b1e913
@sdbondi
Merge branch 'development' into core-prune-inputs
3a48a44 
* development:
  feat: only trigger UTXO scanning when a new block event is received (tari-project#3620)
@sdbondi sdbondi force-pushed the core-prune-inputs branch 5 times, most recently from c4dc231 to f532271 Compare November 26, 2021 09:41
@sdbondi sdbondi changed the title fix(pruned mode)!: prune inputs, sync_utxos fix fix(pruned mode)!: prune inputs, allow horion sync resume and other fixes Nov 26, 2021
@sdbondi sdbondi changed the title fix(pruned mode)!: prune inputs, allow horion sync resume and other fixes fix(pruned mode)!: prune inputs, allow horizon sync resume and other fixes Nov 26, 2021
@aviator-app aviator-app bot merged commit a4341a0 into tari-project:development Dec 3, 2021
@sdbondi sdbondi deleted the core-prune-inputs branch December 4, 2021 06:17
sdbondi added a commit to sdbondi/tari that referenced this pull request Dec 6, 2021
@sdbondi
Merge branch 'development' into core-remove-double-db-lock
b5d5a5b 
* development: (29 commits)
  fix(pruned mode)!: prune inputs, allow horizon sync resume and other fixes (tari-project#3521)
  feat!: sending one-sided transactions in wallet_ffi (tari-project#3634)
  fix: use json 5 for tor identity (regression) (tari-project#3624)
  test: add operation_id to log messages (tari-project#3633)
  fix!: multiple monerod addresses in tari merge mining proxy (tari-project#3628)
  fix: get-peer command works with public key again (tari-project#3636)
  fix!: separate peer seeds to common.network (tari-project#3635)
  test: removed stress test log target (tari-project#3631)
  feat: removed transaction validation redundant events (tari-project#3630)
  feat: improve wallet responsiveness (tari-project#3625)
  feat: add bulletproof rewind profiling (tari-project#3618)
  fix!: console wallet grpc_console_wallet_addresss config (tari-project#3619)
  test: increase timeout in cucumber (tari-project#3621)
  chore: change status line (tari-project#3610)
  feat!: add tcp bypass settings for tor in wallet_ffi (tari-project#3615)
  feat: only trigger UTXO scanning when a new block event is received (tari-project#3620)
  feat: implement dht pooled db connection (tari-project#3596)
  feat: add page for detailed mempool in explorer (tari-project#3613)
  chore: add pub key in the dailes notify (tari-project#3612)
  feat: display network for console wallet (tari-project#3611)
  ...
sdbondi added a commit to sdbondi/tari that referenced this pull request Dec 6, 2021
@sdbondi
Merge branch 'development' into core-bad-blocks-list
17fa7f4 
* development:
  fix(pruned mode)!: prune inputs, allow horizon sync resume and other fixes (tari-project#3521)
@sdbondi sdbondi restored the core-prune-inputs branch February 3, 2022 05:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stringhandler stringhandler stringhandler approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@sdbondi @stringhandler