[WIP] Core PMMR and API updates to support wallet restore #950
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
API calls work now.. wallet restore works to a point, will merge what's there now and address that in another PR. |
yeastplume
pushed a commit
that referenced
this pull request
Apr 13, 2018
* Fix issue where we have no metadata for a block (#938) when restarting node before initial sync completed * Avoid double-locking on add eviction. Fixes #936 * Fix 33c5a98 * Add support for DNS Seed (#940) * Add support for DNS Seed * Add port * Add seed.grin-tech.org * Remove duplicate IPs * minimal stratum server * Modifications for review comments. Move stratum test into its own file, move get_block() into its own rust module, use pool and chain only rather than the entire Miner object * rustfmt * cleanup * cleanup * Introduce extending_readonly to simplify a forcing and cancelling rollbacks (#945) readonly views of the txhashset * Add DNS Seed and make DNSSeed default (#942) * Add dns seed seeding type * Add grin-seed.owncrypto.de and make DNSSeed default * Add email address for each DNS Seed * [WIP] Core PMMR and API updates to support wallet restore (#950) * update pmmr to get batch of elements by insertion position * update pmmr to get batch of elements by insertion position * add api + chain calls to get traversed outputs back out * add api + chain calls to get traversed outputs back out * first pass getting wallet restore to work again with updated utxo-walking api * Update simulation.md * Fix Bus Error (core dumped) when validating fast sync txhashset (#956) This PR fixes #953 by introducing a lock for txhashet_write. It's not enough to synchronize access to in memory data, files also needs to be protected, so a general txhashset lock was introduced. * refactor grin crate into separate modules (#955) * Add total kernel offset to block api (#954) * minimal stratum server * Modifications for review comments. Move stratum test into its own file, move get_block() into its own rust module, use pool and chain only rather than the entire Miner object * rustfmt * cleanup * cleanup * Merge with grin_grin -> servers code reorg * Merge with grin_grin -> servers code reorg * add stratum server stats
dpc
pushed a commit
to dpc/grin
that referenced
this pull request
Oct 24, 2018
…le#950) * update pmmr to get batch of elements by insertion position * update pmmr to get batch of elements by insertion position * add api + chain calls to get traversed outputs back out * add api + chain calls to get traversed outputs back out * first pass getting wallet restore to work again with updated utxo-walking api
dpc
pushed a commit
to dpc/grin
that referenced
this pull request
Oct 24, 2018
* Fix issue where we have no metadata for a block (mimblewimble#938) when restarting node before initial sync completed * Avoid double-locking on add eviction. Fixes mimblewimble#936 * Fix 33c5a98 * Add support for DNS Seed (mimblewimble#940) * Add support for DNS Seed * Add port * Add seed.grin-tech.org * Remove duplicate IPs * minimal stratum server * Modifications for review comments. Move stratum test into its own file, move get_block() into its own rust module, use pool and chain only rather than the entire Miner object * rustfmt * cleanup * cleanup * Introduce extending_readonly to simplify a forcing and cancelling rollbacks (mimblewimble#945) readonly views of the txhashset * Add DNS Seed and make DNSSeed default (mimblewimble#942) * Add dns seed seeding type * Add grin-seed.owncrypto.de and make DNSSeed default * Add email address for each DNS Seed * [WIP] Core PMMR and API updates to support wallet restore (mimblewimble#950) * update pmmr to get batch of elements by insertion position * update pmmr to get batch of elements by insertion position * add api + chain calls to get traversed outputs back out * add api + chain calls to get traversed outputs back out * first pass getting wallet restore to work again with updated utxo-walking api * Update simulation.md * Fix Bus Error (core dumped) when validating fast sync txhashset (mimblewimble#956) This PR fixes mimblewimble#953 by introducing a lock for txhashet_write. It's not enough to synchronize access to in memory data, files also needs to be protected, so a general txhashset lock was introduced. * refactor grin crate into separate modules (mimblewimble#955) * Add total kernel offset to block api (mimblewimble#954) * minimal stratum server * Modifications for review comments. Move stratum test into its own file, move get_block() into its own rust module, use pool and chain only rather than the entire Miner object * rustfmt * cleanup * cleanup * Merge with grin_grin -> servers code reorg * Merge with grin_grin -> servers code reorg * add stratum server stats
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Given we can't rely on full blocks being present on all nodes, for functions such as wallet restore we need to way to return the contents of the UTXO set to a requesting caller in reasonably-sized chunks. The most logical way seems to be via returning UTXOs by their PMMR insertion index, so if there are N utxos, the caller can say 'get me utxos 0..99' .. scan them.. then say 'now get 100..199' etc...
However, because of pruning, it will be more practical to say 'get me the first 100 unpruned outputs starting at position 0, and then have the caller return the actual index of the last returned output for the next query, which the caller can repeat until it's up to the most recent output.
Going to add this in a few parts, with this PR adding the appropriate functions to PMMRs and the API, then do the wallet work separately.