Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: new transaction and output validation protocol #3421

Merged

Conversation

philipr-za
Copy link
Contributor

Description

This PR is to merge the tx-validation2 branch into development. This branch has been used to develop a new Output and Transaction validation protocols in the Wallet. This process relies on new functionality in the Base Node and as such for this branch’s features to work the seed nodes will need to be upgraded to use the code in this PR.

A summary of work in this feature branch:

Changes to the Base Node Wallet RPC service to support the new validation approach

  • Add query_deleted RPC method to allow the wallet to request whether an output is deleted i.e. spent and in which block it was deleted
  • Add utxo_query RPC method to allow for querying the status of a UTXO in more detail including what block it was mined in and the current tip state to aid in detecting reorgs
  • Add a new index to the LMDB database mapping deleted MMR position to block height and hash.

Changes to Wallet

  • Output validation is now a single protocol that validates spent, unspent and invalid outputs replacing three separate protocols.
  • We now store the block data (height and hash) of where an output was spent/received to make it possible to easily detect when a reorg has occurred. by iterating backwards through ordered outputs we can efficiently determine which outputs are affected by the reorg rather than revalidating all of them.
  • Transaction Validation has been updated to replace the old Chain Monitoring protocol. Again by adding block data to each transaction it becomes possible to more efficiently detect reorg events and which transactions are affected by the reorg. This means we only have to revalidate those transactions and not all of them.
  • We integrated the new Wallet Connectivity service into the new validation protocols to use the RPC connection pool instead of creating and managing our own RPC connections in the protocols.
  • Validations are now triggered when the Wallet Base Node Service becomes aware of a new block rather than on a periodic interval.

Wallet FFI

  • TXO validation now only has a single launching method and a single feedback callback.

How Has This Been Tested?

Cucumebr and Rust tests have been updated

stringhandler and others added 16 commits September 3, 2021 12:29
Merging development into this branch to get the latest code.
…elopment

chore: merge development into the `tx-validation2` feature branch
…ject#3352)

This PR cleans up and updates the new Output and Transaction validation strategies.

@mikethetike provided the first draft of this new validation approach for both Outputs and Transactions but the work needed some love to fully integrate into the wallet code. Many edge cases were caught and some bigger changes are made in this PR to make the process reliable.

Some major changes are:
- Output validation now supports checking and reporting on number of confirmations
- The Pending Transaction database table is fully removed and this process is now handled fully within the Outputs table
- Validations are now triggered on receiving a new block from the base node rather than at an interval
- Base node `query_deleted` call is updated to return the block data of where a deletion occured
…-project#3383)

* feat: update coinbase handling for new tx and output validation 

The new tx validation protocol would continuously try and revalidate abandoned coinbase transactions and the TXO validation would try validate abandoned or reforged Coinbases indefinitely.

This PR updates the Tx validation handing of Coinbases to give them the Coinbase status when they are abandoned which means they won’t be revalidated but will still be checked during a reorg. The TXO validation has been updated so that ab abandoned coinbase can be marked as Abandoned when the transaction validation detects the abandonment so that the output will not be validated indefinitely.

A new test is provided and a number of cucumber tests for this branch have been fixed.

* Fix typo
* refactor: reintroduce generic substream in client rpc

Generic substream was removed in order to more closely test RPC/yamux
interaction but that was incorrect. The tests should have just been
updated without removing the generic. This PR reintroduces them so that
an RPC client mock can more easily be created.

* refactor: use wallet connectivity in wallet services

- use wallet connectivity in broadcast protocol
- remove base node update code in multiple services
- use wallet connectivity in transaction validation protocol
- use wallet connectivity in TXO validation task
- update/fix tests
…ari-project#3394)

- add index to optimise query for block height from deleted mmr position
- update query_deleted rpc call to use new call
- check if database should be resynced to avoid confusion around
  db errors resulting from this change
- exit code message showing help for a required db resync
…oject#3400)

This PR adds back the ability to reinstate a CancelledInbound transaction outputs in the case that a cancelled inbound transaction gets a reply via SAF after its been cancelled.

The majority of this PR is actually fixing transaction service tests.

It also adds an extra db migration to clear existing `mined_height` values from the `completed_transaction` table to have all completed transactions get fully revalidated after a migration from an old db schema.
…ject#3417)

This PR brings the tx-validation2 branch up to Development and fixes the last issues in prep for merging the feature into Development.
Just uncommenting some sections of cucumber tests commented for debugging.
@stringhandler stringhandler merged commit 6578d1e into tari-project:development Oct 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants