Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

One sided payments #2422

Merged
merged 359 commits into from Nov 9, 2020
Merged

Conversation

SWvheerden
Copy link
Collaborator

Description

Updates the one sided payments with the latest dev tip

Motivation and Context

How Has This Been Tested?

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Feature refactor (No new feature or functional changes, but performance or technical debt improvements)
  • New Tests
  • Documentation

Checklist:

  • I'm merging against the development branch.
  • I ran cargo-fmt --all before pushing.
  • I ran cargo test successfully before submitting my PR.
  • I have squashed my commits into a single commit.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.

hansieodendaal and others added 30 commits September 30, 2020 16:03
Added 512-bit integer.
Added check_hash function.
Added difficulty to MoneroData.
Added additional helper methods.
Fixed endianess for monero hashes.
Fixed previous submit height for non-proxy submit.
…list

docs: add cargo test requirement to PR checklist
…ps from tari_core

Replaced tari_broadcast_channel with tokio broadcast channel.

Dependencies removed from tari_core:
arrayref
base64
cfg-if
config (moved to dev-dependencies)
crossbeam-channel
lazy_static
rmp-serde
tower_service
…will also prevent spending dust in some cases

Preventing fee to be larger than the amount will also prevent spending dust in
some cases. Added a config option as a general setting to allow the fee to be
larger than the amount in order to enable spending dust for small valued
transactions.
- Change to using the product of Monero and Blake difficulties to determine the strongest chain. Previously this was the square root of the product.
- Refactor chain strength out of validators, since it was not a validator
- Add test blockchain for cleaner tests
- Add limit to find_header_with_achieved_difficulty
- Add test blockchain for cleaner tests
- Add fetch all orphans
- Added ncurses development dependency to README.md
- Removed duplicate text at the bottom of the file
The PR introduces peer syncing and network discovery to improve a node's
overall knowedge of the network.

- Added a state machine that is responsible for syncing peers in rounds
  according to the current node state and previous round.
- If the number of peers in the peer manager is below a threshold, the
  sync will attempt to sync (streamed by RPC) many peers at once. Once
  the threshold has been reached, a small number of peers are periodically
  requested from random peers.
- Event notifies other components when a peer sync round has completed.
- The DHT connectivity actor responds to this event by "refreshing" the
  neighbouring and random peer pools.
- Created base for mocking out RPC calls
- Created DHT mock
- Added some basic tests for network discovery
- Add chain strength comparer
- Update reorg logic to use strength comparer
- Add limit to find_header_with_achieved_difficulty
- Add test blockchain for cleaner tests
- Add fetch all orphans
The PR introduces peer syncing and network discovery to improve a node's
overall knowledge of the network.

- Added a state machine that is responsible for syncing peers in rounds
  according to the current node state and previous round.
- If the number of peers in the peer manager is below a threshold, the sync will attempt
  to sync (streamed by RPC) many peers at once. Once the threshold has been reached,
 a small number of peers are periodically requested from random peers.
- Event notifies other components when a peer sync round has completed.
- The DHT connectivity actor responds to this event by "refreshing" the
  neighbouring and random peer pools.
- Created base for mocking out RPC calls
- Created DHT mock
- Added some basic tests for network discovery
PR adds in a way for keep track in the peer database why a peer was banned.
The MergeMiningError variant on the PowError enum is only compliled in
for `base_node` and `transactions` features.
Add migration to peer db,
Fix unit tests in peer manager
…x feature flag usage

Quick fix for compile error caused by complex feature flag usage
This PR provides a minimal example of how the service framework is used.
Currently the Output Manager Service in the wallet has 2 types of Output validation:
1. Validation Unspent outputs to see if they are still valid spendable outputs
2. Revalidate outputs marked as Invalid (from the previous validation

This PR adds a third type of validation which checks the wallet’s Spent outputs to see if any of them have become Unspent since the last time the wallet started up. This is done because it is possible that during a Reorg a transaction that the wallet thought was successful could be rolled back. In that case currently the wallet will never restore those funds. This validation will be able to restore the outputs spent in such a situation to spendable, returning them to the wallets Available balance.

A third kind TxoValidationType is added to the Txo Validation Protocol and it is started at the same time the Invalid UTXO revalidation protocol is started (i.e. on startup for the Base Node Wallet and when the Base Node Public key is set for the Wallet FFI library).

This PR does not do anything to the transactions associated with the outputs so the Wallets will still show the affected transaction as successful but at least the funds will become available again. Fixing the presentation of the affect Transaction will be the focus of a subsequent PR.
Fixes some compilation errors on development
Added proper handling of proof of work error in block sync.
…tput Manager Service

Currently the Output Manager Service in the wallet has 2 types of Output
validation:
1. Validation Unspent outputs to see if they are still valid
spendable outputs
2. Revalidate outputs marked as Invalid (from the previous
validation

This PR adds a third type of validation which checks the wallet’s Spent outputs
to see if any of them have become Unspent since the last time the wallet
started up. This is done because it is possible that during a Reorg a
transaction that the wallet thought was successful could be rolled back. In
that case currently the wallet will never restore those funds. This validation
will be able to restore the outputs spent in such a situation to spendable,
returning them to the wallets Available balance.

A third kind TxoValidationType is added to the Txo Validation Protocol and it
is started at the same time the Invalid UTXO revalidation protocol is started
(i.e. on startup for the Base Node Wallet and when the Base Node Public key is
set for the Wallet FFI library).

This PR does not do anything to the transactions associated with the outputs so
the Wallets will still show the affected transaction as successful but at least
the funds will become available again. Fixing the presentation of the affect
Transaction will be the focus of a subsequent PR.
…kiness

Added a check for the cancellation event which will help nail down where the test fails on CI. The test is not using a real comms stack and the operation its conducting is not time intensive so I cannot understand what is different on CI compared to local.
```
---- test::test_wallet_ffi stdout ----
thread 'test::test_wallet_ffi' panicked at 'called `Result::unwrap()` on an `Err` value: DialCancelled', base_layer\wallet_ffi\src\lib.rs:5812:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
... peer. ``` ---- test::test_wallet_ffi stdout ---- thread
'test::test_wallet_ffi' panicked at 'called `Result::unwrap()` on an `Err`
value: DialCancelled', base_layer\wallet_ffi\src\lib.rs:5812:13 note: run with
`RUST_BACKTRACE=1` environment variable to display a backtrace ```
One of the monero block headers in the current chain has quotes in them. This
PR will remove them and allow the node to sync past that block.
Byron Hambly and others added 28 commits October 28, 2020 15:05
Adds a base node service to the wallet, to start tracking a connected
base node's state. This will be required for the wallet to know the
base node status, chain height, and when last the information was
updated.

The service has a configurable refresh interval to call
`GetChainMetadata` on the base node.
INFO log messages must be on a single line
- This change adds configurable base node service request timeouts, notably
  differentiating between general service requests (default at 180s), requests
to fetch blocks for block sync (default at 30s) and requests for a complete
current UTXO set included in the blockchain (default at 600s).

- Removed a mempool service request timeout setting in the sample config files
  that was not used.
Correctly hook up mempool sync to comms.

Also took the opportunity to shorten the protocol identifier to conform with other protocol
identifiers.
Update the aes-gcm dependency in wallet, since the older version relied on a
now deprecated and unmaintained block-cipher crate.
… in pruned and archive mode.

Combine validation code to use same function in pruned and archive mode.
This PR adds an exclusive file locking mechanism to the wallet databases, the chain storage LMDB database and the Peer database. This is done to prevent two processes from simultaneously accessing this databases which can cause data corruption. 

This was done using the fs2 crate which contains a cross platform method for acquiring an OS level write lock on a file. For all these database the first step in initialization is to attempt to acquire a firelock on the lock file in the path where the database is located. If the file is not locked then a lock is acquired and the db initialisation proceeds as normal. If the file cannot be locked because the lock is already held by another instance an error is thrown. If the process holding the lock panics the OS level lock is lost so it is not needed to clean up the file explicitly for this method to work.

During testing this mechanism revealed that the wallet services were not being explicitly shutdown properly, we also plumb the shutdown signals into the services and their sub protocols.

The `test_store_and_forward_send_tx` test was being particularly flakey in the PR so I have opted to ignore it for now and come back and examine it properly in the future.


How was this tested:
Tests were written for the wallet and chain storage db’s but it was not trivial to test the peer_manager addition as it is located in the p2p initialiser.

The code was manually tested on:
Mac OS X
Windows 10
Ubuntu 20.01
Android
shaves some time off our circle ci, `test` is the longer task and now
`build` can run independently.
Fixes a subtle error where a valid `TransactionStatus` of 5 was
mistakenly reported as a conversion error. This caused the wallet to
fail on startup if coinbase transactions were in teh database.

The `TransactionStatus::try_from` method was missing the value.

Also added some additional info to conversion error, in case it appears
again it will be easier to narrow down.
…tatus

Fixes a subtle error where a valid `TransactionStatus` of 5 was
mistakenly reported as a conversion error. This caused the wallet to
fail on startup and when fetching coinbase transactions from the database.

The `TransactionStatus::try_from` method was missing the value.

Also added some additional info to conversion error, in case it appears
again it will be easier to narrow down.
Simplified the automated stress test so that it will also automatically
perform any required coin split beforehand. If not enough large valued
UTXOs are available for the coin split, it will create what is possible
and let the user know afterwards what the status is, without performing
the stress test.
Some information was outdated in these files and a large portion
of the example file was deleted by accident recently.
[ci] split build and test tasks on circle-ci
…d Peer db’s

This PR adds an exclusive file locking mechanism to the wallet databases, the
chain storage LMDB database and the Peer database. This is done to prevent two
processes from simultaneously accessing this databases which can cause data
corruption.

This was done using the fs2 crate which contains a cross platform method for
acquiring an OS level write lock on a file. For all these database the first
step in initialization is to attempt to acquire a firelock on the lock file in
the path where the database is located. If the file is not locked then a lock
is acquired and the db initialisation proceeds as normal. If the file cannot be
locked because the lock is already held by another instance an error is thrown.
If the process holding the lock panics the OS level lock is lost so it is not
needed to clean up the file explicitly for this method to work.

During testing this mechanism revealed that the wallet services were not being
explicitly shutdown properly, we also plumb the shutdown signals into the
services and their sub protocols.

The `test_store_and_forward_send_tx` test was being particularly flakey in the
PR so I have opted to ignore it for now and come back and examine it properly
in the future.
Adds a base node service to the wallet, to start tracking a connected base
node's state. This will be required for the wallet to know the base node sync
status, chain height, and when last the information was updated.

The service has a configurable refresh interval to call `GetChainMetadata` on
the base node, as well as a "maximum age" config for requests that remain
unanswered. Future improvements could add latency and other metrics.
- Simplified the automated stress test so that it will also automatically
  perform any required coin split beforehand. If not enough large valued UTXOs
are available for the coin split, it will create what is possible and let the
user know afterwards what the status is, without performing the stress test.

- Improved user feedback during initial phases of the test.
Starts application without CLI parser when --daemon-mode argument is present.
@CjS77 CjS77 merged commit 8c8dda2 into tari-project:one-sided-payments Nov 9, 2020
@SWvheerden SWvheerden deleted the one-sided-payments branch June 30, 2021 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants