Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Twisted Based Wallet Implementation #1155
This PR is split across three repositories,
This is a very major change to the daemon and should be evaluated carefully for security, performance and reliability before making it into a release. Furthermore, this PR also ports lbrynet to Python 3 and is no longer compatible with Python 2. **(see explanation at end)
Here are the main points of interest that should be reviewed:
Coin (UTXO) Selection Algorithm
Torba implements the same coin selection algorithm as latest C++ Bitcoin. Please review the algorithm and tests for completeness and correctness:
Old wallet stored things either in JSON in the wallet file or in memory in Python dictoinaries and lists when running. This has been completely replaced by an SQLite storage solution. Please review the base torba DDL and then the LBRY extension adding extra claim related attributes to the
Getting the DDL right from the beginning is a good idea, because updating code is cheap but updating DDL is time consuming for us (writing migrations) and the users (waiting for migrations to finish running).
Initial Sync & Subsequent Starts
Faster wallet operation is the main reason for this PR; so it's important that the header and TX synchronization code paths fast and correct. Please review the following areas:
Simple Payment Verification (SPV)
Torba is called an SPV wallet because it verifies that TXs are in the blockchain without having a copy of the blockchain. This verification is predicated on having correct headers (or at least the headers and the TX should come from different parties that were unlikely to have conspired with each other).
SPV works by organizing TXs in a block into merkle trees, then headers contain the root hash of this merkle tree and you can separately, for each TX, request all of the hashes between your TX and the root hash. Then you just traverse up the tree from your TX hashing every two complementary nodes until you reach the root hash, if the TX is valid the root hash in the header should mach the root hash you calculated. Done.
Here is the implementation, please verify for correctness:
This feature is currently covered by integration tests and does not have dedicated unit tests.
Torba implements a general purpose extensible bitcoin script parser. LBRY extends this to add our 3 custom op codes.
Can't forget this one, the whole point we're all here :-D
There is a special
All The Things
Above is a list of some of the critical aspects that should get some review before we go to releases. There is a lot more code than outlined above, if you feel motivated please review all of it and provide feedback. I would greatly appreciate it if people find issues that we can fix now before the code goes live! Thanks!
Essentially this means all of
You can downoad automated builds of the new daemon here:
** Python 3 port was done because 1) https://pythonclock.org 2) torba's testing framework orchstr8 loads electrumx (lbryumx) and the wallet into a single Python process using the same asyncio reactor which makes debugging very nice (when you hit breakpoint, both server and client freeze), electrumx is Python 3 only, therefore torba had to straddle Python2/Python3 support to work in both
This is a pure twisted wallet implementation for lbry providing increased stability and performance; completely replaces
Depends on this pull request in lbryum-server: lbryio/lbryum-server#64
Each leaf task below is 5pt (about a day of work).
Core Functionality I1
Core Functionality I2
Core Functionality I3
Separate Issues Created After I3