[pull] master from bitcoin:master#174
Merged
pull[bot] merged 50 commits intocode-watch:masterfrom Oct 15, 2020
Merged
Conversation
In preparation for adding Schnorr versions of `CheckSig`, `VerifySignature`, and `ComputeEntry`, give them an ECDSA specific name. -BEGIN VERIFY SCRIPT- sed -i 's/CheckSig(/CheckECDSASignature(/g' $(git grep -l CheckSig ./src) sed -i 's/VerifySignature(/VerifyECDSASignature(/g' $(git grep -l VerifySignature ./src) sed -i 's/ComputeEntry(/ComputeEntryECDSA(/g' $(git grep -l ComputeEntry ./src) -END VERIFY SCRIPT-
The old name is confusing, as it doesn't store a scriptPubKey, but the actually executed script.
A BIP-341 signature message may commit to the scriptPubKeys and amounts of all spent outputs (including other ones than the input being signed for spends), so keep them available to signature hashing code.
This adds the TaggedHash function as defined by BIP340 to the hash module, which is used in BIP340 and BIP341 to produce domain-separated hashes.
This implements the new sighashing scheme from BIP341, with all relevant whole-transaction values precomputed once and cached. Includes changes to PrecomputedTransactionData by Pieter Wuille.
…BIP 340) This enables the schnorrsig module in libsecp256k1, adds the relevant types and functions to src/pubkey, as well as in higher-level `SignatureChecker` classes. The (verification side of the) BIP340 test vectors is also added.
This includes key path spending and script path spending, but not the Tapscript execution implementation (leaf 0xc0 remains unemcumbered in this commit). Includes constants for various aspects of the consensus rules suggested by Jeremy Rubin.
Instead of recomputing the annex hash every time a signature is verified, compute it once and cache it in a new ScriptExecutionData structure.
This adds a new `SigVersion::TAPSCRIPT`, makes the necessary interpreter changes to make it implement BIP342, and uses them for leaf version 0xc0 in Taproot script path spends.
This adds a `TxoutType::WITNESS_V1_TAPROOT` for P2TR outputs, and permits spending them in standardness rules. No corresponding `CTxDestination` is added for it, as that isn't needed until we want wallet integration. The taproot validation flags are also enabled for mempool transactions, and standardness rules are added (stack item size limit, no annexes).
Define a versionbits-based activation for the new consensus rules on regtest. No activation or activation mechanism is defined for testnet or mainnet.
Add a pure Python implementation of BIP340 signing and verification, tested against the BIP's test vectors.
A large functional test is added that automatically generates random transactions which exercise various aspects of the new rules, and verifies they are accepted into the mempool (when appropriate), and correctly accepted/rejected in (Python-constructed) blocks. Includes sighashing code and many tests by Johnson Lau. Includes a test by Matthew Zipkin. Includes several tests and improvements by Greg Sanders.
This adds a unit test that does generic script verification tests, with positive/negative witnesses/scriptsigs, under various flags. The test data is large (several MB) so it's stored in the qa-assets repo.
This adds a --dumptests flag to the feature_taproot.py test, to dump all its generated test cases to files, in a format compatible with the script_assets_test unit test. A fuzzer for said format is added as well, whose primary purpose is coverage-based minimization of those dumps.
sqlite3 recommends that sqlite3_initialize be called when the application starts, and sqlite3_shutdown when it stops. Since we don't always use sqlite3, we initialize it when a SQLiteDatabse is constructed (calling sqlite3_initialize after initialized is a no-op). We call sqlite3_shutdown when we see that there are no databases opened. The number of open databases is tracked by an atomic g_dbs_open.
Rewrite uses the VACUUM command which does exactly what we want. A specific advertised use case is to compact a database and ensure that any deleted data is actually deleted.
MakeWalletDatabase no longer has a default DatabaseFormat. Instead callers, like CWallet::Create, need to specify the database type to create if the file does not exist. If it exists and NONE is given, then CreateWalletDatabase will try to autodetect the type.
Descriptor wallets don't support dumpwallet, so make the tests that do dumpwallet legacy wallet only.
…d use it for new descriptor wallets c4a29d0 Update wallet_multiwallet.py for descriptor and sqlite wallets (Russell Yanofsky) 310b0fd Run dumpwallet for legacy wallets only in wallet_backup.py (Andrew Chow) 6c6639a Include sqlite3 in documentation (Andrew Chow) f023b7c wallet: Enforce sqlite serialized threading mode (Andrew Chow) 6173269 Set and check the sqlite user version (Andrew Chow) 9d3d2d2 Use network magic as sqlite wallet application ID (Andrew Chow) 9af5de3 Use SQLite for descriptor wallets (Andrew Chow) 9b78f3c walletutil: Wallets can also be sqlite (Andrew Chow) ac38a87 Determine wallet file type based on file magic (Andrew Chow) 6045f77 Implement SQLiteDatabase::MakeBatch (Andrew Chow) 727e6b2 Implement SQLiteDatabase::Verify (Andrew Chow) b4df8fd Implement SQLiteDatabase::Rewrite (Andrew Chow) 010e365 Implement SQLiteDatabase::TxnBegin, TxnCommit, and TxnAbort (Andrew Chow) ac5c161 Implement SQLiteDatabase::Backup (Andrew Chow) f6f9cd6 Implement SQLiteBatch::StartCursor, ReadAtCursor, and CloseCursor (Andrew Chow) bf90e03 Implement SQLiteBatch::ReadKey, WriteKey, EraseKey, and HasKey (Andrew Chow) 7aa4562 Add SetupSQLStatements (Andrew Chow) 6636a26 Implement SQLiteBatch::Close (Andrew Chow) 9382535 Implement SQLiteDatabase::Close (Andrew Chow) a0de833 Implement SQLiteDatabase::Open (Andrew Chow) 3bfa0fe Initialize and Shutdown sqlite3 globals (Andrew Chow) 5a488b3 Constructors, destructors, and relevant private fields for SQLiteDatabase/Batch (Andrew Chow) ca8b7e0 Implement SQLiteDatabaseVersion (Andrew Chow) 7577b6e Add SQLiteDatabase and SQLiteBatch dummy classes (Andrew Chow) e87df82 Add sqlite to travis and depends (Andrew Chow) 54729f3 Add libsqlite3 (Andrew Chow) Pull request description: This PR adds a new class `SQLiteDatabase` which is a subclass of `WalletDatabase`. This provides access to a SQLite database that is used to store the wallet records. To keep compatibility with BDB and to complexity of the change down, we don't make use of many SQLite's features. We use it strictly as a key-value store. We create a table `main` which has two columns, `key` and `value` both with the type `blob`. For new descriptor wallets, we will create a `SQLiteDatabase` instead of a `BerkeleyDatabase`. There is no requirement that all SQLite wallets are descriptor wallets, nor is there a requirement that all descriptor wallets be SQLite wallets. This allows for existing descriptor wallets to work as well as keeping open the option to migrate existing wallets to SQLite. We keep the name `wallet.dat` for SQLite wallets. We are able to determine which database type to use by searching for specific magic bytes in the `wallet.dat` file. SQLite begins it's files with a null terminated string `SQLite format 3`. BDB has `0x00053162` at byte 12 (note that the byte order of this integer depends on the system endianness). So when we see that there is a `wallet.dat` file that we want to open, we check for the magic bytes to determine which database system to use. I decided to keep the `wallet.dat` naming to keep things like backup script to continue to function as they won't need to be modified to look for a different file name. It also simplifies a couple of things in the implementation and the tests as `wallet.dat` is something that is specifically being looked for. If we don't want this behavior, then I do have another branch which creates `wallet.sqlite` files instead, but I find that this direction is easier. ACKs for top commit: Sjors: re-utACK c4a29d0 promag: Tested ACK c4a29d0. fjahr: reACK c4a29d0 S3RK: Re-review ACK c4a29d0 meshcollider: re-utACK c4a29d0 hebasto: re-ACK c4a29d0, only rebased since my [previous](#19077 (review)) review, verified with `git range-diff master d18892dcc c4a29d0`. ryanofsky: Code review ACK c4a29d0. I am honestly confused about reasons for locking into `wallet.dat` again when it's so easy now to use a clean format. I assume I'm just very dense, or there's some unstated reason, because the only thing that's been brought up are unrealistic compatibility scenarios (all require actively creating a wallet with non-default descriptor+sqlite option, then trying to using the descriptor+sqlite wallets with old software or scripts and ignoring the results) that we didn't pay attention to with previous PRs like #11687, which did not require any active interfaction. jonatack: ACK c4a29d0, debug builds and test runs after rebase to latest master @ c2c4dba, some manual testing creating, using, unloading and reloading a few different new sqlite descriptor wallets over several node restarts/shutdowns. Tree-SHA512: 19145732e5001484947352d3175a660b5102bc6e833f227a55bd41b9b2f4d92737bbed7cead64b75b509decf9e1408cd81c185ab1fb4b90561aee427c4f9751c
…ript) 0e2a5e4 tests: dumping and minimizing of script assets data (Pieter Wuille) 4567ba0 tests: add generic qa-asset-based script verification unit test (Pieter Wuille) f06e6d0 tests: functional tests for Schnorr/Taproot/Tapscript (Pieter Wuille) 3c22663 tests: add BIP340 Schnorr signature support to test framework (Pieter Wuille) 206fb18 --- [TAPROOT] Tests --- (Pieter Wuille) d7ff237 Activate Taproot/Tapscript on regtest (BIP 341, BIP 342) (Pieter Wuille) e9a021d Make Taproot spends standard + policy limits (Pieter Wuille) 865d2c3 --- [TAPROOT] Regtest activation and policy --- (Pieter Wuille) 72422ce Implement Tapscript script validation rules (BIP 342) (Johnson Lau) 330de89 Use ScriptExecutionData to pass through annex hash (Pieter Wuille) 8bbed4b Implement Taproot validation (BIP 341) (Pieter Wuille) 0664f5f Support for Schnorr signatures and integration in SignatureCheckers (BIP 340) (Pieter Wuille) 5de246c Implement Taproot signature hashing (BIP 341) (Johnson Lau) 9eb5908 Add TaggedHash function (BIP 340) (Pieter Wuille) 450d2b2 --- [TAPROOT] BIP340/341/342 consensus rules --- (Pieter Wuille) 5d62e3a refactor: keep spent outputs in PrecomputedTransactionData (Pieter Wuille) 8bd2b4e refactor: rename scriptPubKey in VerifyWitnessProgram to exec_script (Pieter Wuille) 107b57d scripted-diff: put ECDSA in name of signature functions (Pieter Wuille) f8c099e --- [TAPROOT] Refactors --- (Pieter Wuille) Pull request description: This is an implementation of the Schnorr/taproot consensus rules proposed by BIPs [340](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki), [341](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki), and [342](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki). See the list of commits [below](#19953 (comment)). No signing or wallet support of any kind is included, as testing is done entirely through the Python test framework. This is a successor to #17977 (see discussion following [this comment](#17977 (comment))), and will have further changes squashed/rebased. The history of this PR can be found in #19997. ACKs for top commit: instagibbs: reACK 0e2a5e4 benthecarman: reACK 0e2a5e4 kallewoof: reACK 0e2a5e4 jonasnick: ACK 0e2a5e4 almost only looked at bip340/libsecp related code jonatack: ACK 0e2a5e4 modulo the last four commits (tests) that I plan to finish reviewing tomorrow fjahr: reACK 0e2a5e4 achow101: ACK 0e2a5e4 Tree-SHA512: 1b00314450a2938a22bccbb4e177230cf08bd365d72055f9d526891f334b364c997e260c10bc19ca78440b6767712c9feea7faad9a1045dd51a5b96f7ca8146e
27fc6a3 DecodeHexTx: Break out transaction decoding logic into own function (Gregory Sanders) 6020ce3 DecodeHexTx: Try case where txn has inputs first (Gregory Sanders) Pull request description: Alternative/complementary to #17773 to avoid random `decoderawtransaction` failures. Most cases this is used now is on complete transactions, especially with the uptake of PSBT. ACKs for top commit: ajtowns: ACK 27fc6a3 achow101: ACK 27fc6a3 Tree-SHA512: 0a836d7c9951bf7d2764507788dbcc871d520f1ea9b77d6b22f051f4d6224ed779aba0e4f28c5c165040095ee0c70b67080c39164d82de61b19158f7ae6fddb2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]. Want to support this open source service? Please star it : )