Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge with Bitcoin 0.17, take 3 #1013

Merged
merged 2,214 commits into from
May 2, 2019
Merged

Merge with Bitcoin 0.17, take 3 #1013

merged 2,214 commits into from
May 2, 2019

Conversation

cmihai
Copy link
Member

@cmihai cmihai commented Apr 23, 2019

This PR applies the changes from upstream to Unit-e, up to version 0.17.1. Fixes #41.

Flaky tests we've found and working on:

  • mempool_persist, probably race between import and rpc threads but very strange. cannot reproduce locally be08361
  • wallet_multiwallet.py --usecli, uninformative logs — added more. ad7c2e7
  • wallet_labels crashes sometimes
  • rpc_getblockstats.py da99d1b

MarcoFalke and others added 30 commits August 13, 2018 07:41
Pre-0.17 branch hardcoded seeds update.
… Python tests scripts. Remove wildcard imports.

68400d8 tests: Use explicit imports (practicalswift)

Pull request description:

  Enable automatic detection of undefined names in Python tests scripts. Remove wildcard imports.

  Wildcard imports make it unclear which names are present in the namespace, confusing both readers and many automated tools.

  An additional benefit of not using wildcard imports in tests scripts is that readers of a test script then can infer the rough testing scope just by looking at the imports.

  Before this commit:

  ```
  $ contrib/devtools/lint-python.sh | head -10
  ./test/functional/feature_rbf.py:8:1: F403 'from test_framework.util import *' used; unable to detect undefined names
  ./test/functional/feature_rbf.py:9:1: F403 'from test_framework.script import *' used; unable to detect undefined names
  ./test/functional/feature_rbf.py:10:1: F403 'from test_framework.mininode import *' used; unable to detect undefined names
  ./test/functional/feature_rbf.py:15:12: F405 bytes_to_hex_str may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
  ./test/functional/feature_rbf.py:17:58: F405 CScript may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
  ./test/functional/feature_rbf.py:25:13: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
  ./test/functional/feature_rbf.py:26:31: F405 satoshi_round may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
  ./test/functional/feature_rbf.py:26:60: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
  ./test/functional/feature_rbf.py:30:41: F405 satoshi_round may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
  ./test/functional/feature_rbf.py:30:68: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
  $
  ```

  After this commit:

  ```
  $ contrib/devtools/lint-python.sh | head -10
  $
  ```

Tree-SHA512: 3f826d39cffb6438388e5efcb20a9622ff8238247e882d68f7b38609877421b2a8e10e9229575f8eb6a8fa42dec4256986692e92922c86171f750a0e887438d9
fad3898 doc: move-only release notes of individual prs (MarcoFalke)

Pull request description:

  Suggestion for reviewers:

  `git diff HEAD~ --color-moved=dimmed_zebra`

Tree-SHA512: 9d8be96d3a578e9df831eda1b5d942798ea4fbe2b1105f4c12a61a13716e62defd5ef8460fcd2411400849c057a8a1b3553ba2f490b70a353a6b46ce5be86acc
176aa5a contrib: fix GetDesirableServiceFlags typo in seeds/README.md (MarcoFalke)
825fb02 net: Update hardcoded seeds (Wladimir J. van der Laan)
61c0d3b contrib: in makeseeds, add 0.16.x to PATTERN_AGENT, remove 0.13.x (Wladimir J. van der Laan)

Pull request description:

  As per release process and https://github.com/bitcoin/bitcoin/blob/master/contrib/seeds/README.md

Tree-SHA512: c1acd43d8462196f2606babd5268ea75dbb844da729ac9592f08b4895c6a4754591ccbc6ac0bc364b2fb082695e72656154c141dd83c847db8973b3af8df74be
Update version after branching off.

Tree-SHA512: a2e2f82e7b50c0027edc8f382ad29429327edb81ca058abe3c35e049afcd5b7bcedae4545bfb59e8f7a1c8b9c97cfe550d8ae9275bd9e90cfadea22f7b835a2b
f0f745d doc: regenerate manpages (fanquake)

Pull request description:

  Done using: `contrib/devtools/gen-manpages.sh`

Tree-SHA512: 523b333768aa5ff289ceee3dbe627698f60c7b4624a1fe2812a40d99a5184aa2f6abd20fa467487918bbcbe1d88693c589456e75a5e747281333f75ab1f5f8b9
0-input transactions can be ambiguously deserialized as being witness
transactions. Since the unsigned transaction is never serialized as
a witness transaction as it has no witnesses, we should always
deserialize it as a non-witness transaction and set the serialization
flags as such.

Also always serialize the unsigned transaction as a non-witness transaction.

GitHub-Pull: #13960
Rebased-From: 43811e6
… as witness

Strip out the witnesses when serializing the non-witness utxo. However
witness serializations are allowed, so make sure we always deserialize
as witness.

GitHub-Pull: #13960
Rebased-From: bd19cc7
GitHub-Pull: #13917
Rebased-From: 8254e99
GitHub-Pull: #13917
Rebased-From: 5df6f08
0333914 More tests of signer checks (Andrew Chow)
8935869 Test that a non-witness script as witness utxo is not signed (Andrew Chow)
dbaadc9 Only wipe wrong UTXO type data if overwritten by wallet (Pieter Wuille)
ad6d845 Additional sanity checks in SignPSBTInput (Pieter Wuille)
517010e Serialize non-witness utxo as a non-witness tx but always deserialize as witness (Andrew Chow)
8c4cd2b Fix PSBT deserialization of 0-input transactions (Andrew Chow)

Pull request description:

  Backports #13917 and #13960 to the 0.17 branch.

Tree-SHA512: b3853aff2a13a53aa0a390b6b4b0c539f0ef0d42f2c517e956efd0b135c74c4ddce6a1d00700849a58c696824fa95951d8cac6ca58b426e8dfcb8bb62f680b7c
Tree-SHA512: 11d0d6a23f47e428661b33fa175aa97cc6841452c0c55845fdb0a903a0b147cd6df65e8fdab8b98823bf411018d5d85006af8c2cf14597286e9e284764d15041
Qt's configure grabs the path to xkb's data root during configure, but the
build changes in 5.8 apparently broke the handling for cross builds. As a
result, the string embedded in the binary depends on whether or not some files
are present in the builder's filesystem.

The "-xkb-config-root" configure setting is intended to allow manual overriding
but it is also broken. See: https://bugreports.qt.io/browse/QTBUG-60005

This has since been fixed upstream, so just hard-code the path for now. We can
drop this patch when we bump to a fixed Qt.

Also, fix the "-qt-xkbcommon-x11" config param which was renamed. This does not
appear to affect build results, presumably because auto-detection is working,
but it does not hurt to be explicit.

Github-Pull: #14000
Rebased-From: de0b4fb
48c8459 depends: fix qt determinism (Cory Fields)

Pull request description:

  Backport for rc2

Tree-SHA512: 990a1b32ca8c80b22595c4b19e801d5033e42b7e86da0f80028e708131a136f6383d7510dab8bd7cd12fe8533f73246fe78c72a8a22a54eb136b2bfada1e67ea
Github-Pull: #13968
Rebased-From: 2252ec5
Tree-SHA512: 1f9978ee25fbe9bb338af19d81b401fa531d314ac8288cdb21c1bf10459edea50b43e2d5e97c9bb5fe24c8db89363f8233c0a3d96066ed85f7bd6d2eb234aac0
Github-Pull: #13968
Rebased-From: 1f18d7b
Tree-SHA512: 90391703181db6880a135c60aca792a9e92c4abcad26907cd6cb0a0378593fe45cf995a22ae142ea7de2767c72a9df444e918ff15e460ce19c0435163917d812
Github-Pull: #13968
Rebased-From: 1f0c428
Tree-SHA512: 1f8b10629a314f623d589801ef2e62724de2cd82a0e523e0ab25a285f92b76a3b31304c1c0418b1b611ec3ca0016016d1e6af410ac81a78449b875c4f89a46d7
Github-Pull: #13968
Rebased-From: faaac5c
Tree-SHA512: 758c0c3e4435897d1a9b03ea93f1b2a1a1b64071eda9450f968acf537c172ee61acf9d962bc22ddb6de26e0ad39d9165cdee6f260bb5a95bf97b4003853f0874
Otherwise, the generated Makefile is included in the NSIS-installed documentation, which can lead to non-determinism (eg, if gawk is installed on some build VMs, but others only have mawk)

Github-Pull: #14018
Rebased-From: 8563341
Tree-SHA512: 2d219a4a2027bcd7359b7320bafc6b7cd3bde3dcf9309ddd6198ff67407470025baa71e6d0ed3d6cec081834ddc9a0247043865eb26737e6fd0d2f09574f5932
Github-Pull: #14071
Rebased-From: faf4a9b
…not thread safe

Github-Pull: #14069
Rebased-From: 737670c
cf3d7f9 Use assert when running from multithreaded code as BOOST_CHECK_* are not thread safe (Jesse Cohen)
fab0fbe qa: Stop txindex thread before calling destructor (MarcoFalke)
b5ec6d4 Docs: Fix help message typo optiona -> optional (Ben Woosley)

Pull request description:

  Fixes to make the unit tests and bench pass with the thread sanitizer (beside the issue with fChecked bitcoin/bitcoin#14058 (comment)).

  For testing: `./configure --with-sanitizers=undefined,thread && make -j 16 && ./src/test/test_bitcoin`

Tree-SHA512: 5cb85ecc278b719dba03240265e93424ed1a28671834da7590adab88c2d43c6e6cbf3269bbe2fd79e5ed3a85ec77a268e05301e7a7421cf6a97d413dddac6327
Signed-off-by: Stanislav Frolov <stanislav@thirdhash.com>
Signed-off-by: Stanislav Frolov <stanislav@thirdhash.com>
frolosofsky and others added 7 commits April 25, 2019 11:54
At the moment we raise an exception but its printer doesn't show stderr. This commit adds logging.

Signed-off-by: Stanislav Frolov <stanislav@thirdhash.com>
Signed-off-by: Stanislav Frolov <stanislav@thirdhash.com>
Signed-off-by: Stanislav Frolov <stanislav@thirdhash.com>
Signed-off-by: Mihai Ciumeica <mihai@thirdhash.com>
The test was trying to encrypt the wallet, then restart the node via RPC.
However, encrypting a wallet already forces a shutdown, and sometimes the
node would shut down its HTTP server before it received the "stop" command
from the test, which caused the test to fail.

The test framework already knew to handle this issue when using RPC. This
commit extends it to CLI.

Signed-off-by: Mihai Ciumeica <mihai@thirdhash.com>
cmihai and others added 6 commits April 30, 2019 11:55
Bitcoin 0.17 introduced a global flag `g_is_mempool_loaded`, which is
set after the import thread finishes, and is checked when saving the
mempool (because we can't save it if we haven't loaded it yet).

This commit adds the same check to `getrawmempool`, so that the functional
tests can at least wait until the flag is set before trying to act on it.

Signed-off-by: Mihai Ciumeica <mihai@thirdhash.com>
Signed-off-by: Andres Correa Casablanca <andres@thirdhash.com>
Signed-off-by: Andres Correa Casablanca <andres@thirdhash.com>
Signed-off-by: Andres Correa Casablanca <andres@thirdhash.com>
@cmihai cmihai requested review from scravy and a team May 2, 2019 09:24
Copy link
Member

@scravy scravy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should and can be bold and go on with merging this, to me it seems that it does not get any better than this. Everything which might not be covered needs to be covered and we're working on these things on master anyhow.

Regarding wallet_labels: Since it's quite flaky and will require a lot of fixing (or might even go away) when merging 0.18 (which we're about to do right after) I think it's fine to disable it on master and then take care of it in the efforts of merging 0.18.

Good-to-go ACK!

@cmihai cmihai merged commit 98191f2 into dtr-org:master May 2, 2019
@cmihai cmihai deleted the merge-bitcoin-0.17 branch May 2, 2019 09:40
scravy added a commit that referenced this pull request May 2, 2019
@cmihai cmihai restored the merge-bitcoin-0.17 branch May 2, 2019 09:45
scravy added a commit that referenced this pull request May 2, 2019
@scravy scravy mentioned this pull request May 2, 2019
4 tasks
castarco pushed a commit to castarco/unit-e that referenced this pull request May 2, 2019
Signed-off-by: Mihai Ciumeica <mihai@thirdhash.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
upstream sync Related to upstream merges
Projects
None yet
Development

Successfully merging this pull request may close these issues.

merge with bitcoin 0.17