Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

travis: Enable functional tests in the ThreadSanitizer (TSan) build job #14829

Merged
merged 3 commits into from Dec 18, 2018

Conversation

@practicalswift
Copy link
Contributor

@practicalswift practicalswift commented Nov 28, 2018

Enable functional tests in the ThreadSanitizer (TSan) build job.

This is a follow-up to @MarcoFalke's #14764 which added TSan but for unit tests only.

@DrahtBot
Copy link
Contributor

@DrahtBot DrahtBot commented Nov 28, 2018

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #14841 (consensus: Move CheckBlock() call to critical section by hebasto)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@@ -99,9 +99,9 @@ jobs:
DOCKER_NAME_TAG=ubuntu:16.04
PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libssl-dev libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev libprotobuf-dev protobuf-compiler libqrencode-dev"
NO_DEPENDS=1
RUN_FUNCTIONAL_TESTS=false # Disabled for now. TODO identify suppressions or exclude specific tests
FUNCTIONAL_TESTS_CONFIG="--exclude feature_block.py,p2p_invalid_messages.py"
Copy link
Member

@MarcoFalke MarcoFalke Nov 29, 2018

Why are they excluded?

Copy link
Contributor Author

@practicalswift practicalswift Dec 3, 2018

feature_block.py and p2p_invalid_message.py fail for some unknown reason. Unfortunately the error log is not very informative so I haven't figured out why or what suppression (if any) would solve it.

Copy link
Member

@MarcoFalke MarcoFalke Dec 3, 2018

They are timing out because the tsan slows down bitcoind so much. You'd probably have to up all the various timeouts (rpcwait and the timeout for polling loops)

Copy link
Member

@MarcoFalke MarcoFalke Dec 3, 2018

At least that is what I assume based on my tsan runs a few days ago

Copy link
Contributor Author

@practicalswift practicalswift Dec 3, 2018

@MarcoFalke Ah, I see. What do you think about doing that in a follow-up PR once this is merged? It would be nice to keep this initial PR minimal to get basic functional testing under Travis. Debugging Travis is quite time consuming so I'd rather split the task in two if possible.

.travis.yml Outdated
GOAL="install"
BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --with-gui=qt5 CPPFLAGS=-DDEBUG_LOCKORDER --with-sanitizers=thread --disable-hardening --disable-asm CC=clang CXX=clang++"
BITCOIN_CONFIG="--enable-zmq --disable-wallet --with-gui=qt5 --with-sanitizers=thread --disable-hardening --disable-asm CC=clang CXX=clang++"
Copy link
Member

@MarcoFalke MarcoFalke Nov 29, 2018

Why is the wallet and DEBUG_LOCKORDER disabled?

Copy link
Contributor Author

@practicalswift practicalswift Dec 3, 2018

For some strange reasons all functional wallets tests fail under Travis (wallet_basic, wallet_dump, etc.). The failure reason is not clear from the logging so I haven't been able to solve this yet.

Copy link
Contributor Author

@practicalswift practicalswift Dec 3, 2018

I've now re-added CPPFLAGS=-DDEBUG_LOCKORDER since it doesn't seem to cause any problems.

Copy link
Member

@MarcoFalke MarcoFalke Dec 3, 2018

Looks like they time out with

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.

https://travis-ci.org/MarcoFalke/bitcoin/jobs/462970867

which shouldn't happen, since we should be printing dots all the time. Not sure what is going on :(

test/sanitizer_suppressions/tsan Outdated Show resolved Hide resolved
@practicalswift practicalswift force-pushed the tsan branch 5 times, most recently from 92a27e0 to f7a33ab Dec 3, 2018
@practicalswift
Copy link
Contributor Author

@practicalswift practicalswift commented Dec 3, 2018

@MarcoFalke Please review the latest version.

I've now re-introduced CPPFLAGS=-DDEBUG_LOCKORDER.

However, I've been unable to make the Travis build pass without FUNCTIONAL_TESTS_CONFIG="--exclude feature_block.py,p2p_invalid_messages.py" and --disable-wallet.

.travis/test_06_script.sh Outdated Show resolved Hide resolved
@practicalswift
Copy link
Contributor Author

@practicalswift practicalswift commented Dec 15, 2018

@MarcoFalke @ken2812221 Updated. Please re-review :-)

@practicalswift practicalswift force-pushed the tsan branch 2 times, most recently from 320d80d to 5e5138a Dec 17, 2018
@MarcoFalke
Copy link
Member

@MarcoFalke MarcoFalke commented Dec 17, 2018

utACK 5e5138a

@practicalswift
Copy link
Contributor Author

@practicalswift practicalswift commented Dec 18, 2018

@MarcoFalke Added suppression (race:InterruptRPC, fix in #14993). Please re-review :-)

@MarcoFalke MarcoFalke merged commit eaf4070 into bitcoin:master Dec 18, 2018
2 checks passed
MarcoFalke added a commit that referenced this issue Dec 18, 2018
…(TSan) build job

eaf4070 Add suppression for InterruptRPC (fRPCRunning) data race (practicalswift)
5e5138a travis: Use trap and set -e errtrace (Chun Kuan Lee)
069752b build: Enable functional tests in the ThreadSanitizer (TSan) build job (practicalswift)

Pull request description:

  Enable functional tests in the ThreadSanitizer (TSan) build job.

  This is a follow-up to @MarcoFalke's #14764 which added TSan but for unit tests only.

Tree-SHA512: dcc24d311fa124772c3036b16c2bf94732ece36c3e22b4bb8fe941772e52157ab2b1a90b1880b81079c2eef2d344ca7e1da58324b75dbf82d16204d591ad49fb
MarcoFalke added a commit that referenced this issue Jul 2, 2020
…se of uninitialized memory

870f0cd build: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory (practicalswift)

Pull request description:

  Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory.

  First UBSan, then ASan followed by TSan... and now: yes, the wait is over -- **MSan is finally here!** :)

  Some historical context:
  * 2017: Continuous compilation with Clang Thread Safety analysis enabled (#10866, #10923)
  * 2018: Continuous testing with trapping on signed integer overflows (`-ftrapv`) (#12686)
  * 2018: Continuous testing of use of locale dependent functions (#13041)
  * 2018: Continuous testing of format strings (#13705)
  * 2018: Continuous compilation with MSVC `TreatWarningAsError` (#14151)
  * 2018: Continuous testing under UndefinedBehaviorSanitizer – UBSan (#14252, #14673, #17006)
  * 2018: Continuous testing under AddressSanitizer – ASan (#14794, #17205, #17674)
  * 2018: Continuous testing under ThreadSanitizer – TSan (#14829)
  * 2019: Continuous testing in an unsigned char environment (`-funsigned-char`) (#15134)
  * 2019: Continuous compile-time testing of assumptions we're making (#15391)
  * 2019: Continuous testing of fuzz test cases under Valgrind (#17633, #18159, #18166)
  * 2020: Finally... MemorySanitizer – MSAN! :)

  What is the next step? What tools should we add to CI to keep bugs from entering `master`? :)

ACKs for top commit:
  MarcoFalke:
    ACK 870f0cd

Tree-SHA512: 38327c8b75679d97d469fe42e704cacd1217447a5a603701dd8a58ee50b3be2c10248f8d68a479ed081c0c4b254589d3081c9183f991640b06ef689061f75578
@practicalswift practicalswift deleted the tsan branch Apr 10, 2021
vijaydasmp added a commit to vijaydasmp/dash that referenced this issue Sep 30, 2021
…itizer (TSan) build job

eaf4070 Add suppression for InterruptRPC (fRPCRunning) data race (practicalswift)
5e5138a travis: Use trap and set -e errtrace (Chun Kuan Lee)
069752b build: Enable functional tests in the ThreadSanitizer (TSan) build job (practicalswift)

Pull request description:

  Enable functional tests in the ThreadSanitizer (TSan) build job.

  This is a follow-up to @MarcoFalke's bitcoin#14764 which added TSan but for unit tests only.

Tree-SHA512: dcc24d311fa124772c3036b16c2bf94732ece36c3e22b4bb8fe941772e52157ab2b1a90b1880b81079c2eef2d344ca7e1da58324b75dbf82d16204d591ad49fb
vijaydasmp added a commit to vijaydasmp/dash that referenced this issue Oct 1, 2021
…itizer (TSan) build job

eaf4070 Add suppression for InterruptRPC (fRPCRunning) data race (practicalswift)
5e5138a travis: Use trap and set -e errtrace (Chun Kuan Lee)
069752b build: Enable functional tests in the ThreadSanitizer (TSan) build job (practicalswift)

Pull request description:

  Enable functional tests in the ThreadSanitizer (TSan) build job.

  This is a follow-up to @MarcoFalke's bitcoin#14764 which added TSan but for unit tests only.

Tree-SHA512: dcc24d311fa124772c3036b16c2bf94732ece36c3e22b4bb8fe941772e52157ab2b1a90b1880b81079c2eef2d344ca7e1da58324b75dbf82d16204d591ad49fb
vijaydasmp added a commit to vijaydasmp/dash that referenced this issue Oct 4, 2021
…etect use of uninitialized memory

870f0cd build: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory (practicalswift)

Pull request description:

  Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory.

  First UBSan, then ASan followed by TSan... and now: yes, the wait is over -- **MSan is finally here!** :)

  Some historical context:
  * 2017: Continuous compilation with Clang Thread Safety analysis enabled (bitcoin#10866, bitcoin#10923)
  * 2018: Continuous testing with trapping on signed integer overflows (`-ftrapv`) (bitcoin#12686)
  * 2018: Continuous testing of use of locale dependent functions (bitcoin#13041)
  * 2018: Continuous testing of format strings (bitcoin#13705)
  * 2018: Continuous compilation with MSVC `TreatWarningAsError` (bitcoin#14151)
  * 2018: Continuous testing under UndefinedBehaviorSanitizer – UBSan (bitcoin#14252, bitcoin#14673, bitcoin#17006)
  * 2018: Continuous testing under AddressSanitizer – ASan (bitcoin#14794, bitcoin#17205, bitcoin#17674)
  * 2018: Continuous testing under ThreadSanitizer – TSan (bitcoin#14829)
  * 2019: Continuous testing in an unsigned char environment (`-funsigned-char`) (bitcoin#15134)
  * 2019: Continuous compile-time testing of assumptions we're making (bitcoin#15391)
  * 2019: Continuous testing of fuzz test cases under Valgrind (bitcoin#17633, bitcoin#18159, bitcoin#18166)
  * 2020: Finally... MemorySanitizer – MSAN! :)

  What is the next step? What tools should we add to CI to keep bugs from entering `master`? :)

ACKs for top commit:
  MarcoFalke:
    ACK 870f0cd

Tree-SHA512: 38327c8b75679d97d469fe42e704cacd1217447a5a603701dd8a58ee50b3be2c10248f8d68a479ed081c0c4b254589d3081c9183f991640b06ef689061f75578
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants