forked from dashpay/dash
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge #2278: [Refactor] Update RNG code from upstream
cecbf6c Use secure.h header for secure allocators (Fuzzbawls) d9f67da net: add ifaddrs.h include (fanquake) e906436 build: check if -lsocket is required with *ifaddrs (fanquake) 414f405 rand: only try and use freeifaddrs if available (fanquake) 3a039d6 build: avoid getifaddrs when unavailable (Cory Fields) 77bddd7 Use GetStrongRandBytes in gmp bignum initialization (Fuzzbawls) b70b26f Fix typo in comment in randomenv.cpp (Fuzzbawls) fec460c Put bounds on the number of CPUID leaves explored (Pieter Wuille) 41ab1ff Fix CPUID subleaf iteration (Pieter Wuille) 8a9bbb1 Move events_hasher into RNGState() (Pieter Wuille) 88c2ae5 random: mark RandAddPeriodic and SeedPeriodic as noexcept (fanquake) 81d382f doc: correct random.h docs after bitcoin#17270 (fanquake) f363ea9 Seed RNG with precision timestamps on receipt of net messages. (Matt Corallo) 7d6ddcb Run background seeding periodically instead of unpredictably (Pieter Wuille) 4679181 Add information gathered through getauxval() (Pieter Wuille) 88d97d0 Feed CPUID data into RNG (Pieter Wuille) 8f5b9c9 Use sysctl for seeding on MacOS/BSD (Pieter Wuille) 67de246 Gather additional entropy from the environment (Pieter Wuille) 6142e1f Seed randomness with process id / thread id / various clocks (Pieter Wuille) 7bde8b7 [MOVEONLY] Move cpuid code from random to compat/cpuid (Fuzzbawls) 52b5336 [MOVEONLY] Move perfmon data gathering to new randomenv module (Pieter Wuille) 27cf995 doc: minor corrections in random.cpp (fanquake) fccd2b8 doc: correct function name in ReportHardwareRand() (fanquake) 909473e Fix FreeBSD build by including utilstrencodings.h (Fuzzbawls) 630931f break circular dependency: random/sync -> util -> random/sync (Fuzzbawls) 5eed08c random: remove call to RAND_screen() (Windows only) (fanquake) ada9868 gui: remove OpenSSL PRNG seeding (Windows, Qt only) (fanquake) 22a7121 Fix non-deterministic coverage of test DoS_mapOrphans (Fuzzbawls) 79e7fd3 Add ChaCha20 bench (Jonas Schnelli) 6966aa9 Add ChaCha20 encryption option (XOR) (Jonas Schnelli) 28c9cdb tests: Add script checking for deterministic line coverage (practicalswift) c82e359 test: Make bloom tests deterministic (MarcoFalke) 7b33223 Document strenghtening (Pieter Wuille) 0190dec Add hash strengthening to the RNG (Pieter Wuille) 67e336d Use RdSeed when available, and reduce RdRand load (Pieter Wuille) 4ffda1f Document RNG design in random.h (Pieter Wuille) 2b6381e Use secure allocator for RNG state (Pieter Wuille) 080deb3 Encapsulate RNGState better (Pieter Wuille) 787d72f DRY: Implement GetRand using FastRandomContext::randrange (Pieter Wuille) 5bc2583 Sprinkle some sweet noexcepts over the RNG code (Pieter Wuille) 774899f Remove hwrand_initialized. (Pieter Wuille) 698d133 Switch all RNG code to the built-in PRNG. (Pieter Wuille) 038a45a Integrate util/system's CInit into RNGState (Fuzzbawls) 5f20e62 Abstract out seeding/extracting entropy into RNGState::MixExtract (Pieter Wuille) 298f97c Add thread safety annotations to RNG state (Pieter Wuille) 2326535 Rename some hardware RNG related functions (Pieter Wuille) d76ee83 Automatically initialize RNG on first use. (Pieter Wuille) 1a5dbc5 Don't log RandAddSeedPerfmon details (Pieter Wuille) 32e6c42 Simplify testing RNG code (Fuzzbawls) 972effa Make unit tests use the insecure_rand_ctx exclusively (Fuzzbawls) af52bf5 Use a FastRandomContext in LimitOrphanTxSize (Fuzzbawls) 746d466 Introduce a Shuffle for FastRandomContext and use it in wallet (Fuzzbawls) 1cdf124 Use a local FastRandomContext in a few more places in net (Fuzzbawls) e862564 Make addrman use its local RNG exclusively (Fuzzbawls) 94b2ead Make FastRandomContext support standard C++11 RNG interface (Pieter Wuille) Pull request description: This is a collection of upstream PRs that have been backported to bring our RNG (`src/random`) code more up-to-date. The following upstream PRs have been included here: - bitcoin#12742 - bitcoin#14624 - some of this had already been merged previously - bitcoin#14955 - bitcoin#15250 - bitcoin#15224 - bitcoin#15324 - bitcoin#15296 - bitcoin#15512 - bitcoin#16878 - bitcoin#17151 - bitcoin#17191 - bitcoin#13236 - bitcoin#13314 - bitcoin#17169 - bitcoin#17270 - omitted last commit as our testing framework doesn't support it currently - omitted bitcoin@64e1e02, to be pulled in after our time utility is updated in a separate PR - bitcoin#17573 - bitcoin#17507 - bitcoin#17670 - bitcoin#17527 - bitcoin#14127 - bitcoin#21486 ACKs for top commit: furszy: ACK cecbf6c with a minor nit that can be easily tackled later. random-zebra: rebase utACK cecbf6c and merging... Tree-SHA512: 3463b693cc9bddc1ec15228d264a794f5c2f159073fafa2ccf6e2563abfeb4369e49505f97ca84f2478ca792bd07b66d2cd83c58044d6a0cae6af42d22f5784b
- Loading branch information
Showing
34 changed files
with
1,695 additions
and
362 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| # Illumos/SmartOS requires linking with -lsocket if | ||
| # using getifaddrs & freeifaddrs | ||
|
|
||
| m4_define([_CHECK_SOCKET_testbody], [[ | ||
| #include <sys/types.h> | ||
| #include <ifaddrs.h> | ||
| int main() { | ||
| struct ifaddrs *ifaddr; | ||
| getifaddrs(&ifaddr); | ||
| freeifaddrs(ifaddr); | ||
| } | ||
| ]]) | ||
|
|
||
| AC_DEFUN([CHECK_SOCKET], [ | ||
| AC_LANG_PUSH(C++) | ||
| AC_MSG_CHECKING([whether ifaddrs funcs can be used without link library]) | ||
| AC_LINK_IFELSE([AC_LANG_SOURCE([_CHECK_SOCKET_testbody])],[ | ||
| AC_MSG_RESULT([yes]) | ||
| ],[ | ||
| AC_MSG_RESULT([no]) | ||
| LIBS="$LIBS -lsocket" | ||
| AC_MSG_CHECKING([whether getifaddrs needs -lsocket]) | ||
| AC_LINK_IFELSE([AC_LANG_SOURCE([_CHECK_SOCKET_testbody])],[ | ||
| AC_MSG_RESULT([yes]) | ||
| ],[ | ||
| AC_MSG_RESULT([no]) | ||
| AC_MSG_FAILURE([cannot figure out how to use getifaddrs]) | ||
| ]) | ||
| ]) | ||
| AC_LANG_POP | ||
| ]) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,134 @@ | ||
| #!/usr/bin/env bash | ||
| # | ||
| # Copyright (c) 2019 The Bitcoin Core developers | ||
| # Distributed under the MIT software license, see the accompanying | ||
| # file COPYING or http://www.opensource.org/licenses/mit-license.php. | ||
| # | ||
| # Test for deterministic coverage across unit test runs. | ||
|
|
||
| export LC_ALL=C | ||
|
|
||
| # Use GCOV_EXECUTABLE="gcov" if compiling with gcc. | ||
| # Use GCOV_EXECUTABLE="llvm-cov gcov" if compiling with clang. | ||
| GCOV_EXECUTABLE="gcov" | ||
|
|
||
| # Disable tests known to cause non-deterministic behaviour and document the source or point of non-determinism. | ||
| NON_DETERMINISTIC_TESTS=( | ||
| "dummy_tests" # We currently don't have any unit tests that have determinism issues | ||
| ) | ||
|
|
||
| TEST_BITCOIN_BINARY="src/test/test_pivx" | ||
|
|
||
| print_usage() { | ||
| echo "Usage: $0 [custom test filter (default: all but known non-deterministic tests)] [number of test runs (default: 2)]" | ||
| } | ||
|
|
||
| N_TEST_RUNS=2 | ||
| BOOST_TEST_RUN_FILTERS="" | ||
| if [[ $# != 0 ]]; then | ||
| if [[ $1 == "--help" ]]; then | ||
| print_usage | ||
| exit | ||
| fi | ||
| PARSED_ARGUMENTS=0 | ||
| if [[ $1 =~ [a-z] ]]; then | ||
| BOOST_TEST_RUN_FILTERS=$1 | ||
| PARSED_ARGUMENTS=$((PARSED_ARGUMENTS + 1)) | ||
| shift | ||
| fi | ||
| if [[ $1 =~ ^[0-9]+$ ]]; then | ||
| N_TEST_RUNS=$1 | ||
| PARSED_ARGUMENTS=$((PARSED_ARGUMENTS + 1)) | ||
| shift | ||
| fi | ||
| if [[ ${PARSED_ARGUMENTS} == 0 || $# -gt 2 || ${N_TEST_RUNS} -lt 2 ]]; then | ||
| print_usage | ||
| exit | ||
| fi | ||
| fi | ||
| if [[ ${BOOST_TEST_RUN_FILTERS} == "" ]]; then | ||
| BOOST_TEST_RUN_FILTERS="$(IFS=":"; echo "!${NON_DETERMINISTIC_TESTS[*]}" | sed 's/:/:!/g')" | ||
| else | ||
| echo "Using Boost test filter: ${BOOST_TEST_RUN_FILTERS}" | ||
| echo | ||
| fi | ||
|
|
||
| if ! command -v gcov > /dev/null; then | ||
| echo "Error: gcov not installed. Exiting." | ||
| exit 1 | ||
| fi | ||
|
|
||
| if ! command -v gcovr > /dev/null; then | ||
| echo "Error: gcovr not installed. Exiting." | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [[ ! -e ${TEST_BITCOIN_BINARY} ]]; then | ||
| echo "Error: Executable ${TEST_BITCOIN_BINARY} not found. Run \"./configure --enable-lcov\" and compile." | ||
| exit 1 | ||
| fi | ||
|
|
||
| get_file_suffix_count() { | ||
| find src/ -type f -name "*.$1" | wc -l | ||
| } | ||
|
|
||
| if [[ $(get_file_suffix_count gcno) == 0 ]]; then | ||
| echo "Error: Could not find any *.gcno files. The *.gcno files are generated by the compiler. Run \"./configure --enable-lcov\" and re-compile." | ||
| exit 1 | ||
| fi | ||
|
|
||
| get_covr_filename() { | ||
| echo "gcovr.run-$1.txt" | ||
| } | ||
|
|
||
| TEST_RUN_ID=0 | ||
| while [[ ${TEST_RUN_ID} -lt ${N_TEST_RUNS} ]]; do | ||
| TEST_RUN_ID=$((TEST_RUN_ID + 1)) | ||
| echo "[$(date +"%Y-%m-%d %H:%M:%S")] Measuring coverage, run #${TEST_RUN_ID} of ${N_TEST_RUNS}" | ||
| find src/ -type f -name "*.gcda" -exec rm {} \; | ||
| if [[ $(get_file_suffix_count gcda) != 0 ]]; then | ||
| echo "Error: Stale *.gcda files found. Exiting." | ||
| exit 1 | ||
| fi | ||
| TEST_OUTPUT_TEMPFILE=$(mktemp) | ||
| if ! BOOST_TEST_RUN_FILTERS="${BOOST_TEST_RUN_FILTERS}" ${TEST_BITCOIN_BINARY} > "${TEST_OUTPUT_TEMPFILE}" 2>&1; then | ||
| cat "${TEST_OUTPUT_TEMPFILE}" | ||
| rm "${TEST_OUTPUT_TEMPFILE}" | ||
| exit 1 | ||
| fi | ||
| rm "${TEST_OUTPUT_TEMPFILE}" | ||
| if [[ $(get_file_suffix_count gcda) == 0 ]]; then | ||
| echo "Error: Running the test suite did not create any *.gcda files. The gcda files are generated when the instrumented test programs are executed. Run \"./configure --enable-lcov\" and re-compile." | ||
| exit 1 | ||
| fi | ||
| GCOVR_TEMPFILE=$(mktemp) | ||
| if ! gcovr --gcov-executable "${GCOV_EXECUTABLE}" -r src/ > "${GCOVR_TEMPFILE}"; then | ||
| echo "Error: gcovr failed. Output written to ${GCOVR_TEMPFILE}. Exiting." | ||
| exit 1 | ||
| fi | ||
| GCOVR_FILENAME=$(get_covr_filename ${TEST_RUN_ID}) | ||
| mv "${GCOVR_TEMPFILE}" "${GCOVR_FILENAME}" | ||
| if grep -E "^TOTAL *0 *0 " "${GCOVR_FILENAME}"; then | ||
| echo "Error: Spurious gcovr output. Make sure the correct GCOV_EXECUTABLE variable is set in $0 (\"gcov\" for gcc, \"llvm-cov gcov\" for clang)." | ||
| exit 1 | ||
| fi | ||
| if [[ ${TEST_RUN_ID} != 1 ]]; then | ||
| COVERAGE_DIFF=$(diff -u "$(get_covr_filename 1)" "${GCOVR_FILENAME}") | ||
| if [[ ${COVERAGE_DIFF} != "" ]]; then | ||
| echo | ||
| echo "The line coverage is non-deterministic between runs. Exiting." | ||
| echo | ||
| echo "The test suite must be deterministic in the sense that the set of lines executed at least" | ||
| echo "once must be identical between runs. This is a necessary condition for meaningful" | ||
| echo "coverage measuring." | ||
| echo | ||
| echo "${COVERAGE_DIFF}" | ||
| exit 1 | ||
| fi | ||
| rm "${GCOVR_FILENAME}" | ||
| fi | ||
| done | ||
|
|
||
| echo | ||
| echo "Coverage test passed: Deterministic coverage across ${N_TEST_RUNS} runs." | ||
| exit |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.