Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

memcmp with constants that contain zero bytes are broken in GCC #20005

Closed
1 task
sipa opened this issue Sep 23, 2020 · 19 comments
Closed
1 task

memcmp with constants that contain zero bytes are broken in GCC #20005

sipa opened this issue Sep 23, 2020 · 19 comments

Comments

@sipa
Copy link
Member

sipa commented Sep 23, 2020

It appears that there is a bug in certain GCC releases (in the version 9 and 10 series) where an optimization step breaks correctness of memcmp when at least one of the arguments is a compile-time constant array that contains at least one zero byte.

GCC bug is here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95189. It was stumbled upon by @roconnor-blockstream in bitcoin-core/secp256k1#822. It is being tracked for libsecp256k1 in bitcoin-core/secp256k1#823.

I have verified that in some instances it also affects C++, and may even affect std::lexicographical_compare.

This may be relevant in some of our code (in particular, the CNetAddr IP range checking does comparisons with constants that contain zeroes, but perhaps more).

Solutions:

  • Build with -fno-builtin-memcmp, but we should measure performance impact.
  • Very carefully inspect the codebase for potential cases, and use a custom memcmp for those.

TODO:

  • Verify if compiler-generated memcmp calls may be affected as well
@sipa
Copy link
Member Author

sipa commented Sep 23, 2020

Another observation by @roconnor-blockstream: it seems memcmp(...) <= 0 is not affected, while memcmp(...) < 0.

@roconnor-blockstream
Copy link
Contributor

Technically my claim was, from my very limited understanding, the bug can only change non-zero results from memcpy to zero results. Still, even in this case if the result of memcmp(x,y) is changed from a 1 to a 0, the value of memcmp(...) <= 0 will change from false to true.

@elichai
Copy link
Contributor

elichai commented Sep 24, 2020

Here is the example of the bug appearing when using the > operator on 2 arrays: https://godbolt.org/z/j5en9d
Here it appears with std::lexicographical_compare: https://godbolt.org/z/a6scYK

-fno-builtin-memcmp Doesn't do anything for either of these (as they didn't call memcmp)

@sipa
Copy link
Member Author

sipa commented Sep 24, 2020

@elichai Great find. That's... unfortunate.

It appears that code paths that use memcmp for (in)equality checks (e.g. memcmp(...) == 0) are unaffected. I suspect this means that automatically generated operator== on simple classes/structs aren't affected either. @real-or-random also found evidence for this in the bug's code.

However, @elichai's finding means that -fno-builtin-memcmp is not enough, as there are C++ headers that directly invoke __builtin_memcmp. Some possibilities (we're not restricted to just one of them, though):

  • Outlaw compiling with affected compilers entirely (by introducing a sanity check, and a unit test)
  • Figure out exactly which GCC -f... optimization flag controls the presence of the bug, and disabled that (always, or selectively on known-bad compilers)
  • Find the subset of affected C++ headers, and remove them from the codebase (replace them with my_memcmp where needed or so).

@elichai
Copy link
Contributor

elichai commented Sep 24, 2020

Another worth mentioning is that arith_uint256 managed to dodge this: https://github.com/bitcoin/bitcoin/blob/master/src/arith_uint256.h#L217-L222

while manually going over all the -O2 flags in my GCC 10.2 I found that disabling these 5 flags make the bug disappear:
-fno-tree-dce -fno-tree-dominator-opts -fno-tree-fre -fno-tree-pre -fno-code-hoisting

quickly reading the description of these they don't sound like something we'd like to disable that quickly :(

@real-or-random
Copy link
Contributor

This is a patch against the broken GCC versions that emits a warning when the bug could be triggered:
https://gist.github.com/real-or-random/1548239059158370416dfdc6a866329a

When I compile the project with this patch, I don't get any warning. That's good news. But unfortunately, this is not perfeclty sound. It catches all known examples but it fails to catch the std::lexicographical_compare example in this thread.... Maybe the reason is buried somewhere in this function but I don't know. https://gcc.gnu.org/onlinedocs/gcc-10.2.0/libstdc++/api/a00596_source.html#l01592

@roconnor-blockstream
Copy link
Contributor

With

               location_t loc = EXPR_HAS_LOCATION(exp) ? EXPR_LOCATION (exp) : UNKNOWN_LOCATION;
               expanded_location s = expand_location (loc);
               fnotice(stderr, "%s:%d: Potentially miscompiled memcmp\n", s.file, s.line);

I get a not so usefully located message, but a message none the less:

/nix/store/034wplfrxzhmljmajhmyya2ahg4z0nbl-gcc-9.3.0/include/c++/9.3.0/bits/stl_algobase.h:940: Potentially miscompiled memcmp

@roconnor-blockstream
Copy link
Contributor

Okay, I think I've cracked the puzzle:

               location_t loc = EXPR_HAS_LOCATION(exp) ? EXPR_LOCATION (exp) : UNKNOWN_LOCATION;
               loc = expansion_point_location_if_in_system_header(loc);
               emit_diagnostic(DK_DEBUG, loc, 0, "Potentially miscompiled memcmp");
In file included from /nix/store/23rhjr5qzbvf4kr0nwb5q3lkbxxl62jq-gcc-9.3.0/include/c++/9.3.0/algorithm:61,
                 from test4.cpp:1:
/nix/store/23rhjr5qzbvf4kr0nwb5q3lkbxxl62jq-gcc-9.3.0/include/c++/9.3.0/bits/stl_algobase.h: In function ‘int square(std::array<unsigned char, 3>)’:
/nix/store/23rhjr5qzbvf4kr0nwb5q3lkbxxl62jq-gcc-9.3.0/include/c++/9.3.0/bits/stl_algobase.h:940:41: debug: Potentially miscompiled memcmp
  940 |      if (int __result = __builtin_memcmp(__first1, __first2, __len))
      |   

expansion_point_location_if_in_system_header probably doesn't play a role, but maybe it doesn't hurt either.

@roconnor-blockstream
Copy link
Contributor

I rebuilt bitcoin-0.20.1 (including libsecp256k1) using emit_diagnostic, and I also did not get any miscompiled memcmp messages.

@theuni
Copy link
Member

theuni commented Sep 30, 2020

I was curious to see how much work it would be to work around this, so I hacked up (and vendored) my libstdc++ headers in order to add detection for the guilty functions, and started a branch of workarounds.

The guilty v9 headers (which call __builtin_memcmp themselves) are:

  • bits/char_traits.h
  • bits/stl_algobase.h

v10 adds:

  • array (I haven't looked into this one yet)

The workarounds mostly involve creating comparators for containers to avoid the default lexicographical_compare and getting rid of some std::equal usage. Those are pretty straightforward. But then I discovered that many of the boost string parsing functions end up being guilty.

For example:
boost::split(vDeploymentParams, strDeployment, boost::is_any_of(":")); ends up internally calling std::__equal<true>::equal().

And while this might be a good excuse to get rid of some of boost usage, I'm afraid that the boost libs themselves are going to be a problem. And I don't think we want to be patching boost to fix this.

Even worse than boost, glibc and libstdc++ themselves are likely affected as well (for different reasons).

I can push up my partial work if anyone is interested, but it's not looking like this is a good way forward.

@theuni
Copy link
Member

theuni commented Sep 30, 2020

I suspect this means that automatically generated operator== on simple classes/structs aren't affected either.

That seems to be the case from what I've seen so far, but we do end up in __builtin_memcmp when using a map/set of an array/vector of bytes, as those will use std::lexicographical_compare by default for comparison. There are lots of these in psbt.h, for example.

Thankfully there's an easy way to specify our own comparator:

struct VecByteComp {
    bool operator()(const std::vector<unsigned char>& lhs, const std::vector<unsigned char>& rhs) const {
        //TODO: fill in
        return true;
    }
};

...

-        std::set<std::vector<unsigned char>> key_lookup;
+        std::set<std::vector<unsigned char>, VecByteComp> key_lookup;

But as I mentioned above, working around those doesn't get us very far :(

jjyr added a commit to nervosnetwork/capsule that referenced this issue Oct 2, 2020
@laanwj laanwj removed this from the 0.21.0 milestone Oct 8, 2020
maflcko pushed a commit that referenced this issue Feb 8, 2021
2ecaf21 gitian: remove execstack workaround for ricv64 & powerpc64le (fanquake)
5baff2b build: use focal in gitian descriptors (fanquake)

Pull request description:

  This PR changes the gitian descriptors to use Ubuntu Focal (20.04), over Bionic (18.04), moving from GCC 7.5 to GCC 8.4 for native Linux builds, mingw-w64 GCC 7.3 to mingw-w64 GCC 9.3 for Windows builds, while continuing to use GCC 8.4 for all cross builds and Clang 8.0.0 for macOS builds.

  It also drops the `-Wl,-z,noexecstack` workaround we've been using for the riscv64 and powerpc64le hosts, as it's no-longer needed. One new package is installed in the osx build, `libtinfo5`, as libtinfo5.so is required by our downloaded Clang 8.

  A bump to Focal will at least be required if we want to update to a newer Qt (5.15, #19716) for 22.0, as we need a newer version of [`g++-mingw-w64`](https://packages.ubuntu.com/focal/g++-mingw-w64-x86-64) and the [`mingw-w64`](https://mingw-w64.org/doku.php) headers. This can still be done while continuing to use GCC 8.4 for Linux builds (see below), however the newer `g++-mingw-w64` will be based off of GCC 9.3.

  **Some considerations**

  GCC 9 is affected by #20005 "memcmp with constants that contain zero bytes are broken in GCC", and the newer `g++-mingw-w64` will be based off of GCC 9.3.

  The `--no-*` variants of the Windows linker flags (i.e `--no-dynamicbase`) we use to [test our `security-check.py` script](https://github.com/bitcoin/bitcoin/blob/16b784d953365bb2d7ae65acd2b20a79ef8ba7b6/contrib/devtools/test-security-check.py#L53) are not patched into the mingw binutils in Focal (they have been re-added in Groovy (20.10)). This isn't currently an issue, however, we might add a call to `test-security-check` for Guix (#20980), and if we wanted to do the same for gitian, it would not work. Note how it's quite "easy" for us to apply the `--no-*` variant patch to our Guix build; it would be quite a bit harder to do in Gitian.

  Gitian Builds @ 2ecaf21

  #### Linux
  ```bash
  8882ea78486fbae4fac574b9089eb1107c6372d0dd7dfcda4f0f930576f9d6c1  bitcoin-2ecaf214331b-aarch64-linux-gnu-debug.tar.gz
  50a9e30943b4eee5163edff3331241e745ff32a2c4463c21a6fdc5986e2d0383  bitcoin-2ecaf214331b-aarch64-linux-gnu.tar.gz
  ec4e55a447fddf033fee33cd5f22bfeda3c3612f059194bcf6238859f7989d7a  bitcoin-2ecaf214331b-arm-linux-gnueabihf-debug.tar.gz
  444fe1b3b933c00bcbd4a9d86888cff3b61c1215b1debccd2843e842d1224777  bitcoin-2ecaf214331b-arm-linux-gnueabihf.tar.gz
  88e486ff465980dc1a4aab9687d142ec6f727ed2c52cf539f69db2877dee83b2  bitcoin-2ecaf214331b-powerpc64-linux-gnu-debug.tar.gz
  66144ac264c65cada9d86446e6026c85b04fb88198b8f41b42840f6031db3e6c  bitcoin-2ecaf214331b-powerpc64-linux-gnu.tar.gz
  34bcc13d78d929d575e34e77a6672f23ca7ea23230b28ec2eed563889352ba86  bitcoin-2ecaf214331b-powerpc64le-linux-gnu-debug.tar.gz
  b4c5f959664f3063df4330edfe343c17120eb6b556ee1c15c4aeb2c1c54ffd49  bitcoin-2ecaf214331b-powerpc64le-linux-gnu.tar.gz
  918fa72ab6f6ebce4e9663c93f72fe26651c260477cbb54749f7eb61438b5cc1  bitcoin-2ecaf214331b-riscv64-linux-gnu-debug.tar.gz
  f704f9f8c053ffe37d854e2e81e0f4c0614c435dad7f5d82518c681b73a76ae6  bitcoin-2ecaf214331b-riscv64-linux-gnu.tar.gz
  b59e3a62f1df9d79f30e916b3c9655f654036fe3a420040c53acc8dd9f4162c5  bitcoin-2ecaf214331b-x86_64-linux-gnu-debug.tar.gz
  a4dc9ca877cc97544e65db11be38406d16f15d74fcdcd2318bb92474729bc60d  bitcoin-2ecaf214331b-x86_64-linux-gnu.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  af6ebc91147778e4e6705eade62608dde4d6e60522d79087fa9129bdb7c01199  bitcoin-core-linux-22-res.yml
  ```

  #### Windows
  ```bash
  121a3970a6911cb8c453b2ce37d03f6cbb43333e29db8fa516c68563fb367f43  bitcoin-2ecaf214331b-win-unsigned.tar.gz
  6294e9efebe935092f9ba119dc60ad4094f18b51c4181324e54d3057524d6101  bitcoin-2ecaf214331b-win64-debug.zip
  5b5a236b63e67f5f6c07ad9aa716aa7b72fb63722c96798b332c6d164738f9cf  bitcoin-2ecaf214331b-win64-setup-unsigned.exe
  c1fa5894c5e02a201637567c80b9bde9024f44673dcd06fd4d489c1709179279  bitcoin-2ecaf214331b-win64.zip
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  665fd7eb61aed368150db58a254f15fb5efb51a4efa5abcc52571cb7a1a5de22  bitcoin-core-win-22-res.yml
  ```

  #### macOS
  ```bash
  6a1deae7662aa782baa82a42590f862c6bcdc4f4e38daa9b8c2a9eed1fbb5397  bitcoin-2ecaf214331b-osx-unsigned.dmg
  1ee843266e84928a4323fa255c833528c2617a2c9fd2f98fb26ba19bbfc1227b  bitcoin-2ecaf214331b-osx-unsigned.tar.gz
  097b64dadc167d8e5b733421bf1541a40760ad952990f7cf3f35adc6ae2616d0  bitcoin-2ecaf214331b-osx64.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  6e378fb543928e40c7119b96be6ff773d38506a9a888f8b02c7f1b8a0801a80e  bitcoin-core-osx-22-res.yml
  ```

ACKs for top commit:
  laanwj:
    Build script changes review ACK 2ecaf21

Tree-SHA512: 975d5830b787d2e08988f43cbc6e839294171c1d94c8219636308b05f9b77041421612ae67be24a631674670cfc9c2d96d8177f2b3158a78fc3deea19631febf
sidhujag pushed a commit to syscoin/syscoin that referenced this issue Feb 8, 2021
2ecaf21 gitian: remove execstack workaround for ricv64 & powerpc64le (fanquake)
5baff2b build: use focal in gitian descriptors (fanquake)

Pull request description:

  This PR changes the gitian descriptors to use Ubuntu Focal (20.04), over Bionic (18.04), moving from GCC 7.5 to GCC 8.4 for native Linux builds, mingw-w64 GCC 7.3 to mingw-w64 GCC 9.3 for Windows builds, while continuing to use GCC 8.4 for all cross builds and Clang 8.0.0 for macOS builds.

  It also drops the `-Wl,-z,noexecstack` workaround we've been using for the riscv64 and powerpc64le hosts, as it's no-longer needed. One new package is installed in the osx build, `libtinfo5`, as libtinfo5.so is required by our downloaded Clang 8.

  A bump to Focal will at least be required if we want to update to a newer Qt (5.15, bitcoin#19716) for 22.0, as we need a newer version of [`g++-mingw-w64`](https://packages.ubuntu.com/focal/g++-mingw-w64-x86-64) and the [`mingw-w64`](https://mingw-w64.org/doku.php) headers. This can still be done while continuing to use GCC 8.4 for Linux builds (see below), however the newer `g++-mingw-w64` will be based off of GCC 9.3.

  **Some considerations**

  GCC 9 is affected by bitcoin#20005 "memcmp with constants that contain zero bytes are broken in GCC", and the newer `g++-mingw-w64` will be based off of GCC 9.3.

  The `--no-*` variants of the Windows linker flags (i.e `--no-dynamicbase`) we use to [test our `security-check.py` script](https://github.com/bitcoin/bitcoin/blob/16b784d953365bb2d7ae65acd2b20a79ef8ba7b6/contrib/devtools/test-security-check.py#L53) are not patched into the mingw binutils in Focal (they have been re-added in Groovy (20.10)). This isn't currently an issue, however, we might add a call to `test-security-check` for Guix (bitcoin#20980), and if we wanted to do the same for gitian, it would not work. Note how it's quite "easy" for us to apply the `--no-*` variant patch to our Guix build; it would be quite a bit harder to do in Gitian.

  Gitian Builds @ 2ecaf21

  #### Linux
  ```bash
  8882ea78486fbae4fac574b9089eb1107c6372d0dd7dfcda4f0f930576f9d6c1  bitcoin-2ecaf214331b-aarch64-linux-gnu-debug.tar.gz
  50a9e30943b4eee5163edff3331241e745ff32a2c4463c21a6fdc5986e2d0383  bitcoin-2ecaf214331b-aarch64-linux-gnu.tar.gz
  ec4e55a447fddf033fee33cd5f22bfeda3c3612f059194bcf6238859f7989d7a  bitcoin-2ecaf214331b-arm-linux-gnueabihf-debug.tar.gz
  444fe1b3b933c00bcbd4a9d86888cff3b61c1215b1debccd2843e842d1224777  bitcoin-2ecaf214331b-arm-linux-gnueabihf.tar.gz
  88e486ff465980dc1a4aab9687d142ec6f727ed2c52cf539f69db2877dee83b2  bitcoin-2ecaf214331b-powerpc64-linux-gnu-debug.tar.gz
  66144ac264c65cada9d86446e6026c85b04fb88198b8f41b42840f6031db3e6c  bitcoin-2ecaf214331b-powerpc64-linux-gnu.tar.gz
  34bcc13d78d929d575e34e77a6672f23ca7ea23230b28ec2eed563889352ba86  bitcoin-2ecaf214331b-powerpc64le-linux-gnu-debug.tar.gz
  b4c5f959664f3063df4330edfe343c17120eb6b556ee1c15c4aeb2c1c54ffd49  bitcoin-2ecaf214331b-powerpc64le-linux-gnu.tar.gz
  918fa72ab6f6ebce4e9663c93f72fe26651c260477cbb54749f7eb61438b5cc1  bitcoin-2ecaf214331b-riscv64-linux-gnu-debug.tar.gz
  f704f9f8c053ffe37d854e2e81e0f4c0614c435dad7f5d82518c681b73a76ae6  bitcoin-2ecaf214331b-riscv64-linux-gnu.tar.gz
  b59e3a62f1df9d79f30e916b3c9655f654036fe3a420040c53acc8dd9f4162c5  bitcoin-2ecaf214331b-x86_64-linux-gnu-debug.tar.gz
  a4dc9ca877cc97544e65db11be38406d16f15d74fcdcd2318bb92474729bc60d  bitcoin-2ecaf214331b-x86_64-linux-gnu.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  af6ebc91147778e4e6705eade62608dde4d6e60522d79087fa9129bdb7c01199  bitcoin-core-linux-22-res.yml
  ```

  #### Windows
  ```bash
  121a3970a6911cb8c453b2ce37d03f6cbb43333e29db8fa516c68563fb367f43  bitcoin-2ecaf214331b-win-unsigned.tar.gz
  6294e9efebe935092f9ba119dc60ad4094f18b51c4181324e54d3057524d6101  bitcoin-2ecaf214331b-win64-debug.zip
  5b5a236b63e67f5f6c07ad9aa716aa7b72fb63722c96798b332c6d164738f9cf  bitcoin-2ecaf214331b-win64-setup-unsigned.exe
  c1fa5894c5e02a201637567c80b9bde9024f44673dcd06fd4d489c1709179279  bitcoin-2ecaf214331b-win64.zip
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  665fd7eb61aed368150db58a254f15fb5efb51a4efa5abcc52571cb7a1a5de22  bitcoin-core-win-22-res.yml
  ```

  #### macOS
  ```bash
  6a1deae7662aa782baa82a42590f862c6bcdc4f4e38daa9b8c2a9eed1fbb5397  bitcoin-2ecaf214331b-osx-unsigned.dmg
  1ee843266e84928a4323fa255c833528c2617a2c9fd2f98fb26ba19bbfc1227b  bitcoin-2ecaf214331b-osx-unsigned.tar.gz
  097b64dadc167d8e5b733421bf1541a40760ad952990f7cf3f35adc6ae2616d0  bitcoin-2ecaf214331b-osx64.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  6e378fb543928e40c7119b96be6ff773d38506a9a888f8b02c7f1b8a0801a80e  bitcoin-core-osx-22-res.yml
  ```

ACKs for top commit:
  laanwj:
    Build script changes review ACK 2ecaf21

Tree-SHA512: 975d5830b787d2e08988f43cbc6e839294171c1d94c8219636308b05f9b77041421612ae67be24a631674670cfc9c2d96d8177f2b3158a78fc3deea19631febf
UdjinM6 pushed a commit to UdjinM6/dash that referenced this issue Oct 23, 2021
2ecaf21 gitian: remove execstack workaround for ricv64 & powerpc64le (fanquake)
5baff2b build: use focal in gitian descriptors (fanquake)

Pull request description:

  This PR changes the gitian descriptors to use Ubuntu Focal (20.04), over Bionic (18.04), moving from GCC 7.5 to GCC 8.4 for native Linux builds, mingw-w64 GCC 7.3 to mingw-w64 GCC 9.3 for Windows builds, while continuing to use GCC 8.4 for all cross builds and Clang 8.0.0 for macOS builds.

  It also drops the `-Wl,-z,noexecstack` workaround we've been using for the riscv64 and powerpc64le hosts, as it's no-longer needed. One new package is installed in the osx build, `libtinfo5`, as libtinfo5.so is required by our downloaded Clang 8.

  A bump to Focal will at least be required if we want to update to a newer Qt (5.15, bitcoin#19716) for 22.0, as we need a newer version of [`g++-mingw-w64`](https://packages.ubuntu.com/focal/g++-mingw-w64-x86-64) and the [`mingw-w64`](https://mingw-w64.org/doku.php) headers. This can still be done while continuing to use GCC 8.4 for Linux builds (see below), however the newer `g++-mingw-w64` will be based off of GCC 9.3.

  **Some considerations**

  GCC 9 is affected by bitcoin#20005 "memcmp with constants that contain zero bytes are broken in GCC", and the newer `g++-mingw-w64` will be based off of GCC 9.3.

  The `--no-*` variants of the Windows linker flags (i.e `--no-dynamicbase`) we use to [test our `security-check.py` script](https://github.com/bitcoin/bitcoin/blob/16b784d953365bb2d7ae65acd2b20a79ef8ba7b6/contrib/devtools/test-security-check.py#L53) are not patched into the mingw binutils in Focal (they have been re-added in Groovy (20.10)). This isn't currently an issue, however, we might add a call to `test-security-check` for Guix (bitcoin#20980), and if we wanted to do the same for gitian, it would not work. Note how it's quite "easy" for us to apply the `--no-*` variant patch to our Guix build; it would be quite a bit harder to do in Gitian.

  Gitian Builds @ 2ecaf21

  #### Linux
  ```bash
  8882ea78486fbae4fac574b9089eb1107c6372d0dd7dfcda4f0f930576f9d6c1  bitcoin-2ecaf214331b-aarch64-linux-gnu-debug.tar.gz
  50a9e30943b4eee5163edff3331241e745ff32a2c4463c21a6fdc5986e2d0383  bitcoin-2ecaf214331b-aarch64-linux-gnu.tar.gz
  ec4e55a447fddf033fee33cd5f22bfeda3c3612f059194bcf6238859f7989d7a  bitcoin-2ecaf214331b-arm-linux-gnueabihf-debug.tar.gz
  444fe1b3b933c00bcbd4a9d86888cff3b61c1215b1debccd2843e842d1224777  bitcoin-2ecaf214331b-arm-linux-gnueabihf.tar.gz
  88e486ff465980dc1a4aab9687d142ec6f727ed2c52cf539f69db2877dee83b2  bitcoin-2ecaf214331b-powerpc64-linux-gnu-debug.tar.gz
  66144ac264c65cada9d86446e6026c85b04fb88198b8f41b42840f6031db3e6c  bitcoin-2ecaf214331b-powerpc64-linux-gnu.tar.gz
  34bcc13d78d929d575e34e77a6672f23ca7ea23230b28ec2eed563889352ba86  bitcoin-2ecaf214331b-powerpc64le-linux-gnu-debug.tar.gz
  b4c5f959664f3063df4330edfe343c17120eb6b556ee1c15c4aeb2c1c54ffd49  bitcoin-2ecaf214331b-powerpc64le-linux-gnu.tar.gz
  918fa72ab6f6ebce4e9663c93f72fe26651c260477cbb54749f7eb61438b5cc1  bitcoin-2ecaf214331b-riscv64-linux-gnu-debug.tar.gz
  f704f9f8c053ffe37d854e2e81e0f4c0614c435dad7f5d82518c681b73a76ae6  bitcoin-2ecaf214331b-riscv64-linux-gnu.tar.gz
  b59e3a62f1df9d79f30e916b3c9655f654036fe3a420040c53acc8dd9f4162c5  bitcoin-2ecaf214331b-x86_64-linux-gnu-debug.tar.gz
  a4dc9ca877cc97544e65db11be38406d16f15d74fcdcd2318bb92474729bc60d  bitcoin-2ecaf214331b-x86_64-linux-gnu.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  af6ebc91147778e4e6705eade62608dde4d6e60522d79087fa9129bdb7c01199  bitcoin-core-linux-22-res.yml
  ```

  #### Windows
  ```bash
  121a3970a6911cb8c453b2ce37d03f6cbb43333e29db8fa516c68563fb367f43  bitcoin-2ecaf214331b-win-unsigned.tar.gz
  6294e9efebe935092f9ba119dc60ad4094f18b51c4181324e54d3057524d6101  bitcoin-2ecaf214331b-win64-debug.zip
  5b5a236b63e67f5f6c07ad9aa716aa7b72fb63722c96798b332c6d164738f9cf  bitcoin-2ecaf214331b-win64-setup-unsigned.exe
  c1fa5894c5e02a201637567c80b9bde9024f44673dcd06fd4d489c1709179279  bitcoin-2ecaf214331b-win64.zip
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  665fd7eb61aed368150db58a254f15fb5efb51a4efa5abcc52571cb7a1a5de22  bitcoin-core-win-22-res.yml
  ```

  #### macOS
  ```bash
  6a1deae7662aa782baa82a42590f862c6bcdc4f4e38daa9b8c2a9eed1fbb5397  bitcoin-2ecaf214331b-osx-unsigned.dmg
  1ee843266e84928a4323fa255c833528c2617a2c9fd2f98fb26ba19bbfc1227b  bitcoin-2ecaf214331b-osx-unsigned.tar.gz
  097b64dadc167d8e5b733421bf1541a40760ad952990f7cf3f35adc6ae2616d0  bitcoin-2ecaf214331b-osx64.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  6e378fb543928e40c7119b96be6ff773d38506a9a888f8b02c7f1b8a0801a80e  bitcoin-core-osx-22-res.yml
  ```

ACKs for top commit:
  laanwj:
    Build script changes review ACK 2ecaf21

Tree-SHA512: 975d5830b787d2e08988f43cbc6e839294171c1d94c8219636308b05f9b77041421612ae67be24a631674670cfc9c2d96d8177f2b3158a78fc3deea19631febf
UdjinM6 pushed a commit to UdjinM6/dash that referenced this issue Oct 23, 2021
2ecaf21 gitian: remove execstack workaround for ricv64 & powerpc64le (fanquake)
5baff2b build: use focal in gitian descriptors (fanquake)

Pull request description:

  This PR changes the gitian descriptors to use Ubuntu Focal (20.04), over Bionic (18.04), moving from GCC 7.5 to GCC 8.4 for native Linux builds, mingw-w64 GCC 7.3 to mingw-w64 GCC 9.3 for Windows builds, while continuing to use GCC 8.4 for all cross builds and Clang 8.0.0 for macOS builds.

  It also drops the `-Wl,-z,noexecstack` workaround we've been using for the riscv64 and powerpc64le hosts, as it's no-longer needed. One new package is installed in the osx build, `libtinfo5`, as libtinfo5.so is required by our downloaded Clang 8.

  A bump to Focal will at least be required if we want to update to a newer Qt (5.15, bitcoin#19716) for 22.0, as we need a newer version of [`g++-mingw-w64`](https://packages.ubuntu.com/focal/g++-mingw-w64-x86-64) and the [`mingw-w64`](https://mingw-w64.org/doku.php) headers. This can still be done while continuing to use GCC 8.4 for Linux builds (see below), however the newer `g++-mingw-w64` will be based off of GCC 9.3.

  **Some considerations**

  GCC 9 is affected by bitcoin#20005 "memcmp with constants that contain zero bytes are broken in GCC", and the newer `g++-mingw-w64` will be based off of GCC 9.3.

  The `--no-*` variants of the Windows linker flags (i.e `--no-dynamicbase`) we use to [test our `security-check.py` script](https://github.com/bitcoin/bitcoin/blob/16b784d953365bb2d7ae65acd2b20a79ef8ba7b6/contrib/devtools/test-security-check.py#L53) are not patched into the mingw binutils in Focal (they have been re-added in Groovy (20.10)). This isn't currently an issue, however, we might add a call to `test-security-check` for Guix (bitcoin#20980), and if we wanted to do the same for gitian, it would not work. Note how it's quite "easy" for us to apply the `--no-*` variant patch to our Guix build; it would be quite a bit harder to do in Gitian.

  Gitian Builds @ 2ecaf21

  #### Linux
  ```bash
  8882ea78486fbae4fac574b9089eb1107c6372d0dd7dfcda4f0f930576f9d6c1  bitcoin-2ecaf214331b-aarch64-linux-gnu-debug.tar.gz
  50a9e30943b4eee5163edff3331241e745ff32a2c4463c21a6fdc5986e2d0383  bitcoin-2ecaf214331b-aarch64-linux-gnu.tar.gz
  ec4e55a447fddf033fee33cd5f22bfeda3c3612f059194bcf6238859f7989d7a  bitcoin-2ecaf214331b-arm-linux-gnueabihf-debug.tar.gz
  444fe1b3b933c00bcbd4a9d86888cff3b61c1215b1debccd2843e842d1224777  bitcoin-2ecaf214331b-arm-linux-gnueabihf.tar.gz
  88e486ff465980dc1a4aab9687d142ec6f727ed2c52cf539f69db2877dee83b2  bitcoin-2ecaf214331b-powerpc64-linux-gnu-debug.tar.gz
  66144ac264c65cada9d86446e6026c85b04fb88198b8f41b42840f6031db3e6c  bitcoin-2ecaf214331b-powerpc64-linux-gnu.tar.gz
  34bcc13d78d929d575e34e77a6672f23ca7ea23230b28ec2eed563889352ba86  bitcoin-2ecaf214331b-powerpc64le-linux-gnu-debug.tar.gz
  b4c5f959664f3063df4330edfe343c17120eb6b556ee1c15c4aeb2c1c54ffd49  bitcoin-2ecaf214331b-powerpc64le-linux-gnu.tar.gz
  918fa72ab6f6ebce4e9663c93f72fe26651c260477cbb54749f7eb61438b5cc1  bitcoin-2ecaf214331b-riscv64-linux-gnu-debug.tar.gz
  f704f9f8c053ffe37d854e2e81e0f4c0614c435dad7f5d82518c681b73a76ae6  bitcoin-2ecaf214331b-riscv64-linux-gnu.tar.gz
  b59e3a62f1df9d79f30e916b3c9655f654036fe3a420040c53acc8dd9f4162c5  bitcoin-2ecaf214331b-x86_64-linux-gnu-debug.tar.gz
  a4dc9ca877cc97544e65db11be38406d16f15d74fcdcd2318bb92474729bc60d  bitcoin-2ecaf214331b-x86_64-linux-gnu.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  af6ebc91147778e4e6705eade62608dde4d6e60522d79087fa9129bdb7c01199  bitcoin-core-linux-22-res.yml
  ```

  #### Windows
  ```bash
  121a3970a6911cb8c453b2ce37d03f6cbb43333e29db8fa516c68563fb367f43  bitcoin-2ecaf214331b-win-unsigned.tar.gz
  6294e9efebe935092f9ba119dc60ad4094f18b51c4181324e54d3057524d6101  bitcoin-2ecaf214331b-win64-debug.zip
  5b5a236b63e67f5f6c07ad9aa716aa7b72fb63722c96798b332c6d164738f9cf  bitcoin-2ecaf214331b-win64-setup-unsigned.exe
  c1fa5894c5e02a201637567c80b9bde9024f44673dcd06fd4d489c1709179279  bitcoin-2ecaf214331b-win64.zip
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  665fd7eb61aed368150db58a254f15fb5efb51a4efa5abcc52571cb7a1a5de22  bitcoin-core-win-22-res.yml
  ```

  #### macOS
  ```bash
  6a1deae7662aa782baa82a42590f862c6bcdc4f4e38daa9b8c2a9eed1fbb5397  bitcoin-2ecaf214331b-osx-unsigned.dmg
  1ee843266e84928a4323fa255c833528c2617a2c9fd2f98fb26ba19bbfc1227b  bitcoin-2ecaf214331b-osx-unsigned.tar.gz
  097b64dadc167d8e5b733421bf1541a40760ad952990f7cf3f35adc6ae2616d0  bitcoin-2ecaf214331b-osx64.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  6e378fb543928e40c7119b96be6ff773d38506a9a888f8b02c7f1b8a0801a80e  bitcoin-core-osx-22-res.yml
  ```

ACKs for top commit:
  laanwj:
    Build script changes review ACK 2ecaf21

Tree-SHA512: 975d5830b787d2e08988f43cbc6e839294171c1d94c8219636308b05f9b77041421612ae67be24a631674670cfc9c2d96d8177f2b3158a78fc3deea19631febf
UdjinM6 pushed a commit to UdjinM6/dash that referenced this issue Oct 23, 2021
2ecaf21 gitian: remove execstack workaround for ricv64 & powerpc64le (fanquake)
5baff2b build: use focal in gitian descriptors (fanquake)

Pull request description:

  This PR changes the gitian descriptors to use Ubuntu Focal (20.04), over Bionic (18.04), moving from GCC 7.5 to GCC 8.4 for native Linux builds, mingw-w64 GCC 7.3 to mingw-w64 GCC 9.3 for Windows builds, while continuing to use GCC 8.4 for all cross builds and Clang 8.0.0 for macOS builds.

  It also drops the `-Wl,-z,noexecstack` workaround we've been using for the riscv64 and powerpc64le hosts, as it's no-longer needed. One new package is installed in the osx build, `libtinfo5`, as libtinfo5.so is required by our downloaded Clang 8.

  A bump to Focal will at least be required if we want to update to a newer Qt (5.15, bitcoin#19716) for 22.0, as we need a newer version of [`g++-mingw-w64`](https://packages.ubuntu.com/focal/g++-mingw-w64-x86-64) and the [`mingw-w64`](https://mingw-w64.org/doku.php) headers. This can still be done while continuing to use GCC 8.4 for Linux builds (see below), however the newer `g++-mingw-w64` will be based off of GCC 9.3.

  **Some considerations**

  GCC 9 is affected by bitcoin#20005 "memcmp with constants that contain zero bytes are broken in GCC", and the newer `g++-mingw-w64` will be based off of GCC 9.3.

  The `--no-*` variants of the Windows linker flags (i.e `--no-dynamicbase`) we use to [test our `security-check.py` script](https://github.com/bitcoin/bitcoin/blob/16b784d953365bb2d7ae65acd2b20a79ef8ba7b6/contrib/devtools/test-security-check.py#L53) are not patched into the mingw binutils in Focal (they have been re-added in Groovy (20.10)). This isn't currently an issue, however, we might add a call to `test-security-check` for Guix (bitcoin#20980), and if we wanted to do the same for gitian, it would not work. Note how it's quite "easy" for us to apply the `--no-*` variant patch to our Guix build; it would be quite a bit harder to do in Gitian.

  Gitian Builds @ 2ecaf21

  #### Linux
  ```bash
  8882ea78486fbae4fac574b9089eb1107c6372d0dd7dfcda4f0f930576f9d6c1  bitcoin-2ecaf214331b-aarch64-linux-gnu-debug.tar.gz
  50a9e30943b4eee5163edff3331241e745ff32a2c4463c21a6fdc5986e2d0383  bitcoin-2ecaf214331b-aarch64-linux-gnu.tar.gz
  ec4e55a447fddf033fee33cd5f22bfeda3c3612f059194bcf6238859f7989d7a  bitcoin-2ecaf214331b-arm-linux-gnueabihf-debug.tar.gz
  444fe1b3b933c00bcbd4a9d86888cff3b61c1215b1debccd2843e842d1224777  bitcoin-2ecaf214331b-arm-linux-gnueabihf.tar.gz
  88e486ff465980dc1a4aab9687d142ec6f727ed2c52cf539f69db2877dee83b2  bitcoin-2ecaf214331b-powerpc64-linux-gnu-debug.tar.gz
  66144ac264c65cada9d86446e6026c85b04fb88198b8f41b42840f6031db3e6c  bitcoin-2ecaf214331b-powerpc64-linux-gnu.tar.gz
  34bcc13d78d929d575e34e77a6672f23ca7ea23230b28ec2eed563889352ba86  bitcoin-2ecaf214331b-powerpc64le-linux-gnu-debug.tar.gz
  b4c5f959664f3063df4330edfe343c17120eb6b556ee1c15c4aeb2c1c54ffd49  bitcoin-2ecaf214331b-powerpc64le-linux-gnu.tar.gz
  918fa72ab6f6ebce4e9663c93f72fe26651c260477cbb54749f7eb61438b5cc1  bitcoin-2ecaf214331b-riscv64-linux-gnu-debug.tar.gz
  f704f9f8c053ffe37d854e2e81e0f4c0614c435dad7f5d82518c681b73a76ae6  bitcoin-2ecaf214331b-riscv64-linux-gnu.tar.gz
  b59e3a62f1df9d79f30e916b3c9655f654036fe3a420040c53acc8dd9f4162c5  bitcoin-2ecaf214331b-x86_64-linux-gnu-debug.tar.gz
  a4dc9ca877cc97544e65db11be38406d16f15d74fcdcd2318bb92474729bc60d  bitcoin-2ecaf214331b-x86_64-linux-gnu.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  af6ebc91147778e4e6705eade62608dde4d6e60522d79087fa9129bdb7c01199  bitcoin-core-linux-22-res.yml
  ```

  #### Windows
  ```bash
  121a3970a6911cb8c453b2ce37d03f6cbb43333e29db8fa516c68563fb367f43  bitcoin-2ecaf214331b-win-unsigned.tar.gz
  6294e9efebe935092f9ba119dc60ad4094f18b51c4181324e54d3057524d6101  bitcoin-2ecaf214331b-win64-debug.zip
  5b5a236b63e67f5f6c07ad9aa716aa7b72fb63722c96798b332c6d164738f9cf  bitcoin-2ecaf214331b-win64-setup-unsigned.exe
  c1fa5894c5e02a201637567c80b9bde9024f44673dcd06fd4d489c1709179279  bitcoin-2ecaf214331b-win64.zip
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  665fd7eb61aed368150db58a254f15fb5efb51a4efa5abcc52571cb7a1a5de22  bitcoin-core-win-22-res.yml
  ```

  #### macOS
  ```bash
  6a1deae7662aa782baa82a42590f862c6bcdc4f4e38daa9b8c2a9eed1fbb5397  bitcoin-2ecaf214331b-osx-unsigned.dmg
  1ee843266e84928a4323fa255c833528c2617a2c9fd2f98fb26ba19bbfc1227b  bitcoin-2ecaf214331b-osx-unsigned.tar.gz
  097b64dadc167d8e5b733421bf1541a40760ad952990f7cf3f35adc6ae2616d0  bitcoin-2ecaf214331b-osx64.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  6e378fb543928e40c7119b96be6ff773d38506a9a888f8b02c7f1b8a0801a80e  bitcoin-core-osx-22-res.yml
  ```

ACKs for top commit:
  laanwj:
    Build script changes review ACK 2ecaf21

Tree-SHA512: 975d5830b787d2e08988f43cbc6e839294171c1d94c8219636308b05f9b77041421612ae67be24a631674670cfc9c2d96d8177f2b3158a78fc3deea19631febf
UdjinM6 pushed a commit to UdjinM6/dash that referenced this issue Dec 4, 2021
2ecaf21 gitian: remove execstack workaround for ricv64 & powerpc64le (fanquake)
5baff2b build: use focal in gitian descriptors (fanquake)

Pull request description:

  This PR changes the gitian descriptors to use Ubuntu Focal (20.04), over Bionic (18.04), moving from GCC 7.5 to GCC 8.4 for native Linux builds, mingw-w64 GCC 7.3 to mingw-w64 GCC 9.3 for Windows builds, while continuing to use GCC 8.4 for all cross builds and Clang 8.0.0 for macOS builds.

  It also drops the `-Wl,-z,noexecstack` workaround we've been using for the riscv64 and powerpc64le hosts, as it's no-longer needed. One new package is installed in the osx build, `libtinfo5`, as libtinfo5.so is required by our downloaded Clang 8.

  A bump to Focal will at least be required if we want to update to a newer Qt (5.15, bitcoin#19716) for 22.0, as we need a newer version of [`g++-mingw-w64`](https://packages.ubuntu.com/focal/g++-mingw-w64-x86-64) and the [`mingw-w64`](https://mingw-w64.org/doku.php) headers. This can still be done while continuing to use GCC 8.4 for Linux builds (see below), however the newer `g++-mingw-w64` will be based off of GCC 9.3.

  **Some considerations**

  GCC 9 is affected by bitcoin#20005 "memcmp with constants that contain zero bytes are broken in GCC", and the newer `g++-mingw-w64` will be based off of GCC 9.3.

  The `--no-*` variants of the Windows linker flags (i.e `--no-dynamicbase`) we use to [test our `security-check.py` script](https://github.com/bitcoin/bitcoin/blob/16b784d953365bb2d7ae65acd2b20a79ef8ba7b6/contrib/devtools/test-security-check.py#L53) are not patched into the mingw binutils in Focal (they have been re-added in Groovy (20.10)). This isn't currently an issue, however, we might add a call to `test-security-check` for Guix (bitcoin#20980), and if we wanted to do the same for gitian, it would not work. Note how it's quite "easy" for us to apply the `--no-*` variant patch to our Guix build; it would be quite a bit harder to do in Gitian.

  Gitian Builds @ 2ecaf21

  #### Linux
  ```bash
  8882ea78486fbae4fac574b9089eb1107c6372d0dd7dfcda4f0f930576f9d6c1  bitcoin-2ecaf214331b-aarch64-linux-gnu-debug.tar.gz
  50a9e30943b4eee5163edff3331241e745ff32a2c4463c21a6fdc5986e2d0383  bitcoin-2ecaf214331b-aarch64-linux-gnu.tar.gz
  ec4e55a447fddf033fee33cd5f22bfeda3c3612f059194bcf6238859f7989d7a  bitcoin-2ecaf214331b-arm-linux-gnueabihf-debug.tar.gz
  444fe1b3b933c00bcbd4a9d86888cff3b61c1215b1debccd2843e842d1224777  bitcoin-2ecaf214331b-arm-linux-gnueabihf.tar.gz
  88e486ff465980dc1a4aab9687d142ec6f727ed2c52cf539f69db2877dee83b2  bitcoin-2ecaf214331b-powerpc64-linux-gnu-debug.tar.gz
  66144ac264c65cada9d86446e6026c85b04fb88198b8f41b42840f6031db3e6c  bitcoin-2ecaf214331b-powerpc64-linux-gnu.tar.gz
  34bcc13d78d929d575e34e77a6672f23ca7ea23230b28ec2eed563889352ba86  bitcoin-2ecaf214331b-powerpc64le-linux-gnu-debug.tar.gz
  b4c5f959664f3063df4330edfe343c17120eb6b556ee1c15c4aeb2c1c54ffd49  bitcoin-2ecaf214331b-powerpc64le-linux-gnu.tar.gz
  918fa72ab6f6ebce4e9663c93f72fe26651c260477cbb54749f7eb61438b5cc1  bitcoin-2ecaf214331b-riscv64-linux-gnu-debug.tar.gz
  f704f9f8c053ffe37d854e2e81e0f4c0614c435dad7f5d82518c681b73a76ae6  bitcoin-2ecaf214331b-riscv64-linux-gnu.tar.gz
  b59e3a62f1df9d79f30e916b3c9655f654036fe3a420040c53acc8dd9f4162c5  bitcoin-2ecaf214331b-x86_64-linux-gnu-debug.tar.gz
  a4dc9ca877cc97544e65db11be38406d16f15d74fcdcd2318bb92474729bc60d  bitcoin-2ecaf214331b-x86_64-linux-gnu.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  af6ebc91147778e4e6705eade62608dde4d6e60522d79087fa9129bdb7c01199  bitcoin-core-linux-22-res.yml
  ```

  #### Windows
  ```bash
  121a3970a6911cb8c453b2ce37d03f6cbb43333e29db8fa516c68563fb367f43  bitcoin-2ecaf214331b-win-unsigned.tar.gz
  6294e9efebe935092f9ba119dc60ad4094f18b51c4181324e54d3057524d6101  bitcoin-2ecaf214331b-win64-debug.zip
  5b5a236b63e67f5f6c07ad9aa716aa7b72fb63722c96798b332c6d164738f9cf  bitcoin-2ecaf214331b-win64-setup-unsigned.exe
  c1fa5894c5e02a201637567c80b9bde9024f44673dcd06fd4d489c1709179279  bitcoin-2ecaf214331b-win64.zip
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  665fd7eb61aed368150db58a254f15fb5efb51a4efa5abcc52571cb7a1a5de22  bitcoin-core-win-22-res.yml
  ```

  #### macOS
  ```bash
  6a1deae7662aa782baa82a42590f862c6bcdc4f4e38daa9b8c2a9eed1fbb5397  bitcoin-2ecaf214331b-osx-unsigned.dmg
  1ee843266e84928a4323fa255c833528c2617a2c9fd2f98fb26ba19bbfc1227b  bitcoin-2ecaf214331b-osx-unsigned.tar.gz
  097b64dadc167d8e5b733421bf1541a40760ad952990f7cf3f35adc6ae2616d0  bitcoin-2ecaf214331b-osx64.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  6e378fb543928e40c7119b96be6ff773d38506a9a888f8b02c7f1b8a0801a80e  bitcoin-core-osx-22-res.yml
  ```

ACKs for top commit:
  laanwj:
    Build script changes review ACK 2ecaf21

Tree-SHA512: 975d5830b787d2e08988f43cbc6e839294171c1d94c8219636308b05f9b77041421612ae67be24a631674670cfc9c2d96d8177f2b3158a78fc3deea19631febf
fanquake added a commit that referenced this issue Jan 7, 2022
84f9931 guix: use upstream python-requests (2.26.0) (fanquake)
187dc1e build: use python-asn1crypto from upstream (fanquake)
b1e8f0b guix: use uptream nsis-x86_64 (fanquake)
3ccfba1 guix: use GCC 10 (over GCC 8) to build releases (fanquake)

Pull request description:

  Guix's `core-updates-frozen` branch has been merged back into `master`, and a [`version-1.4.0`](https://git.savannah.gnu.org/cgit/guix.git/log/?h=version-1.4.0) branch has been created. This is great, as it means the next Guix release is on the horizon, and it contains a number of changes I'd like to take advantage of. In particular, is migrating the version of GCC we use for releases from GCC 8 to GCC 10.3.0 (which is also the new Guix default GCC). This is my preferred method of unblocking progress in #20744, which is currently stalled due to support for `std::filesystem` for Windows not arriving in GCC until version 9, whereas it's usable on Linux starting with GCC 8. The current set of changes in that PR [attempt to backport support](9604eda) for `std::filesystem`, for Windows, to GCC 8, similar to what is currently done by Debian, however that is somewhat convoluted, and using GCC 10 with our current Guix version would require updating at least one Guix patch to GCC, so is not completely straightforward either.

  Other changes included here:
  * Dropping our `--no-*` patch for mingw binutils ld, as we can take advantage of the `--disable-*` flags that are now available in binutlils 2.37. The security check tests are updated accordingly.
  * Dropping our current patch for NSIS, as it's been integrated upstream, however given we are building v3.05, we need a different one (kichik/nsis@229b613) for compiling against GCC 10.
  * Removing our `python-asn1crypto` package definition, as an identical package is available in Guix. Over time we should look at trying to get the rest of the python packages we define here upstreamed.
  * Adding a patch for `python-elfsteem` to fix an issue in the example code when using Python 3.9+.
  * Our base glibc (`2.24`) now inherits from glibc-2.31. Guix has removed packages of glibc < 2.29, and the current version of glibc is `2.33`. However glibc-2.31 is the newest version that still contains a workaround for installing sunrpc data, which we need, so inheriting from that version seemed like the most straightforward solution.
  * As mentioned, Guix has removed glibcs < 2.29, so we add our own package definition for glibc 2.27, which we use for our RISC-V toolchain (also inheriting from 2.31).

  The guix commit hash currently points to the head of the `version-1.4.0` branch. This can be updated to an official release tag when one is available.

  Looking for Concept ACKs on migrating to using GCC 10.3 for building releases. Keeping in mind issues like #20005, however that particular bug should be fixed in GCC 10.3.0+, according to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95189.

  Guix Builds:
  ```bash
  bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  ea56ef38bd94dbcb11b9d10e2f10c205109daad03fea4313f79892fc497ba68d  guix-build-84f9931cb449/output/aarch64-linux-gnu/SHA256SUMS.part
  01123ab23e5a09dc06a897837389e859d302ba2b18fbe827936ec8983765e7df  guix-build-84f9931cb449/output/aarch64-linux-gnu/bitcoin-84f9931cb449-aarch64-linux-gnu-debug.tar.gz
  7a24e25c2237e5aeb14508b91c5c6954572814e1767e892c164494f32d73b0c0  guix-build-84f9931cb449/output/aarch64-linux-gnu/bitcoin-84f9931cb449-aarch64-linux-gnu.tar.gz
  0e1dba0233da1f487222b128964980d50393e61a6971bcf4c71951c29fdf3993  guix-build-84f9931cb449/output/arm-linux-gnueabihf/SHA256SUMS.part
  8cd4c6f42abc81427f1d2500f86daced2a4ee78882dd9d03b5a0211a1d96306e  guix-build-84f9931cb449/output/arm-linux-gnueabihf/bitcoin-84f9931cb449-arm-linux-gnueabihf-debug.tar.gz
  c180db6bffb1a54b6dc65929d86d5eba9adf876a28ad320590ed230233e57299  guix-build-84f9931cb449/output/arm-linux-gnueabihf/bitcoin-84f9931cb449-arm-linux-gnueabihf.tar.gz
  4efcda7b63646eb46dabea7122fb026f2c063d2919a9dcbbffbc0929b9c56ced  guix-build-84f9931cb449/output/dist-archive/bitcoin-84f9931cb449.tar.gz
  1e35e96034fed00674f362d6471fb402dd2758cec2860ded4fd7e37c38935a44  guix-build-84f9931cb449/output/powerpc64-linux-gnu/SHA256SUMS.part
  96a0b7f54d3b3935c134f8c2aaaf11a314b54c9d7924ba751503caa16bd1c840  guix-build-84f9931cb449/output/powerpc64-linux-gnu/bitcoin-84f9931cb449-powerpc64-linux-gnu-debug.tar.gz
  ae05137b6fb3494120f5413bf8a94ca3c1b0c047e1f512e6c2c5a0b1f122f075  guix-build-84f9931cb449/output/powerpc64-linux-gnu/bitcoin-84f9931cb449-powerpc64-linux-gnu.tar.gz
  c22e5fbcdcdbfa5d385537e2c1dab55004d9e94396ebccef0bc3d216edfacbbe  guix-build-84f9931cb449/output/powerpc64le-linux-gnu/SHA256SUMS.part
  52602b41e81a921435d93f2a3ae29549aa65a4147cdbf1ed7d9e4a44c4dc902a  guix-build-84f9931cb449/output/powerpc64le-linux-gnu/bitcoin-84f9931cb449-powerpc64le-linux-gnu-debug.tar.gz
  a2cc7e9385452163a7bda99f6f9aa630fd35d4ba13d4fd9a4dd7e8062206650d  guix-build-84f9931cb449/output/powerpc64le-linux-gnu/bitcoin-84f9931cb449-powerpc64le-linux-gnu.tar.gz
  e75fadf1b1c7e4ae3d52e7a8051a881de17bd4d9d32c1ca29ca0ddbb8028ee51  guix-build-84f9931cb449/output/riscv64-linux-gnu/SHA256SUMS.part
  3b643c33842a15befb5d36d13b598a5e628c11b95671336c8dea51b5eed9c79a  guix-build-84f9931cb449/output/riscv64-linux-gnu/bitcoin-84f9931cb449-riscv64-linux-gnu-debug.tar.gz
  e9a1ee7451502508cde73dc300aca8a421e379ac08c3f4adaf8c768fbfa942ac  guix-build-84f9931cb449/output/riscv64-linux-gnu/bitcoin-84f9931cb449-riscv64-linux-gnu.tar.gz
  c0508a0872cf1415a47983d2ebbc9e5a46282ce7b6453afac544e0d1315b7bf9  guix-build-84f9931cb449/output/x86_64-apple-darwin/SHA256SUMS.part
  7c02267cb91e2649088af5e96f81142beaad67f6a1a0588355174a4157b31458  guix-build-84f9931cb449/output/x86_64-apple-darwin/bitcoin-84f9931cb449-osx-unsigned.dmg
  46dbf5a911abfa63e3c5aa8440289da5fdea89da013253c08768ce58b798a99d  guix-build-84f9931cb449/output/x86_64-apple-darwin/bitcoin-84f9931cb449-osx-unsigned.tar.gz
  ab2e2360f18cb1b80bfd37f1a9508a938e89237767120472f932402cc809f0eb  guix-build-84f9931cb449/output/x86_64-apple-darwin/bitcoin-84f9931cb449-osx64.tar.gz
  f58aa000692f7ea09ab8e7ec159a806d3a665f0f70558e62a53d56afb361eb02  guix-build-84f9931cb449/output/x86_64-linux-gnu/SHA256SUMS.part
  78a76aef8469b07a41588e019a6dfa890c36fd5becf2c8d73a71c9e72bcabde6  guix-build-84f9931cb449/output/x86_64-linux-gnu/bitcoin-84f9931cb449-x86_64-linux-gnu-debug.tar.gz
  5e6e0040b37ff035de41c8fcfee5d498bd19fa489024704dd4caa0ab9f566450  guix-build-84f9931cb449/output/x86_64-linux-gnu/bitcoin-84f9931cb449-x86_64-linux-gnu.tar.gz
  d6e6af70f277d9c9ef9b4773ec05920355ac07ebec71ff3e179676047329964b  guix-build-84f9931cb449/output/x86_64-w64-mingw32/SHA256SUMS.part
  37f24f6899e7803ed07bd0f5eb3f0fb6237ac1254dd72f446e9e4e488a927c8e  guix-build-84f9931cb449/output/x86_64-w64-mingw32/bitcoin-84f9931cb449-win-unsigned.tar.gz
  14f7d1c14a5fc3b4c336d301f936c5578d6e31d61ec720dfc9d4129445d1e2a2  guix-build-84f9931cb449/output/x86_64-w64-mingw32/bitcoin-84f9931cb449-win64-debug.zip
  c8049dcc0308a76f21dd781e8561ebbafa84034fbf8e3afa7d4017866d7fd195  guix-build-84f9931cb449/output/x86_64-w64-mingw32/bitcoin-84f9931cb449-win64-setup-unsigned.exe
  fb1e6580c25b073118f121aabaa04aa09643bc97cfeaea7c9a24bbe65c33cbb6  guix-build-84f9931cb449/output/x86_64-w64-mingw32/bitcoin-84f9931cb449-win64.zip
  ```

ACKs for top commit:
  hebasto:
    re-ACK 84f9931

Tree-SHA512: 2f5f4f6bb1f55a048dba88523f235320e51c4af963355abf6a86b7035623b2100ae3dc44396c76fbeea89ae9cfbc5342abd3e2c41760ede8b689d7757d6e7f25
sidhujag pushed a commit to syscoin/syscoin that referenced this issue Jan 7, 2022
84f9931 guix: use upstream python-requests (2.26.0) (fanquake)
187dc1e build: use python-asn1crypto from upstream (fanquake)
b1e8f0b guix: use uptream nsis-x86_64 (fanquake)
3ccfba1 guix: use GCC 10 (over GCC 8) to build releases (fanquake)

Pull request description:

  Guix's `core-updates-frozen` branch has been merged back into `master`, and a [`version-1.4.0`](https://git.savannah.gnu.org/cgit/guix.git/log/?h=version-1.4.0) branch has been created. This is great, as it means the next Guix release is on the horizon, and it contains a number of changes I'd like to take advantage of. In particular, is migrating the version of GCC we use for releases from GCC 8 to GCC 10.3.0 (which is also the new Guix default GCC). This is my preferred method of unblocking progress in bitcoin#20744, which is currently stalled due to support for `std::filesystem` for Windows not arriving in GCC until version 9, whereas it's usable on Linux starting with GCC 8. The current set of changes in that PR [attempt to backport support](bitcoin@9604eda) for `std::filesystem`, for Windows, to GCC 8, similar to what is currently done by Debian, however that is somewhat convoluted, and using GCC 10 with our current Guix version would require updating at least one Guix patch to GCC, so is not completely straightforward either.

  Other changes included here:
  * Dropping our `--no-*` patch for mingw binutils ld, as we can take advantage of the `--disable-*` flags that are now available in binutlils 2.37. The security check tests are updated accordingly.
  * Dropping our current patch for NSIS, as it's been integrated upstream, however given we are building v3.05, we need a different one (kichik/nsis@229b613) for compiling against GCC 10.
  * Removing our `python-asn1crypto` package definition, as an identical package is available in Guix. Over time we should look at trying to get the rest of the python packages we define here upstreamed.
  * Adding a patch for `python-elfsteem` to fix an issue in the example code when using Python 3.9+.
  * Our base glibc (`2.24`) now inherits from glibc-2.31. Guix has removed packages of glibc < 2.29, and the current version of glibc is `2.33`. However glibc-2.31 is the newest version that still contains a workaround for installing sunrpc data, which we need, so inheriting from that version seemed like the most straightforward solution.
  * As mentioned, Guix has removed glibcs < 2.29, so we add our own package definition for glibc 2.27, which we use for our RISC-V toolchain (also inheriting from 2.31).

  The guix commit hash currently points to the head of the `version-1.4.0` branch. This can be updated to an official release tag when one is available.

  Looking for Concept ACKs on migrating to using GCC 10.3 for building releases. Keeping in mind issues like bitcoin#20005, however that particular bug should be fixed in GCC 10.3.0+, according to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95189.

  Guix Builds:
  ```bash
  bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  ea56ef38bd94dbcb11b9d10e2f10c205109daad03fea4313f79892fc497ba68d  guix-build-84f9931cb449/output/aarch64-linux-gnu/SHA256SUMS.part
  01123ab23e5a09dc06a897837389e859d302ba2b18fbe827936ec8983765e7df  guix-build-84f9931cb449/output/aarch64-linux-gnu/bitcoin-84f9931cb449-aarch64-linux-gnu-debug.tar.gz
  7a24e25c2237e5aeb14508b91c5c6954572814e1767e892c164494f32d73b0c0  guix-build-84f9931cb449/output/aarch64-linux-gnu/bitcoin-84f9931cb449-aarch64-linux-gnu.tar.gz
  0e1dba0233da1f487222b128964980d50393e61a6971bcf4c71951c29fdf3993  guix-build-84f9931cb449/output/arm-linux-gnueabihf/SHA256SUMS.part
  8cd4c6f42abc81427f1d2500f86daced2a4ee78882dd9d03b5a0211a1d96306e  guix-build-84f9931cb449/output/arm-linux-gnueabihf/bitcoin-84f9931cb449-arm-linux-gnueabihf-debug.tar.gz
  c180db6bffb1a54b6dc65929d86d5eba9adf876a28ad320590ed230233e57299  guix-build-84f9931cb449/output/arm-linux-gnueabihf/bitcoin-84f9931cb449-arm-linux-gnueabihf.tar.gz
  4efcda7b63646eb46dabea7122fb026f2c063d2919a9dcbbffbc0929b9c56ced  guix-build-84f9931cb449/output/dist-archive/bitcoin-84f9931cb449.tar.gz
  1e35e96034fed00674f362d6471fb402dd2758cec2860ded4fd7e37c38935a44  guix-build-84f9931cb449/output/powerpc64-linux-gnu/SHA256SUMS.part
  96a0b7f54d3b3935c134f8c2aaaf11a314b54c9d7924ba751503caa16bd1c840  guix-build-84f9931cb449/output/powerpc64-linux-gnu/bitcoin-84f9931cb449-powerpc64-linux-gnu-debug.tar.gz
  ae05137b6fb3494120f5413bf8a94ca3c1b0c047e1f512e6c2c5a0b1f122f075  guix-build-84f9931cb449/output/powerpc64-linux-gnu/bitcoin-84f9931cb449-powerpc64-linux-gnu.tar.gz
  c22e5fbcdcdbfa5d385537e2c1dab55004d9e94396ebccef0bc3d216edfacbbe  guix-build-84f9931cb449/output/powerpc64le-linux-gnu/SHA256SUMS.part
  52602b41e81a921435d93f2a3ae29549aa65a4147cdbf1ed7d9e4a44c4dc902a  guix-build-84f9931cb449/output/powerpc64le-linux-gnu/bitcoin-84f9931cb449-powerpc64le-linux-gnu-debug.tar.gz
  a2cc7e9385452163a7bda99f6f9aa630fd35d4ba13d4fd9a4dd7e8062206650d  guix-build-84f9931cb449/output/powerpc64le-linux-gnu/bitcoin-84f9931cb449-powerpc64le-linux-gnu.tar.gz
  e75fadf1b1c7e4ae3d52e7a8051a881de17bd4d9d32c1ca29ca0ddbb8028ee51  guix-build-84f9931cb449/output/riscv64-linux-gnu/SHA256SUMS.part
  3b643c33842a15befb5d36d13b598a5e628c11b95671336c8dea51b5eed9c79a  guix-build-84f9931cb449/output/riscv64-linux-gnu/bitcoin-84f9931cb449-riscv64-linux-gnu-debug.tar.gz
  e9a1ee7451502508cde73dc300aca8a421e379ac08c3f4adaf8c768fbfa942ac  guix-build-84f9931cb449/output/riscv64-linux-gnu/bitcoin-84f9931cb449-riscv64-linux-gnu.tar.gz
  c0508a0872cf1415a47983d2ebbc9e5a46282ce7b6453afac544e0d1315b7bf9  guix-build-84f9931cb449/output/x86_64-apple-darwin/SHA256SUMS.part
  7c02267cb91e2649088af5e96f81142beaad67f6a1a0588355174a4157b31458  guix-build-84f9931cb449/output/x86_64-apple-darwin/bitcoin-84f9931cb449-osx-unsigned.dmg
  46dbf5a911abfa63e3c5aa8440289da5fdea89da013253c08768ce58b798a99d  guix-build-84f9931cb449/output/x86_64-apple-darwin/bitcoin-84f9931cb449-osx-unsigned.tar.gz
  ab2e2360f18cb1b80bfd37f1a9508a938e89237767120472f932402cc809f0eb  guix-build-84f9931cb449/output/x86_64-apple-darwin/bitcoin-84f9931cb449-osx64.tar.gz
  f58aa000692f7ea09ab8e7ec159a806d3a665f0f70558e62a53d56afb361eb02  guix-build-84f9931cb449/output/x86_64-linux-gnu/SHA256SUMS.part
  78a76aef8469b07a41588e019a6dfa890c36fd5becf2c8d73a71c9e72bcabde6  guix-build-84f9931cb449/output/x86_64-linux-gnu/bitcoin-84f9931cb449-x86_64-linux-gnu-debug.tar.gz
  5e6e0040b37ff035de41c8fcfee5d498bd19fa489024704dd4caa0ab9f566450  guix-build-84f9931cb449/output/x86_64-linux-gnu/bitcoin-84f9931cb449-x86_64-linux-gnu.tar.gz
  d6e6af70f277d9c9ef9b4773ec05920355ac07ebec71ff3e179676047329964b  guix-build-84f9931cb449/output/x86_64-w64-mingw32/SHA256SUMS.part
  37f24f6899e7803ed07bd0f5eb3f0fb6237ac1254dd72f446e9e4e488a927c8e  guix-build-84f9931cb449/output/x86_64-w64-mingw32/bitcoin-84f9931cb449-win-unsigned.tar.gz
  14f7d1c14a5fc3b4c336d301f936c5578d6e31d61ec720dfc9d4129445d1e2a2  guix-build-84f9931cb449/output/x86_64-w64-mingw32/bitcoin-84f9931cb449-win64-debug.zip
  c8049dcc0308a76f21dd781e8561ebbafa84034fbf8e3afa7d4017866d7fd195  guix-build-84f9931cb449/output/x86_64-w64-mingw32/bitcoin-84f9931cb449-win64-setup-unsigned.exe
  fb1e6580c25b073118f121aabaa04aa09643bc97cfeaea7c9a24bbe65c33cbb6  guix-build-84f9931cb449/output/x86_64-w64-mingw32/bitcoin-84f9931cb449-win64.zip
  ```

ACKs for top commit:
  hebasto:
    re-ACK 84f9931

Tree-SHA512: 2f5f4f6bb1f55a048dba88523f235320e51c4af963355abf6a86b7035623b2100ae3dc44396c76fbeea89ae9cfbc5342abd3e2c41760ede8b689d7757d6e7f25
@fanquake
Copy link
Member

fanquake commented Aug 4, 2022

According to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95189#c33, this has been fixed in GCC 10.3.0, which we currently use to build releases. The relevant backported commit, on the releases/gcc-10 branch, is gcc-mirror/gcc@e4c9aac.

So the fix is in GCC master, 12, 11 and 10 branches. Currently we support GCC 8.1+.

fanquake added a commit to fanquake/bitcoin that referenced this issue Aug 14, 2022
This has been fixed in GCC 10.3.0 (which we use for releases) and later,
however according to dependencies.md we still support building with GCC
8.1+.

Add a test case that will fail when someone is using a broken version of
GCC. I've tested this with g++ (Ubuntu 9.4.0-1ubuntu1~20.04.1).

See discussion in bitcoin#20005.

Closes bitcoin#20005.
fanquake added a commit to fanquake/bitcoin that referenced this issue Aug 14, 2022
This has been fixed in GCC 10.3.0 (which we use for releases) and later,
however according to dependencies.md we still support building with GCC
8.1+.

Add a test case that will fail when someone is using a broken version of
GCC. I've tested this with g++ (Ubuntu 9.4.0-1ubuntu1~20.04.1).

See discussion in bitcoin#20005.

Closes bitcoin#20005.
fanquake added a commit to fanquake/bitcoin that referenced this issue Aug 31, 2022
This has been fixed in GCC 10.3.0 (which we use for releases) and later,
however according to dependencies.md we still support building with GCC
8.1+.

Add a test case that will fail when someone is using a broken version of
GCC. I've tested this with g++ (Ubuntu 9.4.0-1ubuntu1~20.04.1).

See discussion in bitcoin#20005.

Closes bitcoin#20005.
@fanquake
Copy link
Member

Does anyone have an opinion on what we should do here going forward? We can just wait until we bump out minimum required GCC to 10.3.0+ (unclear if that'd happen for 25.x), and then mark this as resolved. We could introduce a new compiler bug unit test, something like fanquake@f6e62c6, which will fail when someone is using a broken GCC, but that's hard to do while we are still supporting / testing older compilers in CI. Or we can just continue to leave this open as a known (potential) problem?

@ryanofsky
Copy link
Contributor

We could introduce a new compiler bug unit test, something like fanquake@f6e62c6, which will fail when someone is using a broken GCC, but that's hard to do while we are still supporting / testing older compilers in CI.

Could add the test but make it pass if IGNORE_GCC_BUG_95189 environment variable is set, and set the environment variable in relevant CI variants. That seems like a way to resolve the issue.

@maflcko
Copy link
Member

maflcko commented Aug 31, 2022

Instead of setting IGNORE_GCC_BUG_95189 in the CI, the CI could be patched to assert when the bug is hit in real code

@theuni
Copy link
Member

theuni commented Aug 31, 2022

I worked up a clang AST query that can be used to detect the test-case. I'm not sure if it's perfect though as I don't know exactly what gcc will screw up (copies of pointers, de-references, etc.). If this is useful, it'd be trivial to hook it up to a clang-tidy check, or even a compiler plugin to make it easier to check depends as well.

memcmp.cpp

(exact test-case taken from gcc bug report):

using size_t = decltype(sizeof(0));
int memcmp(const void *s1, const void *s2, size_t n);

static const float z[1] = {0};
int f(float x)
{
    return memcmp(&x, z, sizeof(x));
}

In action:

$ clang-query ../memcmp.cpp -- -std=c++17
clang-query> m callExpr(hasAnyArgument(ignoringImpCasts(declRefExpr(to(varDecl(hasType(qualType(constantArrayType(),isConstQualified())),hasDescendant(integerLiteral(equals(0)))))))),callee(functionDecl(hasName("memcmp"))))

Match #1:

/home/cory/dev/bitcoin-tidy-experiments/build/../memcmp.cpp:7:12: note: "root" binds here
    return memcmp(&x, z, sizeof(x));
           ^~~~~~~~~~~~~~~~~~~~~~~~
1 match.

The query:

callExpr(
    hasAnyArgument(ignoringImpCasts(
        declRefExpr(to(varDecl(
            hasType(qualType(
                constantArrayType(),
                isConstQualified()
            )),
            hasDescendant(integerLiteral(equals(0)))
        ))))
    ),
    callee(functionDecl(hasName("memcmp")))
)

also indentation

@roconnor-blockstream
Copy link
Contributor

Particularly for C++ code, I don't think the clang query is going to be that useful because indirect calls to memcmp can also be problematic. We searched for problematic code by instrumenting GCC itself and I don't think anything else would be particularly reliable. (See https://blog.cri.epita.fr/post/2020-12-05-a-gcc-bug-tracking-affected-software/)

@theuni
Copy link
Member

theuni commented Sep 1, 2022

@roconnor-blockstream Yep, makes perfect sense to use a patched GCC instead. I also hadn't considered implicit or self-generated calls to memcmp, which I assume may be created like for memcpy.

I took a quick look at some of the offenders, many of which are comparisons against string literals like this one: https://github.com/xiph/flac/blob/f764434a39e8a8715d5871bb263189e5a7298280/src/flac/decode.c#L1310

Which indeed the above query doesn't catch. I won't bother with this any further as hacking on gcc clearly makes more sense. Thanks for that work!

PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this issue May 10, 2023
84f9931 guix: use upstream python-requests (2.26.0) (fanquake)
187dc1e build: use python-asn1crypto from upstream (fanquake)
b1e8f0b guix: use uptream nsis-x86_64 (fanquake)
3ccfba1 guix: use GCC 10 (over GCC 8) to build releases (fanquake)

Pull request description:

  Guix's `core-updates-frozen` branch has been merged back into `master`, and a [`version-1.4.0`](https://git.savannah.gnu.org/cgit/guix.git/log/?h=version-1.4.0) branch has been created. This is great, as it means the next Guix release is on the horizon, and it contains a number of changes I'd like to take advantage of. In particular, is migrating the version of GCC we use for releases from GCC 8 to GCC 10.3.0 (which is also the new Guix default GCC). This is my preferred method of unblocking progress in bitcoin#20744, which is currently stalled due to support for `std::filesystem` for Windows not arriving in GCC until version 9, whereas it's usable on Linux starting with GCC 8. The current set of changes in that PR [attempt to backport support](bitcoin@9604eda) for `std::filesystem`, for Windows, to GCC 8, similar to what is currently done by Debian, however that is somewhat convoluted, and using GCC 10 with our current Guix version would require updating at least one Guix patch to GCC, so is not completely straightforward either.

  Other changes included here:
  * Dropping our `--no-*` patch for mingw binutils ld, as we can take advantage of the `--disable-*` flags that are now available in binutlils 2.37. The security check tests are updated accordingly.
  * Dropping our current patch for NSIS, as it's been integrated upstream, however given we are building v3.05, we need a different one (kichik/nsis@229b613) for compiling against GCC 10.
  * Removing our `python-asn1crypto` package definition, as an identical package is available in Guix. Over time we should look at trying to get the rest of the python packages we define here upstreamed.
  * Adding a patch for `python-elfsteem` to fix an issue in the example code when using Python 3.9+.
  * Our base glibc (`2.24`) now inherits from glibc-2.31. Guix has removed packages of glibc < 2.29, and the current version of glibc is `2.33`. However glibc-2.31 is the newest version that still contains a workaround for installing sunrpc data, which we need, so inheriting from that version seemed like the most straightforward solution.
  * As mentioned, Guix has removed glibcs < 2.29, so we add our own package definition for glibc 2.27, which we use for our RISC-V toolchain (also inheriting from 2.31).

  The guix commit hash currently points to the head of the `version-1.4.0` branch. This can be updated to an official release tag when one is available.

  Looking for Concept ACKs on migrating to using GCC 10.3 for building releases. Keeping in mind issues like bitcoin#20005, however that particular bug should be fixed in GCC 10.3.0+, according to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95189.

  Guix Builds:
  ```bash
  bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  ea56ef38bd94dbcb11b9d10e2f10c205109daad03fea4313f79892fc497ba68d  guix-build-84f9931cb449/output/aarch64-linux-gnu/SHA256SUMS.part
  01123ab23e5a09dc06a897837389e859d302ba2b18fbe827936ec8983765e7df  guix-build-84f9931cb449/output/aarch64-linux-gnu/bitcoin-84f9931cb449-aarch64-linux-gnu-debug.tar.gz
  7a24e25c2237e5aeb14508b91c5c6954572814e1767e892c164494f32d73b0c0  guix-build-84f9931cb449/output/aarch64-linux-gnu/bitcoin-84f9931cb449-aarch64-linux-gnu.tar.gz
  0e1dba0233da1f487222b128964980d50393e61a6971bcf4c71951c29fdf3993  guix-build-84f9931cb449/output/arm-linux-gnueabihf/SHA256SUMS.part
  8cd4c6f42abc81427f1d2500f86daced2a4ee78882dd9d03b5a0211a1d96306e  guix-build-84f9931cb449/output/arm-linux-gnueabihf/bitcoin-84f9931cb449-arm-linux-gnueabihf-debug.tar.gz
  c180db6bffb1a54b6dc65929d86d5eba9adf876a28ad320590ed230233e57299  guix-build-84f9931cb449/output/arm-linux-gnueabihf/bitcoin-84f9931cb449-arm-linux-gnueabihf.tar.gz
  4efcda7b63646eb46dabea7122fb026f2c063d2919a9dcbbffbc0929b9c56ced  guix-build-84f9931cb449/output/dist-archive/bitcoin-84f9931cb449.tar.gz
  1e35e96034fed00674f362d6471fb402dd2758cec2860ded4fd7e37c38935a44  guix-build-84f9931cb449/output/powerpc64-linux-gnu/SHA256SUMS.part
  96a0b7f54d3b3935c134f8c2aaaf11a314b54c9d7924ba751503caa16bd1c840  guix-build-84f9931cb449/output/powerpc64-linux-gnu/bitcoin-84f9931cb449-powerpc64-linux-gnu-debug.tar.gz
  ae05137b6fb3494120f5413bf8a94ca3c1b0c047e1f512e6c2c5a0b1f122f075  guix-build-84f9931cb449/output/powerpc64-linux-gnu/bitcoin-84f9931cb449-powerpc64-linux-gnu.tar.gz
  c22e5fbcdcdbfa5d385537e2c1dab55004d9e94396ebccef0bc3d216edfacbbe  guix-build-84f9931cb449/output/powerpc64le-linux-gnu/SHA256SUMS.part
  52602b41e81a921435d93f2a3ae29549aa65a4147cdbf1ed7d9e4a44c4dc902a  guix-build-84f9931cb449/output/powerpc64le-linux-gnu/bitcoin-84f9931cb449-powerpc64le-linux-gnu-debug.tar.gz
  a2cc7e9385452163a7bda99f6f9aa630fd35d4ba13d4fd9a4dd7e8062206650d  guix-build-84f9931cb449/output/powerpc64le-linux-gnu/bitcoin-84f9931cb449-powerpc64le-linux-gnu.tar.gz
  e75fadf1b1c7e4ae3d52e7a8051a881de17bd4d9d32c1ca29ca0ddbb8028ee51  guix-build-84f9931cb449/output/riscv64-linux-gnu/SHA256SUMS.part
  3b643c33842a15befb5d36d13b598a5e628c11b95671336c8dea51b5eed9c79a  guix-build-84f9931cb449/output/riscv64-linux-gnu/bitcoin-84f9931cb449-riscv64-linux-gnu-debug.tar.gz
  e9a1ee7451502508cde73dc300aca8a421e379ac08c3f4adaf8c768fbfa942ac  guix-build-84f9931cb449/output/riscv64-linux-gnu/bitcoin-84f9931cb449-riscv64-linux-gnu.tar.gz
  c0508a0872cf1415a47983d2ebbc9e5a46282ce7b6453afac544e0d1315b7bf9  guix-build-84f9931cb449/output/x86_64-apple-darwin/SHA256SUMS.part
  7c02267cb91e2649088af5e96f81142beaad67f6a1a0588355174a4157b31458  guix-build-84f9931cb449/output/x86_64-apple-darwin/bitcoin-84f9931cb449-osx-unsigned.dmg
  46dbf5a911abfa63e3c5aa8440289da5fdea89da013253c08768ce58b798a99d  guix-build-84f9931cb449/output/x86_64-apple-darwin/bitcoin-84f9931cb449-osx-unsigned.tar.gz
  ab2e2360f18cb1b80bfd37f1a9508a938e89237767120472f932402cc809f0eb  guix-build-84f9931cb449/output/x86_64-apple-darwin/bitcoin-84f9931cb449-osx64.tar.gz
  f58aa000692f7ea09ab8e7ec159a806d3a665f0f70558e62a53d56afb361eb02  guix-build-84f9931cb449/output/x86_64-linux-gnu/SHA256SUMS.part
  78a76aef8469b07a41588e019a6dfa890c36fd5becf2c8d73a71c9e72bcabde6  guix-build-84f9931cb449/output/x86_64-linux-gnu/bitcoin-84f9931cb449-x86_64-linux-gnu-debug.tar.gz
  5e6e0040b37ff035de41c8fcfee5d498bd19fa489024704dd4caa0ab9f566450  guix-build-84f9931cb449/output/x86_64-linux-gnu/bitcoin-84f9931cb449-x86_64-linux-gnu.tar.gz
  d6e6af70f277d9c9ef9b4773ec05920355ac07ebec71ff3e179676047329964b  guix-build-84f9931cb449/output/x86_64-w64-mingw32/SHA256SUMS.part
  37f24f6899e7803ed07bd0f5eb3f0fb6237ac1254dd72f446e9e4e488a927c8e  guix-build-84f9931cb449/output/x86_64-w64-mingw32/bitcoin-84f9931cb449-win-unsigned.tar.gz
  14f7d1c14a5fc3b4c336d301f936c5578d6e31d61ec720dfc9d4129445d1e2a2  guix-build-84f9931cb449/output/x86_64-w64-mingw32/bitcoin-84f9931cb449-win64-debug.zip
  c8049dcc0308a76f21dd781e8561ebbafa84034fbf8e3afa7d4017866d7fd195  guix-build-84f9931cb449/output/x86_64-w64-mingw32/bitcoin-84f9931cb449-win64-setup-unsigned.exe
  fb1e6580c25b073118f121aabaa04aa09643bc97cfeaea7c9a24bbe65c33cbb6  guix-build-84f9931cb449/output/x86_64-w64-mingw32/bitcoin-84f9931cb449-win64.zip
  ```

ACKs for top commit:
  hebasto:
    re-ACK 84f9931

Tree-SHA512: 2f5f4f6bb1f55a048dba88523f235320e51c4af963355abf6a86b7035623b2100ae3dc44396c76fbeea89ae9cfbc5342abd3e2c41760ede8b689d7757d6e7f25
@maflcko
Copy link
Member

maflcko commented Feb 8, 2024

Closing for now. Feel free to reopen, but I don't think anything is going to be done about gcc 10.1 and 10.2 at this point.

See also #29091

@maflcko maflcko closed this as completed Feb 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

10 participants