Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add patch to make codesign_allocate compatible with Apple's #20644

Merged
merged 1 commit into from
Dec 17, 2020

Conversation

sipa
Copy link
Member

@sipa sipa commented Dec 13, 2020

This is an alternative to #20638.

The problem is that Apple's codesign(_allocate) apparently rounds the "vmsize" attribute on the __LINKEDIT section to a multiple of 0x2000 on x86_64 rather than 0x1000 (as their published source code does). This divergence means that the binary signed by codesign is slightly different from the one recreated by our reattach-sig-to-gitian-output process, and the signature being invalid.

This fixes it by patching our codesign_allocate source code to also use 0x2000. In tests, this appears to result in matching binaries.

@sipa sipa marked this pull request as draft December 13, 2020 19:38
@sipa sipa force-pushed the 202012_codesign_allocate_segalign branch 2 times, most recently from 6e9f0fa to e4eb1ac Compare December 13, 2020 20:01
@sipa sipa force-pushed the 202012_codesign_allocate_segalign branch from e4eb1ac to a4118c6 Compare December 13, 2020 20:21
@achow101
Copy link
Member

achow101 commented Dec 14, 2020

Tried this with a self signed certificate. Did not work and vmsize was still a multiple of 0x1000

Apparently my gitian is doing something incorrectly. Running the apply locally works as expected and the correct binary is produced.

@jonasschnelli
Copy link
Contributor

Tested ACK a4118c6 - removed the osx cache, built commit a4118c6 for osx in gitian (dependency where built, patch was applied), signed on my signing mac (detach-sig-create), ran gitian osx signer with the produces signature and the a4118c6 build (detach-sig-apply), signature then was successful verified on my Mac (codesign -v /Volumes/Bitcoin-Core/Bitcoin-Qt.app)

@DrahtBot
Copy link
Contributor

Gitian builds

File commit eec9366
(master)
commit 31bfd2e
(master and this pull)
bitcoin-core-linux-22-res.yml caad1af8b3475755... 00851859c56a74ae...
bitcoin-core-osx-22-res.yml 3bddb6b4015d0e9d... f816f03f12ff6a21...
bitcoin-core-win-22-res.yml c59950d0031b62e3... e3796b3f030f5dbc...
*-aarch64-linux-gnu-debug.tar.gz 6a2e5aa01a536671... ca976b32630f326f...
*-aarch64-linux-gnu.tar.gz 948f89703e4cb382... 4556bd5a4324ed1b...
*-arm-linux-gnueabihf-debug.tar.gz 14f17381f0384ae0... 0f29b6c666bf6fac...
*-arm-linux-gnueabihf.tar.gz 4efabff95bcb5e83... b877c11e1f78598e...
*-osx-unsigned.dmg d21116cc35358ed2... 3f9df3878506a86e...
*-osx64.tar.gz c0bd9ab7e7cfc9c5... 9e90928a2e943937...
*-riscv64-linux-gnu-debug.tar.gz 99118103d12d1481... 038a9260144205e3...
*-riscv64-linux-gnu.tar.gz 70529a59e9c10989... c3698393ece83975...
*-win64-debug.zip 8d3ae190278bc27e... 639af3228a260cf1...
*-win64-setup-unsigned.exe 2726732d5dc84e6c... d0d8d78a4b3069ab...
*-win64.zip 76f84b3b35661702... 5503e485f3f0a8d3...
*-x86_64-linux-gnu-debug.tar.gz 79cdcff7e7ca8e17... 6b292d5f6873a285...
*-x86_64-linux-gnu.tar.gz c72263ec5ee2aee3... 1a308d34c3570743...
*.tar.gz dc2391bb1f1e716e... f610cd75b84033b5...
linux-build.log 2b46d6d6dec7a861... 2801729fe970a2ea...
osx-build.log e7cea61e8c3348d1... 623a5e4796e31d80...
win-build.log aefa4d9075aa6f7c... 6b91a0eb8e3d489b...
bitcoin-core-linux-22-res.yml.diff a802e5dbf1144b84...
bitcoin-core-osx-22-res.yml.diff 29440e9d277d80fa...
bitcoin-core-win-22-res.yml.diff 1fcbd5bae2f329d4...
linux-build.log.diff bece3b1301c590aa...
osx-build.log.diff 45e8069606d0a956...
win-build.log.diff 06816c34dda0ba0d...

@fanquake
Copy link
Member

While this patch looks simple, it feels like black magic (at least without the PR description), so I currently prefer #20638. Also given this is patching libstuff, the change ends up in all of the tools, rather than being targeted to codesign* in some way.

@maflcko maflcko marked this pull request as ready for review December 17, 2020 19:40
@maflcko
Copy link
Member

maflcko commented Dec 17, 2020

Concept ACK a4118c6

@sipa
Copy link
Member Author

sipa commented Dec 17, 2020

Updated PR description.

@fanquake It's easy to trace which functions in cctools call get_segalign_from_flag (the only function that accesses the modified field): it's codesign_allocate, lipo, segedit, and bitcode_strip. I believe we only use the former.

@laanwj laanwj merged commit 816314e into bitcoin:master Dec 17, 2020
maflcko pushed a commit to maflcko/bitcoin-core that referenced this pull request Dec 17, 2020
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Dec 17, 2020
…ith Apple's

a4118c6 Add patch to make codesign_allocate compatible with Apple's (Pieter Wuille)

Pull request description:

  This is an alternative to bitcoin#20638.

  The problem is that Apple's codesign(_allocate) apparently rounds the "vmsize" attribute on the __LINKEDIT section to a multiple of 0x2000 on x86_64 rather than 0x1000 (as their published source code does). This divergence means that the binary signed by codesign is slightly different from the one recreated by our reattach-sig-to-gitian-output process, and the signature being invalid.

  This fixes it by patching our codesign_allocate source code to also use 0x2000. In tests, this appears to result in matching binaries.

ACKs for top commit:
  jonasschnelli:
    Tested ACK a4118c6 - removed the osx cache, built commit a4118c6 for osx in gitian (dependency where built, patch was applied), signed on my signing mac (detach-sig-create), ran gitian osx signer with the produces signature and the a4118c6 build (detach-sig-apply), signature then was successful verified on my Mac (codesign -v /Volumes/Bitcoin-Core/Bitcoin-Qt.app)
  MarcoFalke:
    Concept ACK a4118c6

Tree-SHA512: 07b8cdf8216249ddfe4bd38b39f2b48b2e190d4002b84d8981e62197bbbc9f25ac5c137bcc32057b23fbf38cbb2889ef95101ce008edfbf608cd170b88b3acbc
@fanquake
Copy link
Member

Being backported to 0.21 in #20669.

maflcko pushed a commit to maflcko/bitcoin-core that referenced this pull request Dec 21, 2020
maflcko pushed a commit to maflcko/bitcoin-core that referenced this pull request Dec 21, 2020
fanquake added a commit to fanquake/bitcoin that referenced this pull request Dec 22, 2020
Github-Pull: bitcoin#20644
Rebased-From: a4118c6

Co-authored-by: Pieter Wuille <pieter@wuille.net>
fanquake added a commit to fanquake/bitcoin that referenced this pull request Dec 22, 2020
Github-Pull: bitcoin#20644
Rebased-From: a4118c6

Co-authored-by: Pieter Wuille <pieter@wuille.net>
@fanquake
Copy link
Member

Being backported to 0.19 in #20739 & 0.20 in #20738.

maflcko pushed a commit to maflcko/bitcoin-core that referenced this pull request Dec 22, 2020
fanquake added a commit to fanquake/bitcoin that referenced this pull request Dec 22, 2020
Github-Pull: bitcoin#20644
Rebased-From: a4118c6

Co-authored-by: Pieter Wuille <pieter@wuille.net>
fanquake added a commit to fanquake/bitcoin that referenced this pull request Dec 22, 2020
Github-Pull: bitcoin#20644
Rebased-From: a4118c6

Co-authored-by: Pieter Wuille <pieter@wuille.net>
maflcko pushed a commit to maflcko/bitcoin-core that referenced this pull request Jan 5, 2021
…atible with Apple's"

a0eb4c5 Revert "Add patch to make codesign_allocate compatible with Apple's" (Pieter Wuille)

Pull request description:

  This reverts bitcoin#20644.

  It appears that Apple has recently changed their `codesign_allocate` tool back to using 4k alignment on x86_64, at least in some cases, so this patch isn't causing our cctools-based version to be exactly compatible.

  Furthermore, if codesigning were to change to use https://github.com/achow101/signapple instead, there is no need anymore to try to mimick Apple.

ACKs for top commit:
  laanwj:
    ACK a0eb4c5
  MarcoFalke:
    checked-clean-revert ACK a0eb4c5
  jonasschnelli:
    ACK a0eb4c5

Tree-SHA512: 529719a76811006122406689233d1e80995107fe1ac1fc862a4ac53ca21685748ed76cac7ca648dd70f0ea43dd8dcf2e29d559beeab10e1d30dc5542ac95fd97
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Jan 5, 2021
…atible with Apple's"

a0eb4c5 Revert "Add patch to make codesign_allocate compatible with Apple's" (Pieter Wuille)

Pull request description:

  This reverts bitcoin#20644.

  It appears that Apple has recently changed their `codesign_allocate` tool back to using 4k alignment on x86_64, at least in some cases, so this patch isn't causing our cctools-based version to be exactly compatible.

  Furthermore, if codesigning were to change to use https://github.com/achow101/signapple instead, there is no need anymore to try to mimick Apple.

ACKs for top commit:
  laanwj:
    ACK a0eb4c5
  MarcoFalke:
    checked-clean-revert ACK a0eb4c5
  jonasschnelli:
    ACK a0eb4c5

Tree-SHA512: 529719a76811006122406689233d1e80995107fe1ac1fc862a4ac53ca21685748ed76cac7ca648dd70f0ea43dd8dcf2e29d559beeab10e1d30dc5542ac95fd97
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Feb 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants