Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: replace libdmg and genisoimage with xorriso #18151

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile.am
Expand Up @@ -147,7 +147,7 @@ $(APP_DIST_DIR)/Applications:
$(APP_DIST_EXTRAS): $(APP_DIST_DIR)/$(OSX_APP)/Contents/MacOS/Bitcoin-Qt

$(OSX_DMG): $(APP_DIST_EXTRAS)
$(GENISOIMAGE) -no-cache-inodes -D -l -probe -V "$(OSX_VOLNAME)" -no-pad -r -dir-mode 0755 -apple -o $@ dist
$(XORRISOFS) -D -l -V "$(OSX_VOLNAME)" -no-pad -r -dir-mode 0755 -o $@ dist

dpi%.$(OSX_BACKGROUND_IMAGE): contrib/macdeploy/$(OSX_BACKGROUND_SVG)
sed 's/PACKAGE_NAME/$(PACKAGE_NAME)/' < "$<" | $(RSVG_CONVERT) -f png -d $* -p $* | $(IMAGEMAGICK_CONVERT) - $@
Expand Down
2 changes: 1 addition & 1 deletion configure.ac
Expand Up @@ -582,7 +582,7 @@ case $host in
*)
AC_PATH_TOOL([INSTALLNAMETOOL], [install_name_tool], install_name_tool)
AC_PATH_TOOL([OTOOL], [otool], otool)
AC_PATH_PROGS([GENISOIMAGE], [genisoimage mkisofs],genisoimage)
AC_PATH_PROGS([XORRISOFS], [xorrisofs],xorrisofs)
AC_PATH_PROGS([RSVG_CONVERT], [rsvg-convert rsvg],rsvg-convert)
AC_PATH_PROGS([IMAGEMAGICK_CONVERT], [convert],convert)
AC_PATH_PROGS([TIFFCP], [tiffcp],tiffcp)
Expand Down
6 changes: 3 additions & 3 deletions contrib/gitian-descriptors/gitian-osx-signer.yml
Expand Up @@ -7,6 +7,7 @@ architectures:
- "amd64"
packages:
- "faketime"
- "xorriso"
remotes:
- "url": "https://github.com/bitcoin-core/bitcoin-detached-sigs.git"
"dir": "signature"
Expand All @@ -18,7 +19,7 @@ script: |
WRAP_DIR=$HOME/wrapped
mkdir -p ${WRAP_DIR}
export PATH="$PWD":$PATH
FAKETIME_PROGS="dmg genisoimage"
FAKETIME_PROGS="xorrisofs"

# Create global faketime wrappers
for prog in ${FAKETIME_PROGS}; do
Expand All @@ -36,5 +37,4 @@ script: |
tar -xf ${UNSIGNED}
OSX_VOLNAME="$(cat osx_volname)"
./detached-sig-apply.sh ${UNSIGNED} signature/osx
${WRAP_DIR}/genisoimage -no-cache-inodes -D -l -probe -V "${OSX_VOLNAME}" -no-pad -r -dir-mode 0755 -apple -o uncompressed.dmg signed-app
${WRAP_DIR}/dmg dmg uncompressed.dmg ${OUTDIR}/${SIGNED}
xorrisofs -D -l -V "${OSX_VOLNAME}" -no-pad -r -dir-mode 0755 -o ${OUTDIR}/${SIGNED} signed-app
8 changes: 4 additions & 4 deletions contrib/gitian-descriptors/gitian-osx.yml
Expand Up @@ -28,6 +28,7 @@ packages:
- "python3-dev"
- "python3-setuptools"
- "fonts-tuffy"
- "xorriso"
remotes:
- "url": "https://github.com/bitcoin/bitcoin.git"
"dir": "bitcoin"
Expand All @@ -38,9 +39,9 @@ script: |

WRAP_DIR=$HOME/wrapped
HOSTS="x86_64-apple-darwin16"
CONFIGFLAGS="--enable-reduce-exports --disable-bench --disable-gui-tests GENISOIMAGE=$WRAP_DIR/genisoimage"
CONFIGFLAGS="--enable-reduce-exports --disable-bench --disable-gui-tests"
FAKETIME_HOST_PROGS=""
FAKETIME_PROGS="ar ranlib date dmg genisoimage"
FAKETIME_PROGS="ar ranlib date xorrisofs"

export QT_RCC_TEST=1
export QT_RCC_SOURCE_DATE_OVERRIDE=1
Expand Down Expand Up @@ -148,7 +149,6 @@ script: |
cp osx_volname unsigned-app-${i}/
cp contrib/macdeploy/detached-sig-apply.sh unsigned-app-${i}
cp contrib/macdeploy/detached-sig-create.sh unsigned-app-${i}
cp ${BASEPREFIX}/${i}/native/bin/dmg ${BASEPREFIX}/${i}/native/bin/genisoimage unsigned-app-${i}
cp ${BASEPREFIX}/${i}/native/bin/${i}-codesign_allocate unsigned-app-${i}/codesign_allocate
cp ${BASEPREFIX}/${i}/native/bin/${i}-pagestuff unsigned-app-${i}/pagestuff
mv dist unsigned-app-${i}
Expand All @@ -157,7 +157,7 @@ script: |
popd

make deploy
${WRAP_DIR}/dmg dmg "${OSX_VOLNAME}.dmg" ${OUTDIR}/${DISTNAME}-osx-unsigned.dmg
mv "${OSX_VOLNAME}.dmg" ${OUTDIR}/${DISTNAME}-osx-unsigned.dmg

cd installed
find . -name "lib*.la" -delete
Expand Down
21 changes: 4 additions & 17 deletions contrib/macdeploy/README.md
Expand Up @@ -103,19 +103,7 @@ created using these tools. The build process has been designed to avoid includin
SDK's files in Gitian's outputs. All interim tarballs are fully deterministic and may be freely
redistributed.

`genisoimage` is used to create the initial DMG. It is not deterministic as-is, so it has been
patched. A system `genisoimage` will work fine, but it will not be deterministic because
the file-order will change between invocations. The patch can be seen here: [cdrkit-deterministic.patch](https://github.com/bitcoin/bitcoin/blob/master/depends/patches/native_cdrkit/cdrkit-deterministic.patch).
No effort was made to fix this cleanly, so it likely leaks memory badly, however it's only used for
a single invocation, so that's no real concern.

`genisoimage` cannot compress DMGs, so afterwards, the DMG tool from the
`libdmg-hfsplus` project is used to compress it. There are several bugs in this tool and its
maintainer has seemingly abandoned the project.

The DMG tool has the ability to create DMGs from scratch as well, but this functionality is
broken. Only the compression feature is currently used. Ideally, the creation could be fixed
and `genisoimage` would no longer be necessary.
[`xorrisofs`](https://www.gnu.org/software/xorriso/) is used to create the DMG.

Background images and other features can be added to DMG files by inserting a
`.DS_Store` before creation. This is generated by the script `contrib/macdeploy/custom_dsstore.py`.
Expand All @@ -127,9 +115,8 @@ deterministic. Here's how it works:

- Builders use Gitian to create an unsigned release. This outputs an unsigned DMG which
users may choose to bless and run. It also outputs an unsigned app structure in the form
of a tarball, which also contains all of the tools that have been previously (deterministically)
built in order to create a final DMG.
of a tarball.
- The Apple keyholder uses this unsigned app to create a detached signature, using the
script that is also included there. Detached signatures are available from this [repository](https://github.com/bitcoin-core/bitcoin-detached-sigs).
- Builders feed the unsigned app + detached signature back into Gitian. It uses the
pre-built tools to recombine the pieces into a deterministic DMG.
- Builders feed the unsigned app + detached signature back into Gitian. It then
recombines all the pieces into a final, deterministic DMG.
26 changes: 0 additions & 26 deletions depends/packages/native_cdrkit.mk

This file was deleted.

24 changes: 0 additions & 24 deletions depends/packages/native_libdmg-hfsplus.mk

This file was deleted.

2 changes: 1 addition & 1 deletion depends/packages/packages.mk
Expand Up @@ -21,5 +21,5 @@ upnp_packages=miniupnpc
darwin_native_packages = native_biplist native_ds_store native_mac_alias

ifneq ($(build_os),darwin)
darwin_native_packages += native_cctools native_cdrkit native_libdmg-hfsplus
darwin_native_packages += native_cctools
endif
86 changes: 0 additions & 86 deletions depends/patches/native_cdrkit/cdrkit-deterministic.patch

This file was deleted.

This file was deleted.