fix(shared-libs): correctly resolve libwebp libraries #88
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jimeh
force-pushed
the
refactor-shared-library-bundler-logic
branch
from
0ce6900
to
f1d5bcf
Compare
jimeh
changed the title
This was referenced Nov 14, 2023
|
Just did a local build using this branch. Not quite there on my arm64 Mac; maybe the % /Applications/Emacs.app/Contents/MacOS/Emacs
dyld[32900]: Library not loaded: @loader_path/../../../../opt/gettext/lib/libintl.8.dylib
Referenced from: <A95699CE-2BA7-3EA0-A074-A47CAB443C4D> /Applications/Emacs.app/Contents/Frameworks/libgnutls.30.dylib
Reason: tried: '/Applications/Emacs.app/Contents/Frameworks/../../../../opt/gettext/lib/libintl.8.dylib' (no such file), '/usr/local/lib/libintl.8.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/usr/lib/libintl.8.dylib' (no such file, not in dyld cache)
[1] 32900 abort /Applications/Emacs.app/Contents/MacOS/Emacs
% file /Applications/Emacs.app/Contents/MacOS/Emacs
/Applications/Emacs.app/Contents/MacOS/Emacs: Mach-O 64-bit executable arm64
% fd libintl /Applications/Emacs.app
/Applications/Emacs.app/Contents/Frameworks/libintl.8.dylib
% file $(fd libintl /Applications/Emacs.app)
/Applications/Emacs.app/Contents/Frameworks/libintl.8.dylib: Mach-O 64-bit dynamically linked shared library arm64
% otool -L /Applications/Emacs.app/Contents/Frameworks/libgnutls.30.dylib
/Applications/Emacs.app/Contents/Frameworks/libgnutls.30.dylib:
@rpath/libgnutls.30.dylib (compatibility version 67.0.0, current version 67.0.0)
/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 60420.101.2)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1971.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
@loader_path/../../../../opt/gettext/lib/libintl.8.dylib (compatibility version 12.0.0, current version 12.0.0)
@loader_path/../../../../opt/p11-kit/lib/libp11-kit.0.dylib (compatibility version 4.0.0, current version 4.0.0)
@loader_path/../../../../opt/libidn2/lib/libidn2.0.dylib (compatibility version 4.0.0, current version 4.8.0)
@loader_path/../../../../opt/libunistring/lib/libunistring.5.dylib (compatibility version 6.0.0, current version 6.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)
@loader_path/../../../../opt/libtasn1/lib/libtasn1.6.dylib (compatibility version 13.0.0, current version 13.3.0)
@loader_path/../../../../opt/nettle/lib/libnettle.8.dylib (compatibility version 8.0.0, current version 8.8.0)
@loader_path/../../../../opt/nettle/lib/libhogweed.6.dylib (compatibility version 6.0.0, current version 6.8.0)
@loader_path/../../../../opt/gmp/lib/libgmp.10.dylib (compatibility version 15.0.0, current version 15.1.0)⇒ update: it's not a libgnutls problem; my older working build has basically the same otool -L output for that lib. Maybe that my older, working build is still depending on homebrew libs but the newer one doesn't. % otool -L /Applications/Emacs.app/Contents/MacOS/Emacs | grep intl # old build
/opt/homebrew/opt/gettext/lib/libintl.8.dylib (compatibility version 13.0.0, current version 13.0.0)
% otool -L /tmp/Emacs.app/Contents/MacOS/Emacs | grep intl # new build
@rpath/libintl.8.dylib (compatibility version 13.0.0, current version 13.0.0)The new way ought to work because the Emacs exe sets |
|
@garyo Ok, those Would you mind running the build script with (if you want to speed things up, you can also use For reference this what the links look like in my local build for |
|
@garyo Also, what version of gnutls do you have installed via Homebrew? |
|
gnutls: % brew info gnutls
==> gnutls: stable 3.8.1 (bottledDebug log is here |
|
Perhaps this is significant -- it thinks |
|
Maybe this is helpful: why are there 4 ... so you may need some |
|
Interesting, what do the links look like in the dylib within homebrew? Here's mine: And I'll have a quick stab at adding a realpath lookup now to see if that solves it for you. |
|
This fix works for me, though I'm no ruby programmer: % git diff
diff --git a/build-emacs-for-macos b/build-emacs-for-macos
index a0a2a31..e350526 100755
--- a/build-emacs-for-macos
+++ b/build-emacs-for-macos
@@ -1105,7 +1105,8 @@ class LibEmbedder < AbstractEmbedder
fatal "Could not resolve path: #{path}" if abs.nil?
end
- File.expand_path(abs)
+ result = File.expand_path(abs)
+ File.exist?(result) && File.realpath(result) || result
end
def build_bundle_plan(macho_file, copy_macho_file: false) |
|
To your question about my % otool -L /opt/homebrew/opt/gnutls/lib/libgnutls.30.dylib
/opt/homebrew/opt/gnutls/lib/libgnutls.30.dylib:
/opt/homebrew/opt/gnutls/lib/libgnutls.30.dylib (compatibility version 67.0.0, current version 67.0.0)
/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 60420.101.2)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1971.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
@loader_path/../../../../opt/gettext/lib/libintl.8.dylib (compatibility version 12.0.0, current version 12.0.0)
@loader_path/../../../../opt/p11-kit/lib/libp11-kit.0.dylib (compatibility version 4.0.0, current version 4.0.0)
@loader_path/../../../../opt/libidn2/lib/libidn2.0.dylib (compatibility version 4.0.0, current version 4.8.0)
@loader_path/../../../../opt/libunistring/lib/libunistring.5.dylib (compatibility version 6.0.0, current version 6.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)
@loader_path/../../../../opt/libtasn1/lib/libtasn1.6.dylib (compatibility version 13.0.0, current version 13.3.0)
@loader_path/../../../../opt/nettle/lib/libnettle.8.dylib (compatibility version 8.0.0, current version 8.8.0)
@loader_path/../../../../opt/nettle/lib/libhogweed.6.dylib (compatibility version 6.0.0, current version 6.8.0)
@loader_path/../../../../opt/gmp/lib/libgmp.10.dylib (compatibility version 15.0.0, current version 15.1.0)I wonder why mine is different? |
|
@garyo I've pushed a fix that should hopefully work for you. Just saw your patch too, mine is similar, but should yield the same result more or less. If that works, I'd then be curious to see what happens with the links in |
|
Works for me! Also just to confirm, the resulting emacs has no dependencies from |
This is a near complete rewrite of the shared library embedder/bundler code. It now correctly resolves shared libraries that link to other shared library files in the same package via use of `@path`, which the libwebp package does extensively between `libwebpdemux.2.dylib`, `libwebp.7.dylib` and `libsharpyuv.0.dylib`. These relative shared library links were not understood at all by the old library bundler code. The new replacement here fully understands and resolves all `@<something>` placeholders in links to shared libraries. It operates in a two-step process where it first walks down the three of all shared libraries that Emacs links to, and all that they link to, etc. building a copy and relink plan. In a second step it then executes the copy plan, followed by the relinking plan. The result is a fully self-contained Emacs.app binary, which has no dependencies on Homebrew or anything installed via Homebrew. Builds have been tested on a fresh install of macOS Sonoma VM, with nothing but Xcode Command Line Tools installed. Everything from native compilation to webp, svg, png rendering, and more worked as expected.
This should resolve issues on Apple Silicon machines where macOS refuses to run applications without any signatures. On Intel machines it seems to make not difference. If you want to skip the self-signing step, use the `--no-self-sign` flag.
jimeh
force-pushed
the
refactor-shared-library-bundler-logic
branch
from
05ffe6d
to
a534760
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a near complete rewrite of the shared library embedder/bundler code. It
now correctly resolves shared libraries that link to other shared library files
in the same package via use of
@path, which the libwebp package doesextensively between
libwebpdemux.2.dylib,libwebp.7.dylibandlibsharpyuv.0.dylib.These relative shared library links were not understood at all by the old
library bundler code.
The new replacement here fully understands and resolves all
@<something>placeholders in links to shared libraries. It operates in a two-step process
where it first walks down the three of all shared libraries that Emacs links to,
and all that they link to, etc. building a copy and relink plan.
In a second step it then executes the copy plan, followed by the relinking plan.
The result is a fully self-contained Emacs.app binary, which has no dependencies
on Homebrew or anything installed via Homebrew. Builds have been tested on a
fresh install of macOS Sonoma VM, with nothing but Xcode Command Line Tools
installed. Everything from native compilation to webp, svg, png rendering, and
more worked as expected.