Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dyld: Library not loaded: /usr/local/opt/libsndfile/lib/libsndfile.1.dylib #4007

Closed
prko opened this issue Aug 22, 2018 · 25 comments
Closed
Labels
comp: build CMake build system os: macOS
Projects
Milestone

Comments

@prko
Copy link
Contributor

prko commented Aug 22, 2018

Environment

  • Your SuperCollider version: 3.10.0-beta1[ee63db9]
  • Your operating system and version: 10.13.6 (the system is completely new.)

Steps to reproduce (for bugs)

  1. In SC-IDE

    1. launch SC
    2. evaluate:
      s.boot
    3. see the post window
  2. In terminal

    1. run sclang
      /Applications/SuperCollider/SuperCollider.app/Contents/MacOS/sclang
    2. evaluate:
      s.boot
    3. see the terminal window

Error message (for bugs)

  1. error in SC post window
Booting server 'localhost' on address 127.0.0.1:57110.
Server 'localhost' exited with exit code 0.
  1. error in terminal
Last login: Wed Aug 22 16:06:32 on ttys001
prkos-MacBook-Pro:~ prko$ /Applications/SuperCollider/SuperCollider.app/Contents/MacOS/sclang
compiling class library...
	Found 836 primitives.
	Compiling directory '/Applications/SuperCollider/SuperCollider.app/Contents/Resources/SCClassLibrary'
	Compiling directory '/Users/prko/Library/Application Support/SuperCollider/Extensions'
	numentries = 813176 / 11795768 = 0.069
	5323 method selectors, 2216 classes
	method table size 12977832 bytes, big table size 94366144
	Number of Symbols 11911
	Byte Code Size 356908
	compiled 316 files in 0.68 seconds
compile done
localhost : setting clientID to 0.
internal : setting clientID to 0.
Class tree inited in 0.02 seconds

*** Welcome to SuperCollider 3.10.0-beta1. *** For help type cmd-d.
sc3> s.boot
-> localhost
sc3> Booting server 'localhost' on address 127.0.0.1:57110.
dyld: Library not loaded: /usr/local/opt/libsndfile/lib/libsndfile.1.dylib
  Referenced from: /Applications/SuperCollider/SuperCollider.app/Contents/Resources/../Resources/scsynth
  Reason: image not found
Server 'localhost' exited with exit code 0.

Expected Behavior

The server must boot.

Current Behavior

The server does not boot.

@adcxyz
Copy link
Contributor

adcxyz commented Aug 22, 2018

Can report the same thing with supernova too:
For tests, I put a 3.10beta based Standalone on several clean systems yesterday,
and got recurring missing libraries, such as libportaudio, libogg, libvorbis etc.
Console log shows it was looking for them in /usr/local/opt/ which is where brew puts the aliases;
so it seems what is missing is the path within the app's Contents to the local dylib files.

otool shows the hardcoded path:

adc:MacOS adc$ cd ../Resources/
adc:Resources adc$ otool -L scsynth 
scsynth:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.50.2)
----- here:
	/usr/local/opt/libsndfile/lib/libsndfile.1.dylib (compatibility version 2.0.0, current version 2.28.0)
	/System/Library/Frameworks/CoreAudio.framework/Versions/A/CoreAudio (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
	/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 775.19.0)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1349.63.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 307.5.0)
	/System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork (compatibility version 1.0.0, current version 811.4.18)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1349.64.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
adc:Resources adc$ 

and here for supernova:

supernova:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
--- these two:
	/usr/local/opt/portaudio/lib/libportaudio.2.dylib (compatibility version 3.0.0, current version 3.0.0)
	/usr/local/opt/libsndfile/lib/libsndfile.1.dylib (compatibility version 2.0.0, current version 2.26.0)
----
	/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
	/System/Library/Frameworks/CoreAudio.framework/Versions/A/CoreAudio (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 822.19.0)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1450.15.0)
	/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 50.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1450.15.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
adc:Resources adc$ 

@mossheim
Copy link
Contributor

This should be easy to fix - just run fixup_bundle on scsynth and supernova.
https://gitlab.kitware.com/cmake/community/wikis/doc/cpack/BundleUtilities

@nhthn nhthn added this to the 3.10 milestone Aug 23, 2018
@nhthn
Copy link
Contributor

nhthn commented Aug 23, 2018

this is an urgent one since the beta is broken -- if someone with macOS would like to try out fixup_bundle and see if it works, please do. brian's busy as of late and i don't have a mac.

@prko
Copy link
Contributor Author

prko commented Aug 23, 2018

Sorry, I would like to fix this, but I think I do not correctly understand it.

@adcxyz
Copy link
Contributor

adcxyz commented Aug 23, 2018 via email

@mossheim
Copy link
Contributor

example usage is shown in the link. you will need to find some place in the CMakeLists to make the call, my guess would be just after macdeployqt is invoked. it's not as if I know exactly what to do, this is just a hunch and it's going to take trial and error.

@adcxyz
Copy link
Contributor

adcxyz commented Aug 23, 2018 via email

@nhthn nhthn added this to To do in 3.10 Aug 24, 2018
@adcxyz
Copy link
Contributor

adcxyz commented Aug 25, 2018

ok, got the dylib lookups fixed on my local copy of SC:
otool to check and install_name_tool -change to set the paths.

Before putting those fixups in the the CMake script, I was wondering whether this is really the best way to go, and it seems to me that compiling scsynth and supernova to to completely independent is actually preferable. I.e. if we compile all external libs into the scsynth and supernova binaries, then they can be used really flexibly, e.g. also as sound engine for non-sclang frontends and standalones.

Is this correct, or am I missing anything here?

@mossheim
Copy link
Contributor

That is what fixup_bundle does, it runs otool and install_name_tool programmatically to fix paths in executables, and also verifies that the paths are correct afterwards.

And no, statically linking each of these libraries into each of 3 executables is not the way to go, that's just bloat. I'm assuming that's what you mean by "compile into". If you want to distribute these binaries as separate programs for macOS the best thing to do would be to register them as packages with homebrew and list the relevant dylibs as dependencies.

If you really want to support statically linking you can add a separate config flag in our project to do so, but I do not support this because it is more to maintain for dubious benefit.

@adcxyz
Copy link
Contributor

adcxyz commented Aug 26, 2018

@brianlheim - thanks for the background info, so dylibs it is.

In the CMake scripts, I don't see where best to do the fixups,
so cmake gurus, please help!

Here is the list of all the library paths that I fixed upby hand
for understanding what the paths should be,
and that need to be fixup_bundled in the cmake scripts:

scsynth:
is:     /usr/local/opt/libsndfile/lib/libsndfile.1.dylib
should be: @loader_path/../Frameworks/libsndfile.1.dylib

supernova:
is:     /usr/local/opt/libsndfile/lib/libsndfile.1.dylib
       /usr/local/opt/portaudio/lib/libportaudio.2.dylib
should be: @loader_path/../Frameworks/libsndfile.1.dylib
should be: @loader_path/../Frameworks/libportaudio.2.dylib

BOTH DiskIO_Ugens.scx, DiskIO_Ugens_supernova.scx:
is:     /usr/local/opt/libsndfile/lib/libsndfile.1.dylib
should be: @loader_path/../../Frameworks/libsndfile.1.dylib

A similar path error causes the help browser to fail:
QtWebEngineProcess has several broken loadpaths -
Console says help browser fails here:

 @loader_path/../../../../../../../QtQuick.framework/Versions/5/QtQuick
  Referenced from: /Volumes/VOLUME/Users/Shared/*/STS_Projects310_b1.app/Contents/Frameworks/QtWebEngineCore.framework/Helpers/QtWebEngineProcess.app/Contents/MacOS/QtWebEngineProcess
  Reason: image not found

The likely fix seems to be to set the proper depth for all these paths:
QtWebEngineCore.framework/Helpers/QtWebEngineProcess.app/Contents/MacOS/

is: @loader_path/../../../../../../../QtQuick.framework/Versions/5/QtQuick
-> @loader_path/../../../../../QtQuick.framework/Versions/5/QtQuick
... etc for QtQuick, QtGui, QtCore, QtQml, QtNetwork, QtPositioning

@mossheim
Copy link
Contributor

@adcxyz The last error you're getting because you're breaking symlinks inside the app bundle. Those relative paths are correct, but you must be putting it through some operation that resolves symlinks to their targets. It looks in this case like you might be moving it onto some networked filesystem?

@mossheim
Copy link
Contributor

I am going to work on this on my flight later today.

@adcxyz
Copy link
Contributor

adcxyz commented Aug 26, 2018 via email

@mossheim
Copy link
Contributor

@adcxyz
Copy link
Contributor

adcxyz commented Aug 26, 2018 via email

@nhthn
Copy link
Contributor

nhthn commented Aug 31, 2018

how goes? hate to crack the whip, but if this is as serious as i think it is, it's a showstopper for 3.10-beta2.

@adcxyz
Copy link
Contributor

adcxyz commented Sep 1, 2018 via email

@mossheim
Copy link
Contributor

mossheim commented Sep 1, 2018

I am not actively working on this.

@adcxyz
Copy link
Contributor

adcxyz commented Sep 3, 2018

@brianlheim

I am going to work on this on my flight later today.

Did you find any time at all to look into this?
My (superficial) impression of the cmake script was that these libraries
are not in the file path lists that go into the existing fixup_bundle calls;
but I did not find an obvious place where and how to add them.

@mossheim
Copy link
Contributor

mossheim commented Sep 3, 2018

I don't have any further comments on this right now. I understand this is a pressing issue for the project but I simply don't have the energy to work on it in the immediate future.

@adcxyz
Copy link
Contributor

adcxyz commented Sep 4, 2018

sure, no problem - hopefully someone with better cmake knowledge than me has time soon to solve this.

jpburstrom added a commit to jpburstrom/supercollider that referenced this issue Sep 6, 2018
Adds 4 new executables to macdeployqt command: scsynth, supernova,
and DiskIO plugins. A quick-fix for supercollider#4007.
jpburstrom added a commit to jpburstrom/supercollider that referenced this issue Sep 6, 2018
Adds 4 new executables to macdeployqt command: scsynth, supernova,
and DiskIO plugins. A quick-fix for supercollider#4007.
jpburstrom added a commit to jpburstrom/supercollider that referenced this issue Sep 6, 2018
Adds 4 new executables to macdeployqt command: scsynth, supernova,
and DiskIO plugins. A quick-fix for supercollider#4007.
@jpburstrom
Copy link
Contributor

The commit referenced above might be a temporary solution. It adds some --executable paths (scsynth, supernova and DiskIO plugins) to the macdeployqt command in editors/sc-ide/CMakeLists.txt, which seems to be the place where the bundle is created and fixed-up. It's a bit of a kludge, but seems to do the job.

@mossheim
Copy link
Contributor

mossheim commented Sep 6, 2018

@jpburstrom I think that should work for our default builds! However can you test with a build with -DSUPERNOVA=OFF? I'm not sure if macdeployqt will consider missing executables a failure or not. If it does, you will need to use a generator expression or something similar to check for that.

We should also call verify_app (from BundleUtilities) on the package after running macdeployqt. Even if we don't use fixup_bundle. I regret not adding that call in when I reworked this code earlier this year.

@jpburstrom jpburstrom mentioned this issue Sep 7, 2018
4 tasks
@jpburstrom
Copy link
Contributor

@brianlheim Thanks for the feedback. I've made a pull request. I've tested with and without supernova, and it seems to be ok. But as stated in the pull request, I don't know if macdeployqt in other qt versions behaves differently.

I've also added a verify_app call. It works fine, but from the output it seems like it just checks the executables, and not the plugins, which might be an issue in the long run.

@mossheim
Copy link
Contributor

mossheim commented Sep 9, 2018

from the output it seems like it just checks the executables, and not the plugins, which might be an issue in the long run.

Yes, that's true, and there doesn't seem to be a way around it. Since this is only going to run on macOS we could just roll our own verification as a shell script, something like what I gave here: #4047 (comment)

Thanks again! Closed in #4047.

@mossheim mossheim closed this as completed Sep 9, 2018
@nhthn nhthn moved this from To do to Done in 3.10 Sep 18, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp: build CMake build system os: macOS
Projects
No open projects
3.10
Done
Development

No branches or pull requests

5 participants