Cannot load dylib on Intel macbooks #296

quantumjot · 2023-04-05T12:46:04Z

From Ania:

1.4 GHz Intel Core i5

File /opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/libwrapper.py:84, in load_library(filename)
     83 try:
---> 84     lib = ctypes.cdll.LoadLibrary(full_lib_file)
     85     logger.info(f"Loaded btrack: {full_lib_file}")

File /opt/anaconda3/envs/btrack-env/lib/python3.10/ctypes/__init__.py:452, in LibraryLoader.LoadLibrary(self, name)
    451 def LoadLibrary(self, name):
--> 452     return self._dlltype(name)

File /opt/anaconda3/envs/btrack-env/lib/python3.10/ctypes/__init__.py:374, in CDLL.__init__(self, name, mode, handle, use_errno, use_last_error, winmode)
    373 if handle is None:
--> 374     self._handle = _dlopen(self._name, mode)
    375 else:

OSError: dlopen(/opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/libs/libtracker.dylib, 6): no suitable image found.  Did find:
	/opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/libs/libtracker.dylib: cannot load 'libtracker.dylib' (load command 0x80000034 is unknown)
	/opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/libs/libtracker.dylib: cannot load 'libtracker.dylib' (load command 0x80000034 is unknown)

The above exception was the direct cause of the following exception:

OSError                                   Traceback (most recent call last)
Cell In[12], line 1
----> 1 with btrack.BayesianTracker() as tracker:
      2 
      3     # configure the tracker using a config file
      4     tracker.configure(cfg)
      5     tracker.max_search_radius = 20

File /opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/core.py:130, in BayesianTracker.__init__(self, verbose)
    128 # load the library, get an instance of the engine
    129 self._initialised = False
--> 130 self._lib = libwrapper.get_library()
    131 self._engine = self._lib.new_interface(verbose)
    133 if not verbose:

File /opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/libwrapper.py:94, in get_library()
     92 def get_library():  # noqa: PLR0915
     93     """Loads and returns the btrack shared library."""
---> 94     lib = load_library(os.path.join(BTRACK_PATH, "libs", "libtracker"))
     96     # deal with constructors/destructors
     97     lib.new_interface.restype = ctypes.c_void_p

File /opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/libwrapper.py:87, in load_library(filename)
     85     logger.info(f"Loaded btrack: {full_lib_file}")
     86 except OSError as err:
---> 87     raise OSError(f"Cannot load shared library {full_lib_file}") from err
     89 return lib

OSError: Cannot load shared library /opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/libs/libtracker.dylib

The text was updated successfully, but these errors were encountered:

paddyroddy · 2023-04-05T14:11:40Z

@p-j-smith it works for you though right?

p-j-smith · 2023-04-05T15:56:29Z

hmmm, yeah I just made a new test environment and installed btrack from PyPI and the example notebook runs okay. Plus I see the files are where they should be:

-rw-r--r--   1 paul  staff   3.2M  5 Apr 16:43 libtracker.DLL
-rw-r--r--   1 paul  staff   889K  5 Apr 16:43 libtracker.dylib
drwxr-xr-x   5 paul  staff   160B  5 Apr 16:43 .
-rw-r--r--   1 paul  staff   542K  5 Apr 16:43 libtracker.so

I'm also on an Intel Mac, Monterey 12.6.3. Do you know how Ania installed btrack? And the mac os version?

quantumjot · 2023-04-05T16:03:50Z

Yes - it's weird. It seems to find the file (and it's there!), but doesn't like the symbols:
(load command 0x80000034 is unknown)

I've asked for a few more details on the machine.

ania-m-b · 2023-04-05T21:52:42Z

To answer your questions I am using Mojave 10.14.6. I installed it with both variations one from the installation page and the other from developer guide.

This is the type of error that pops up:

/SDKs/MacOSX10.14.sdk/usr/include/string.h:152:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/sys/_types/_ssize_t.h:31:9: error:
unknown type name '__darwin_ssize_t'; did you mean '__darwin_blksize_t'?
typedef __darwin_ssize_t ssize_t;
^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/sys/_types.h:56:25: note:
'__darwin_blksize_t' declared here
typedef __int32_t __darwin_blksize_t; /* preferred block size */
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make: *** [btrack/obj/bayes.o] Error 1

paddyroddy · 2023-04-13T10:24:11Z

Hard to tell from looking at that alone, but Eigen (which btrack is built on) requires C++17. I wonder if you are using an older version (Mojave is quite old now). Have you tried brew install g++ and following the appropriate instructions?

Or easier still, update your Mac if possible.

nthndy · 2023-04-13T14:09:44Z

I've just come across what looks like the same error when updating to v0.5.2

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
File ~/miniconda3/envs/btrack_dev/lib/python3.9/site-packages/btrack/libwrapper.py:84, in load_library(filename)
     83 try:
---> 84     lib = ctypes.cdll.LoadLibrary(full_lib_file)
     85     logger.info(f"Loaded btrack: {full_lib_file}")

File ~/miniconda3/envs/btrack_dev/lib/python3.9/ctypes/__init__.py:460, in LibraryLoader.LoadLibrary(self, name)
    459 def LoadLibrary(self, name):
--> 460     return self._dlltype(name)

File ~/miniconda3/envs/btrack_dev/lib/python3.9/ctypes/__init__.py:382, in CDLL.__init__(self, name, mode, handle, use_errno, use_last_error, winmode)
    381 if handle is None:
--> 382     self._handle = _dlopen(self._name, mode)
    383 else:

OSError: /home/dayn/miniconda3/envs/btrack_dev/lib/python3.9/site-packages/btrack/libs/libtracker.so: cannot open shared object file: No such file or directory

The above exception was the direct cause of the following exception:

OSError                                   Traceback (most recent call last)
Cell In[3], line 1
----> 1 with btrack.BayesianTracker() as tracker:
      2         # configure the tracker using a config file
      3         tracker.configure(config_fn)

File ~/miniconda3/envs/btrack_dev/lib/python3.9/site-packages/btrack/core.py:130, in BayesianTracker.__init__(self, verbose)
    128 # load the library, get an instance of the engine
    129 self._initialised = False
--> 130 self._lib = libwrapper.get_library()
    131 self._engine = self._lib.new_interface(verbose)
    133 if not verbose:

File ~/miniconda3/envs/btrack_dev/lib/python3.9/site-packages/btrack/libwrapper.py:94, in get_library()
     92 def get_library():  # noqa: PLR0915
     93     """Loads and returns the btrack shared library."""
---> 94     lib = load_library(os.path.join(BTRACK_PATH, "libs", "libtracker"))
     96     # deal with constructors/destructors
     97     lib.new_interface.restype = ctypes.c_void_p

File ~/miniconda3/envs/btrack_dev/lib/python3.9/site-packages/btrack/libwrapper.py:87, in load_library(filename)
     85     logger.info(f"Loaded btrack: {full_lib_file}")
     86 except OSError as err:
---> 87     raise OSError(f"Cannot load shared library {full_lib_file}") from err
     89 return lib

OSError: Cannot load shared library /home/dayn/miniconda3/envs/btrack_dev/lib/python3.9/site-packages/btrack/libs/libtracker.so

nthndy · 2023-04-13T14:13:08Z

am running on ubuntu 20.04 Intel Core i9

paddyroddy · 2023-04-13T17:10:50Z

I've just tried on a ubuntu Intel image (with the platform because I'm on ARM)

docker run -it --rm --platform=linux/x86_64 python:3.10 sh

And was able to install btrack. @quantumjot is there an easy test to make sure the C++ is installed?

quantumjot · 2023-04-17T08:28:37Z

No easy test, but a couple of thoughts:

run the pytest suite locally, or
import btrack and create an instance of BayesianTracker - that should force import of the shared library

paddyroddy · 2023-04-17T09:28:07Z

Okay so

docker run -it --rm --platform=linux/x86_64 python:3.10 sh

Then within the container

pip install btrack ipython

Then start ipython and run

import btrack
btrack.BayesianTracker()

Fails with Cannot load shared library /usr/local/lib/python3.10/site-packages/btrack/libs/libtracker.so. Problem is in local testing is that I have an ARM Mac which we've not built for. It works for @p-j-smith locally. Will investigate.

paddyroddy · 2023-04-17T17:50:40Z

Can you try with v0.5.11 @nthndy, @ania-m-b, @quantumjot, @p-j-smith?

paddyroddy · 2023-04-17T18:19:46Z

Can you try with v0.5.11 @nthndy, @ania-m-b, @quantumjot, @p-j-smith?

False alarm. From the docker I can see this hasn't worked.

ania-m-b · 2023-04-17T19:19:44Z

Hard to tell from looking at that alone, but Eigen (which btrack is built on) requires C++17. I wonder if you are using an older version (Mojave is quite old now). Have you tried brew install g++ and following the appropriate instructions?

Or easier still, update your Mac if possible.

Hi! I have done this previously brew install g++ when installing a different package and I wanted to avoid upgrading to the next MacOS. I am able to run btrack up to version 5.0.

paddyroddy · 2023-04-17T21:19:16Z

Okay, can everyone try btrack==0.5.13? Note the binaries are built with ubuntu=20.04 and/or macos-11. I'm afraid if you have earlier than that then it might not work.

dstansby · 2023-04-17T21:58:00Z

I went down a self inflicted sort-of-related rabbit hole this evening (courtesy of @paddyroddy 😉), and while I was down there at one point I was getting errors like:

[1/2] Compiling C++ object tracker.p/tracker.cc.o
FAILED: tracker.p/tracker.cc.o 
ccache c++ -Itracker.p -I. -I.. -I../../include -I../../../include -I/opt/homebrew/Cellar/eigen/3.4.0_1/include/eigen3 -fcolor-diagnostics -Wall -Winvalid-pch -O0 -g -std=c++17 '-arch arm64' -MD -MQ tracker.p/tracker.cc.o -MF tracker.p/tracker.cc.o.d -o tracker.p/tracker.cc.o -c ../tracker.cc
clang: warning: argument unused during compilation: '-arch arm64' [-Wunused-command-line-argument]
In file included from ../tracker.cc:17:
../../include/tracker.h:268:20: error: 'path' is unavailable: introduced in macOS 10.15
  std::filesystem::path m_debug_filepath;

I think this means btrack is using c++ calls only available in macOS >= 10.15, so it won't work with macOS < 10.15.

quantumjot · 2023-04-18T07:13:31Z

I think this means btrack is using c++ calls only available in macOS >= 10.15, so it won't work with macOS < 10.15.

Interesting! Well this bit of the code base is really only for debugging, so we could remove/refactor this one line for improved compatibility.

quantumjot · 2023-04-18T07:15:19Z

Also @paddyroddy - I just tried a fresh install (btrack 0.5.13 from pypi) on:

Intel® Core™ i9-10940X CPU
Ubuntu 20.04.5 LTS
Python 3.9

...and everything seemed to work ok.

paddyroddy · 2023-04-18T11:54:16Z

Fixed in #300.

quantumjot added the bug label Apr 5, 2023

paddyroddy mentioned this issue Apr 17, 2023

Build wheels on earlier OS versions & verify binaries are included #300

Merged

paddyroddy closed this as completed Apr 18, 2023

quantumjot linked a pull request Apr 27, 2023 that will close this issue

Testing C++11 compilation #309

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot load dylib on Intel macbooks #296

Cannot load dylib on Intel macbooks #296

quantumjot commented Apr 5, 2023

paddyroddy commented Apr 5, 2023

p-j-smith commented Apr 5, 2023

quantumjot commented Apr 5, 2023

ania-m-b commented Apr 5, 2023 •

edited

Loading

paddyroddy commented Apr 13, 2023 •

edited

Loading

nthndy commented Apr 13, 2023

nthndy commented Apr 13, 2023

paddyroddy commented Apr 13, 2023

quantumjot commented Apr 17, 2023

paddyroddy commented Apr 17, 2023

paddyroddy commented Apr 17, 2023

paddyroddy commented Apr 17, 2023

ania-m-b commented Apr 17, 2023

paddyroddy commented Apr 17, 2023 •

edited

Loading

dstansby commented Apr 17, 2023

quantumjot commented Apr 18, 2023

quantumjot commented Apr 18, 2023

paddyroddy commented Apr 18, 2023

Cannot load dylib on Intel macbooks #296

Cannot load dylib on Intel macbooks #296

Comments

quantumjot commented Apr 5, 2023

paddyroddy commented Apr 5, 2023

p-j-smith commented Apr 5, 2023

quantumjot commented Apr 5, 2023

ania-m-b commented Apr 5, 2023 • edited Loading

paddyroddy commented Apr 13, 2023 • edited Loading

nthndy commented Apr 13, 2023

nthndy commented Apr 13, 2023

paddyroddy commented Apr 13, 2023

quantumjot commented Apr 17, 2023

paddyroddy commented Apr 17, 2023

paddyroddy commented Apr 17, 2023

paddyroddy commented Apr 17, 2023

ania-m-b commented Apr 17, 2023

paddyroddy commented Apr 17, 2023 • edited Loading

dstansby commented Apr 17, 2023

quantumjot commented Apr 18, 2023

quantumjot commented Apr 18, 2023

paddyroddy commented Apr 18, 2023

ania-m-b commented Apr 5, 2023 •

edited

Loading

paddyroddy commented Apr 13, 2023 •

edited

Loading

paddyroddy commented Apr 17, 2023 •

edited

Loading