Import Error - dlopen Expected in: flat namespace

[lauziere]

Hello,

I've been trying to install stardist today.
Mac OSX Mojave 10.14.4

Anaconda 3.6.7
gcc-9 and g++-9 compilers

I run "pip install stardist":

| => pip install stardist
Collecting stardist
Requirement already satisfied: csbdeep>=0.4.0 in /Users/guest/anaconda3/lib/python3.6/site-packages (from stardist) (0.4.1)
Requirement already satisfied: scikit-image in /Users/guest/anaconda3/lib/python3.6/site-packages (from stardist) (0.15.0)
Requirement already satisfied: numba in /Users/guest/anaconda3/lib/python3.6/site-packages (from stardist) (0.43.1)
Requirement already satisfied: numpy in /Users/guest/anaconda3/lib/python3.6/site-packages (from csbdeep>=0.4.0->stardist) (1.16.4)
Requirement already satisfied: matplotlib in /Users/guest/anaconda3/lib/python3.6/site-packages (from csbdeep>=0.4.0->stardist) (3.1.0)
Requirement already satisfied: keras>=2.1.2 in /Users/guest/anaconda3/lib/python3.6/site-packages (from csbdeep>=0.4.0->stardist) (2.2.2)
Requirement already satisfied: tifffile in /Users/guest/anaconda3/lib/python3.6/site-packages (from csbdeep>=0.4.0->stardist) (0.15.1)
Requirement already satisfied: tqdm in /Users/guest/anaconda3/lib/python3.6/site-packages (from csbdeep>=0.4.0->stardist) (4.32.1)
Requirement already satisfied: h5py in /Users/guest/anaconda3/lib/python3.6/site-packages (from csbdeep>=0.4.0->stardist) (2.9.0)
Requirement already satisfied: scipy in /Users/guest/anaconda3/lib/python3.6/site-packages (from csbdeep>=0.4.0->stardist) (1.3.0)
Requirement already satisfied: six in /Users/guest/anaconda3/lib/python3.6/site-packages (from csbdeep>=0.4.0->stardist) (1.12.0)
Requirement already satisfied: networkx>=2.0 in /Users/guest/anaconda3/lib/python3.6/site-packages (from scikit-image->stardist) (2.3)
Requirement already satisfied: pillow>=4.3.0 in /Users/guest/anaconda3/lib/python3.6/site-packages (from scikit-image->stardist) (6.0.0)
Requirement already satisfied: imageio>=2.0.1 in /Users/guest/anaconda3/lib/python3.6/site-packages (from scikit-image->stardist) (2.5.0)
Requirement already satisfied: PyWavelets>=0.4.0 in /Users/guest/anaconda3/lib/python3.6/site-packages (from scikit-image->stardist) (1.0.3)
Requirement already satisfied: llvmlite>=0.28.0dev0 in /Users/guest/anaconda3/lib/python3.6/site-packages (from numba->stardist) (0.28.0)
Requirement already satisfied: python-dateutil>=2.1 in /Users/guest/anaconda3/lib/python3.6/site-packages (from matplotlib->csbdeep>=0.4.0->stardist) (2.8.0)
Requirement already satisfied: cycler>=0.10 in /Users/guest/anaconda3/lib/python3.6/site-packages (from matplotlib->csbdeep>=0.4.0->stardist) (0.10.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /Users/guest/anaconda3/lib/python3.6/site-packages (from matplotlib->csbdeep>=0.4.0->stardist) (2.4.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /Users/guest/anaconda3/lib/python3.6/site-packages (from matplotlib->csbdeep>=0.4.0->stardist) (1.1.0)
Requirement already satisfied: keras-preprocessing==1.0.2 in /Users/guest/anaconda3/lib/python3.6/site-packages (from keras>=2.1.2->csbdeep>=0.4.0->stardist) (1.0.2)
Requirement already satisfied: pyyaml in /Users/guest/anaconda3/lib/python3.6/site-packages (from keras>=2.1.2->csbdeep>=0.4.0->stardist) (5.1)
Requirement already satisfied: keras-applications==1.0.4 in /Users/guest/anaconda3/lib/python3.6/site-packages (from keras>=2.1.2->csbdeep>=0.4.0->stardist) (1.0.4)
Requirement already satisfied: decorator>=4.3.0 in /Users/guest/anaconda3/lib/python3.6/site-packages (from networkx>=2.0->scikit-image->stardist) (4.4.0)
Requirement already satisfied: setuptools in /Users/guest/anaconda3/lib/python3.6/site-packages (from kiwisolver>=1.0.1->matplotlib->csbdeep>=0.4.0->stardist) (41.0.1)
Installing collected packages: stardist
Successfully installed stardist-0.3.4

Then on command: python -c "import stardist" I get:

Traceback (most recent call last):
File "", line 1, in
File "/Users/guest/anaconda3/lib/python3.6/site-packages/stardist/init.py", line 8, in
from .geometry import star_dist, polygons_to_label, relabel_image_stardist, ray_angles, dist_to_coord
File "/Users/guest/anaconda3/lib/python3.6/site-packages/stardist/geometry/init.py", line 5, in
from .geom2d import star_dist, polygons_to_label, relabel_image_stardist, ray_angles, dist_to_coord
File "/Users/guest/anaconda3/lib/python3.6/site-packages/stardist/geometry/geom2d.py", line 11, in
from ..lib.stardist2d import c_star_dist
ImportError: dlopen(/Users/guest/anaconda3/lib/python3.6/site-packages/stardist/lib/stardist2d.cpython-36m-darwin.so, 2): Symbol not found: _GOMP_loop_nonmonotonic_dynamic_next
Referenced from: /Users/guest/anaconda3/lib/python3.6/site-packages/stardist/lib/stardist2d.cpython-36m-darwin.so
Expected in: flat namespace
in /Users/guest/anaconda3/lib/python3.6/site-packages/stardist/lib/stardist2d.cpython-36m-darwin.so

Any idea what I can do here. Happy to provide other information.

Thank you for the help!

[maweigert]

Hi,

I had a similar issue a while back ago that was caused by some strange compiler/conda combination. Could you try the following:

1. Install gcc-8/g++-8
   e.g. brew install gcc@8

2. Install stardist with gcc-8
   pip uninstall -y stardist
CC="gcc-8" CXX="g++-8" pip install --no-cache-dir stardist

Btw, using stardist on OSX might not be ideal, as OSX doesn't support tensorflow/GPU (a general limitation) and training will take very long :)

[lauziere]

Hey @maweigert Thank you for the quick reply.

Yes I am just toying around with the utils on my local machine to get a feel for it. Once I have a better understanding I will install it on our cluster (running cent OS).

I followed your two steps. The installation was successful. Unfortunately I get the same error:

| => python3
Python 3.6.7 | packaged by conda-forge | (default, Feb 28 2019, 02:16:08)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import stardist
Traceback (most recent call last):
File "", line 1, in
File "/Users/guest/anaconda3/lib/python3.6/site-packages/stardist/init.py", line 8, in
from .geometry import star_dist, polygons_to_label, relabel_image_stardist, ray_angles, dist_to_coord
File "/Users/guest/anaconda3/lib/python3.6/site-packages/stardist/geometry/init.py", line 5, in
from .geom2d import star_dist, polygons_to_label, relabel_image_stardist, ray_angles, dist_to_coord
File "/Users/guest/anaconda3/lib/python3.6/site-packages/stardist/geometry/geom2d.py", line 11, in
from ..lib.stardist2d import c_star_dist
ImportError: dlopen(/Users/guest/anaconda3/lib/python3.6/site-packages/stardist/lib/stardist2d.cpython-36m-darwin.so, 2): Symbol not found: _GOMP_parallel
Referenced from: /Users/guest/anaconda3/lib/python3.6/site-packages/stardist/lib/stardist2d.cpython-36m-darwin.so
Expected in: flat namespace
in /Users/guest/anaconda3/lib/python3.6/site-packages/stardist/lib/stardist2d.cpython-36m-darwin.so

Could the issue be with python? I noticed just now the lines:

019, 02:16:08)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin

when I load python3 in command line.

Also I find that this gcc (gcc-8) or the other version (gcc-9) are not my default c compilers.

When I run "which gcc" I get /usr/bin/gcc

However when I run "gcc -v" I get:

Using built-in specs.
COLLECT_GCC=/usr/local/bin/gcc-9
COLLECT_LTO_WRAPPER=/usr/local/Cellar/gcc/9.2.0/libexec/gcc/x86_64-apple-darwin18/9.2.0/lto-wrapper
Target: x86_64-apple-darwin18
Configured with: ../configure --build=x86_64-apple-darwin18 --prefix=/usr/local/Cellar/gcc/9.2.0 --libdir=/usr/local/Cellar/gcc/9.2.0/lib/gcc/9 --disable-nls --enable-checking=release --enable-languages=c,c++,objc,obj-c++,fortran --program-suffix=-9 --with-gmp=/usr/local/opt/gmp --with-mpfr=/usr/local/opt/mpfr --with-mpc=/usr/local/opt/libmpc --with-isl=/usr/local/opt/isl --with-system-zlib --with-pkgversion='Homebrew GCC 9.2.0' --with-bugurl=https://github.com/Homebrew/homebrew-core/issues --disable-multilib --with-native-system-header-dir=/usr/include --with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
Thread model: posix
gcc version 9.2.0 (Homebrew GCC 9.2.0)

Any thoughts?

Thanks, Andrew

[maweigert]

Hi,

The gcc version differences are fine, as for the package to work it only matters which gcc was used during compilation (and both gcc-8 and gcc-9 should be fine).
Your error message looks like that the wrong OMP library (libgomp.1.dylib) is used during dynamic library loading, which does not contain the missing symbols. E.g. the one provided by conda is missing some necessary symbols (e.g. _GOMP_parallel), which the gcc-8/9 ones contain.

Could you provide the output of the following command?

DYLD_PRINT_LIBRARIES=1 python -c "import stardist;" 2>&1 | grep libgomp

[lauziere]

Yes -

| => DYLD_PRINT_LIBRARIES=1 python -c "import stardist;" 2>&1 | grep libgomp

dyld: loaded: /Users/guest/anaconda3/lib/libgomp.1.dylib
dyld: unloaded: /Users/guest/anaconda3/lib/libgomp.1.dylib

[maweigert]

Yeah, so it uses the wrong library. Will be hard to debug.
A (brute-force) workaround is to find out where your gcc-9 libgomp.1.dylib is located and then change the dylib dependency in stardist explicitly.
For this, you need to know the path of the lib that is currently loaded (src) and the path of the gcc lib (dst). The first you can find out via

otool -L  /Users/guest/anaconda3/lib/python3.6/site-packages/stardist/lib/stardist3d.cpython-36m-darwin.so

which for me is @rpath/libgomp.1.dylib.

Afterwards you can reset the lib dependencies:

# adapt this to your locations 
src="@rpath/libgomp.1.dylib"
dst="/usr/local/Cellar/gcc/9.1.0/lib/gcc/9/libgomp.1.dylib"

# set it in the stardist shared libs
install_name_tool -change ${src} ${dst} /Users/guest/anaconda3/lib/python3.6/site-packages/stardist/lib/stardist2d.cpython-36m-darwin.so
install_name_tool -change ${src} ${dst} /Users/guest/anaconda3/lib/python3.6/site-packages/stardist/lib/stardist3d.cpython-36m-darwin.so

Of course, this is only a temporary workaround.

[lauziere]

The first command gets me:

| => otool -L /Users/guest/anaconda3/lib/python3.6/site-packages/stardist/lib/stardist3d.cpython-36m-darwin.so
/Users/guest/anaconda3/lib/python3.6/site-packages/stardist/lib/stardist3d.cpython-36m-darwin.so:
/usr/local/opt/gcc@8/lib/gcc/8/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.25.0)
@rpath/libgomp.1.dylib (compatibility version 2.0.0, current version 2.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
/usr/local/lib/gcc/8/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

So our src is the same - @rpath/libgomp.1.dylib

I have libgomp.1.dylib in the following:

/usr/local/Cellar/gcc/9.2.0/lib/gcc/9/libgomp.1.dylib

So I ran your second block with src the same, then my dst="/usr/local/Cellar/gcc/9.2.0/lib/gcc/9/libgomp.1.dylib"

Afterwards I am able to import stardist.

Thank you so much Martin.

Why is this only a temporary workaround?

[maweigert]

Ha, great to hear that this worked! :)

Why is this only a temporary workaround?

Temporary was probably the wrong word - I meant something like "unsatisfactory ad-hoc workaround that doesn't really address the underlying problem" :)

[lauziere]

Fair enough - thanks again for your help.

I'll let you know how your method works for our data

[maweigert]

I'll let you know how your method works for our data

That'd be great! Any feedback is appreciated

[sofroniewn]

I can confirm this also worked for me on mac os x catalina 10.15.4, thanks for the detailed instructions!! And thanks for the amazing tool - I'm having fun segmenting with stardist inside napari :-)

[maweigert]

Hahah, great! :)

[markbuckup]

Hello,

I've followed these steps and have now had a new error appear when importing Stardist:

Reason: Incompatible library version: stardist2d.cpython-37m-darwin.so requires version 2.0.0 or later, but libgcc_s.1.dylib provides version 1.0.0

Is there a way to update libgcc_s.1.dylib? Any insight?

Thanks!

[mihaela-mihaylova]

Hi,
I have a similar problem.
I have a MacBook Pro M1 (2020) with MacOS 13.3.1 (Ventura).
I get the following error when trying to run StarDIst(tried both installing it through pip and from github, following these instructions):
ImportError: dlopen(/Users/mmihaylova/tf-stardist/lib/python3.10/site-packages/stardist-0.8.3-py3.10-macosx-11.1-arm64.egg/stardist/lib/stardist2d.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '___kmpc_critical'.
When I run otool -L /Users/mmihaylova/tf-stardist/lib/python3.10/site-packages/stardist-0.8.3-py3.10-macosx-11.1-arm64.egg/stardist/lib/stardist2d.cpython-310-darwin.so I get the following paths:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 904.4.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1).

Now, I replaced the first path with a path to the same library, which is available in miniconda. However, there are no other versions of libSystems.B.dylib, which I can find on my Mac. I also tried to provide a path to the libSystems.B.tbd, which I found, but this didn't work.

Do you have any idea why this would be? I also installed gcc11 and tried to build the github version, but it didn't work. Any suggestions would be appreciated.

Thanks for your time!

[blancfrederic]

Hi,
I've got the same issue as mihaela-mihaylova with a new MacBook Pro M2 Max (2023) + MacOS 13.3.1 (Ventura) ... I don't know how to proceed !
Thanks for your help !

[blancfrederic]

Finally it works !!

i) install gcc13 with homebrew (su installation)

ii) create symbolic links :
sudo ln -s /opt/homebrew/bin/g++-13 /usr/local/bin/
sudo ln -s /opt/homebrew/bin/gcc-13 /usr/local/bin/

iii) install stardist
conda create -y -n stardist-env python=3.9
conda activate stardist-env
conda install -c apple tensorflow-deps
pip install tensorflow-macos tensorflow-metal
CC=gcc-13 CXX=g++-13 pip install stardist

that's all folks !

[maweigert]

Finally it works !!

Indeed this error seems to pop up from time to time on some envs in a rather annoying way. So thanks a lot for digging into it @blancfrederic !

[vgastaldi]

Finally it works !!

i) install gcc13 with homebrew (su installation)

ii) create symbolic links : sudo ln -s /opt/homebrew/bin/g++-13 /usr/local/bin/ sudo ln -s /opt/homebrew/bin/gcc-13 /usr/local/bin/

iii) install stardist conda create -y -n stardist-env python=3.9 conda activate stardist-env conda install -c apple tensorflow-deps pip install tensorflow-macos tensorflow-metal CC=gcc-13 CXX=g++-13 pip install stardist

that's all folks !

Which Mac OS version are you using? On my system with 13.4.1 I tried your solution and couldn't make it work.

I still get the same issues as @mihaela-mihaylova.

[blancfrederic]

@vgastaldi
I've got the MacOs Version : 13.3.1 (a) (22E772610a)
What I did not mention is that I've got two accounts on my computer : an Admin and a Standard. As far I can remember homebrew was installed in the admin account.

[vgastaldi]

@vgastaldi I've got the MacOs Version : 13.3.1 (a) (22E772610a) What I did not mention is that I've got two accounts on my computer : an Admin and a Standard. As far I can remember homebrew was installed in the admin account.

Thanks for the reply @blancfrederic!

I have 13.4.1 here, but no idea if that would be the difference. Regarding the admin account, my account was not the first one created in the computer (IT did it first), but mine is marked as admin too.

I do have a newer version of gcc installed, but I both followed your instructions and set aliases for it.

[blancfrederic]

@vgastaldi Hope you find a solution !

[hey2homie]

@vgastaldi, I've had a similar problems to what you'd encountered and the solutions above didn't help. If you still have the environment, pip uninstall stardist followed by pip install --no-binary :all: stardist solved the problem for me.

[vgastaldi]

@vgastaldi, I've had a similar problems to what you'd encountered and the solutions above didn't help. If you still have the environment, pip uninstall stardist followed by pip install --no-binary :all: stardist solved the problem for me.

Thanks for the reply! I Unfortunately it hasn't worked yet as the installation of numpy always returns an error. I will still try to figure this out.

In the meantime I installed stardist in a Windows computer and it's running fine.