MacOS PyInstaller - No module named 'skimage.filters.rank.core_cy_3d'

[ArekMC]

Description

I cannot bundle scikit with MacOS BigSur. It runs without issues when not bundled.
I figured out, that is is related to the segmentation module.
All other modules that I am using (e.g. transform, measure, draw), I was able to bundle them with pyinstaller successfully.

I already tried:

• Different python versions (3.7, 3.9)
• Different scikit-versions (1.8.1, dev)
• Build scikit from source
• Different pyinstaller versions (4.3, dev)

But no success. It always tells me, that ModuleNotFoundError: No module named 'skimage.filters.rank.core_cy_3d'.

The build is running without any error message and completes successfully.

Way to reproduce

Create file called main.py.

from skimage import segmentation

print("OK")

Build code

pyinstaller -y --onedir --distpath=./bin --specpath=./bin -n scibuild ./main.py

Execute programm

./bin/scibuild/scibuild

Version information

3.9.5 (default, May  4 2021, 03:36:27) 
[Clang 12.0.0 (clang-1200.0.32.29)]
macOS-11.3.1-x86_64-i386-64bit
scikit-image version: 0.19.0.dev0
numpy version: 1.20.3

Traceback

Traceback (most recent call last):
  File "main.py", line 1, in <module>
    from skimage import segmentation
  File "PyInstaller/loader/pyimod03_importers.py", line 540, in exec_module
  File "skimage/segmentation/__init__.py", line 3, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 540, in exec_module
  File "skimage/segmentation/active_contour_model.py", line 6, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 540, in exec_module
  File "skimage/filters/__init__.py", line 19, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 540, in exec_module
  File "skimage/filters/rank/__init__.py", line 1, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 540, in exec_module
  File "skimage/filters/rank/generic.py", line 58, in <module>
  File "skimage/filters/rank/generic_cy.pyx", line 1, in init skimage.filters.rank.generic_cy
ModuleNotFoundError: No module named 'skimage.filters.rank.core_cy_3d'
[56974] Failed to execute script main

[jni]

I'm not very experienced with pyinstaller, but as I recall, there's a bit of a trick to using pyinstaller, in that you have to explicitly point it to any binary dynamic linked libraries that need to be bundled, if they are not directly imported by some Python code. That particular file only contains C code so it cannot be imported from Python.

So, I think you need to build the binary file in-place (python setup.py build_ext -i to make the binaries, and then use the --add-binary skimage/filters/rank/core_cy_3d.cpython-....so option for pyinstaller. (The ... is Python version and platform specific, you'll have to look for the right stuff to fill in in your case.)

Some more resources:

https://stackoverflow.com/questions/38791685/how-do-i-include-dll-file-in-executable-using-pyinstaller

https://pyinstaller.readthedocs.io/en/stable/spec-files.html#adding-binary-files

I think this is a PyInstaller issue and not a scikit-image bug, so I am going to close the issue, but feel free to continue the discussion if the above links don't quite solve your problem. I also know that Big Sur is funny with .so files now, so there might be additional tricks needed.

[ArekMC]

For those who have similar issues and are looking for a fix:
I found two solutions that worked out for me:

1. Add --hidden-import="skimage.filters.rank.core_cy_3d" to the pyinstaller build command
2. Import the package into the source code with from skimage.filters.rank import core_cy_3d

Best regards

[AbdelsalamHaa]

@ArekMC Thanks a lot. Your solution worked for me.
I'm using windows though.