Python wrapping broken in 5.1.1

[MrTzschr]

Hello everyone,

I successfully compiled ITK 5.1 from the release branch using the following setup:

• Windows 10
• Visual Studio 2019
• Python 3.7.6 from Anaconda3
• CUDA Toolkit 10.1

The CMake flags i used:

BUILD_EXAMPLES=OFF
BUILD_SHARED_LIBS=OFF
BUILD_TESTING=OFF
ITK_LEGACY_SILENT=ON
ITK_USE_GPU=ON
ITK_WRAP_PYTHON=ON
ITK_WRAP_unsigned_long_long=ON
ITK_WRAP_signed_long_long=ON
Module_RTK=ON
Module_ITKCudaCommon=ON
RTK_BUILD_APPLICATIONS=ON
RTK_USE_CUDA=ON
CUDA_HAVE_GPU=ON

Compilation works without errors but python integration seems broken:

python -c "import itk"
ModuleNotFoundError: No module named 'itk'

Looking at the content of the WrapITK.pth file I suspected the paths (shortened here to the relevant parts) are incorrect, I have these:

C:/[...]/ITK-5.1-bin/Wrapping/Generators/Python/Release
C:/[...]/ITK-5.1-bin/Wrapping/Generators/Python/itk/Release
C:/[...]/ITK-5.1-bin/Wrapping/Generators/Python/itk

The last entry looks wrong to me and I thought it should point to the folder above, I also tried that and the import works but accessing itk modules doesn't:

python -c "import itk; itk.Image"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\[...]\ITK-5.1-bin\Wrapping\Generators\Python\Release\itkLazy.py", line 52, in __getattribute__
    itkBase.LoadModule(module, namespace)
  File "C:\[...]\ITK-5.1-bin\Wrapping\Generators\Python\Release\itkBase.py", line 91, in LoadModule
    LoadModule(dep, namespace)
  File "C:\[...]\ITK\ITK-5.1-bin\Wrapping\Generators\Python\Release\itkBase.py", line 99, in LoadModule
    module = loader.load(swig_module_name)
  File "C:\[...]\ITK-5.1-bin\Wrapping\Generators\Python\Release\itkBase.py", line 220, in load
    module.__loader__.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 724, in exec_module
  File "<frozen importlib._bootstrap_external>", line 859, in get_code
  File "<frozen importlib._bootstrap_external>", line 916, in get_data
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\[...]\\ITK-5.1-bin\\Wrapping\\Generators\\Python\\Release\\itk\\ITKPyBasePython.py'

I noticed that the structure of python files within ITK changed from 5.0.1 (ITK-5.1-bin/lib) to 5.1 (ITK-5.1-bin/Wrapping/Generators/Python/). Maybe something got messed up here.
Any help is appreciated.

[dzenanz]

I am experiencing similar issues. @thewtex is on vacation next two weeks, I am afraid this will have to wait until he comes back.

[brad-t-moore]

I've been digging through it as well. I've worked through some path issues that were causing the errors, but I now have a crash whenever the lazy loading kicks in. If I can get that nailed down tomorrow, I'll offer a workaround. I wouldn't suggest making it a PR unless I could test that the Python wheel-making process works, but it might get folks working until @thewtex gets back.

[brad-t-moore]

Not 100% confirmed, but this @aylward says it's working for him with Ninja + Visual Studio (which uses Ninja's path layout), so it might be the path issue is specific to Visual Studio's output folder structure.

[brad-t-moore]

(1) C:\src\ITK-fork-build\Wrapping\Generators\Python\Release
-- contains itkBase.py, itkLazy.py, itkConfig.py, WrapITK.pth, etc.

(2) C:\src\ITK-fork-build\Wrapping\Generators\Python\itk
-- contains __init__.py, itkAbsImageFilterPython.py, etc.

(3) C:\src\ITK-fork-build\Wrapping\Generators\Python\itk\Release
-- contains _ITKAnisotropicSmoothingPython.pyd, _ITKAntiAliasPython.pyd, etc.

WrapITK.pth contents:

# Python pth file to add WrapITK's path to sys.path.
C:/src/ITK-fork-build/Wrapping/Generators/Python/Release
C:/src/ITK-fork-build/Wrapping/Generators/Python/itk/Release
C:/src/ITK-fork-build/Wrapping/Generators/Python/itk

OK, from what I gather, itkBase.py etc. need to be in a root directory, itk needs to be a subdirectory within that root, and all the _ITKBlahBlahPython.pyd files need to be alongside the itkBlahBlahPython.py files in the itk subdirectory.

So, move all the files in (3) into (2), then move the folder of (2) into (1). You WrapITK.pth file will now look like:

# Python pth file to add WrapITK's path to sys.path.
C:/src/ITK-fork-build-modified/Wrapping/Generators/Python/Release
C:/src/ITK-fork-build-modified/Wrapping/Generators/Python/Release/itk

[MrTzschr]

Thanks @brad-t-moore. I can confirm that building with CMake, Ninja and VS2019 works fine and WrapITK.pth shows only two paths:

../ITK-5.1-bin-ninja/Wrapping/Generators/Python
../ITK-5.1-bin-ninja/Wrapping/Generators/Python/itk

For the standard VS2019 (without ninja) I also have a folder ${BUILD_TYPE} containing an empty folder named itk under
/Wrapping/Generators/Python
so definitely path layout issues.

EDIT: Of course the recommend way of moving files works as expected, so that all files reside in Python/Release and Python/Release/itk.
Thank you very much!

[thewtex]

@MrTzschr @brad-t-moore a patch is available to enable debugging with WrapITK.pth in #2032