ImportError [...]/_filepath.cpython-39-x86_64-linux-gnu.so: undefined symbol: VSIAllocFilesystemPluginCallbacksStruct

[sebastic]

The test suite fails for the Debian package build:

I: pybuild pybuild:300: cp -r /<<PKGBUILDDIR>>/tests /<<PKGBUILDDIR>>/.pybuild/cpython3_3.9_rasterio/build
I: pybuild base:239: cd /<<PKGBUILDDIR>>/.pybuild/cpython3_3.9_rasterio/build; python3.9 -m pytest tests
ImportError while loading conftest '/<<PKGBUILDDIR>>/.pybuild/cpython3_3.9_rasterio/build/tests/conftest.py'.
tests/conftest.py:17: in <module>
    import rasterio
rasterio/__init__.py:14: in <module>
    from rasterio.crs import CRS
rasterio/crs.pyx:1: in init rasterio.crs
    ???
rasterio/_base.pyx:22: in init rasterio._base
    ???
rasterio/dtypes.py:10: in <module>
    from rasterio.env import GDALVersion
rasterio/env.py:16: in <module>
    from rasterio._env import (
rasterio/_env.pyx:1: in init rasterio._env
    ???
E   ImportError: /<<PKGBUILDDIR>>/.pybuild/cpython3_3.9_rasterio/build/rasterio/_filepath.cpython-39-x86_64-linux-gnu.so: undefined symbol: VSIAllocFilesystemPluginCallbacksStruct
E: pybuild pybuild:369: test: plugin distutils failed with: exit code=4: cd /<<PKGBUILDDIR>>/.pybuild/cpython3_3.9_rasterio/build; python3.9 -m pytest tests

Expected behavior and actual behavior.

No undefined symbols when importing rasterio.

Steps to reproduce the problem.

Run the testsuite as part of the Debian package build.

Environment Information

• rasterio version: 1.3.0
• GDAL version: 3.5.0
• Default Python version: 3.10.5 (main, Jun 8 2022, 09:26:22) [GCC 11.3.0]
• Additional Python version: 3.9.13 (main, Jun 8 2022, 09:45:57) [GCC 11.3.0]
• Operation System Information: Linux-5.17.0-2-amd64-x86_64-with-glibc2.33

Installation Method

Debian package (source package build)

[sebastic]

From port/cpl_vsi.h in GDAL:

/**
 * return a VSIFilesystemPluginCallbacksStruct to be populated at runtime with handler callbacks
 * @since GDAL 3.0
 */
VSIFilesystemPluginCallbacksStruct CPL_DLL *VSIAllocFilesystemPluginCallbacksStruct( void );

The symbol is not marked as extern "C" so C++ symbol mangling may be the cause.

[sgillies]

Thanks for the report @sebastic ! Do you have any idea how this slipped past our CI tests?

[sebastic]

Likely because the build environments are different.

GDAL 3.5.0 is also used in CI, so either that build is different (e.g. older compilers with different behaviour) or my suspicion about the symbol is wrong.

CI runs tests from the root of the source tree, pybuild runs the tests from the build directory to which the relevant files from the source tree are copied.

[sebastic]

Rebuilding rasterio with GDAL 3.5.1 resolves the issue, because of this change: OSGeo/gdal@c48fe91

[sgillies]

@rouault if you have a moment, can you help me understand why this error didn't occur in testing rasterio's manylinux2014, win64 (amd), macos (10.11 deployment target) wheels? Manylinux 2014 would have have a max glibc version of 2.17.

[rouault]

The symbol is not marked as extern "C" so C++ symbol mangling may be the cause.

the whole file is within CPL_C_START / CPL_C_END which expand to extern "C" { ... }, so the symbol is C exported

@rouault if you have a moment, can you help me understand why this error didn't occur in testing rasterio's manylinux2014, win64 (amd), macos (10.11 deployment target) wheels

Perhaps because you build GDAL with autoconf rather than cmake ? (as pointed by @sebastic, GDAL 3.5.0 had an issue in which the file defining VSIAllocFilesystemPluginCallbacksStruct() was not built in CMake builds)

[sgillies]

Thanks @rouault !