Segmentation fault when importing mapnik

[smomapz]

Hi,

I am trying to install Mapnik and the Mapnik Python bindings with Python3 support in Docker but I am failing at multiple points. Best result I could get was successful compilation of mapnik and the bindings but when I import mapnik I get a Segmentation fault and Python crashes.

This is my setup:

• ubuntu:focal
• Python3.8
• Mapnik branch v3.0.x
• python-mapnik branch v3.0.x

I am compiling like this:

WORKDIR /src
RUN git clone https://github.com/mapnik/mapnik.git
WORKDIR /src/mapnik
RUN git checkout v3.0.x
RUN git submodule update --init
RUN ./configure CUSTOM_DEFINES="-DACCEPT_USE_OF_DEPRECATED_PROJ_API_H=1"
RUN make
RUN make install
RUN python3 -m venv /venv
RUN /venv/bin/pip install --upgrade 'setuptools<45.0.0'
RUN /venv/bin/pip install -r requirements.txt
WORKDIR /src
RUN git clone https://github.com/mapnik/python-mapnik.git
WORKDIR /src/python-mapnik
RUN git checkout --force v3.0.x
ENV BOOST_PYTHON_LIB='boost_python38'
RUN sed -i 's/sys.exec_prefix/sys.base_exec_prefix/g' setup.py # Fix wrong pathing when using Python virtual environment
ENV PYCAIRO='true'
RUN setup.py install

Note: I need to compile the libraries from source because I need to apply custom patches to both mapnik and python-mapnik. The tests I ran so far are without the patches though to exclude them from the list of possible causes for these issues.

[smomapz]

I tried analysing what is happening when importing mapnik using gdb giving me this backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x00007fb015471ac4 in boost::python::objects::class_type() () from /lib/x86_64-linux-gnu/libboost_python38.so.1.71.0
(gdb) backtrace
#0  0x00007fb015471ac4 in boost::python::objects::class_type() () from /lib/x86_64-linux-gnu/libboost_python38.so.1.71.0
#1  0x00007fb0154729ca in boost::python::objects::class_base::class_base(char const*, unsigned long, boost::python::type_info const*, char const*) ()
   from /lib/x86_64-linux-gnu/libboost_python38.so.1.71.0
#2  0x00007fb016736ad5 in boost::python::class_<mapnik::query, boost::python::detail::not_specified, boost::python::detail::not_specified, boost::python::detail::not_specified>::class_<boost::python::init<mapnik::box2d<double>, std::tuple<double, double> const&, double, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_> > (i=..., 
    doc=0x7fb016819688 "a spatial query data object", name=0x7fb0168196a4 "Query", this=0x7ffc191244c0) at /usr/include/boost/mpl/for_each.hpp:97
#3  export_query () at src/mapnik_query.cpp:93
#4  0x00007fb016725677 in init_module__mapnik () at src/mapnik_python.cpp:726
#5  0x00007fb01547b233 in boost::python::handle_exception_impl(boost::function0<void>) () from /lib/x86_64-linux-gnu/libboost_python38.so.1.71.0
#6  0x00007fb01547c679 in ?? () from /lib/x86_64-linux-gnu/libboost_python38.so.1.71.0
#7  0x00007fb01547c49a in boost::python::detail::init_module(PyModuleDef&, void (*)()) () from /lib/x86_64-linux-gnu/libboost_python38.so.1.71.0
#8  0x000000000067ce1a in _PyImport_LoadDynamicModuleWithSpec ()
#9  0x000000000067e23d in ?? ()
#10 0x00000000005c078c in ?? ()
#11 0x00000000005f1669 in PyVectorcall_Call ()
#12 0x000000000056d299 in _PyEval_EvalFrameDefault ()
#13 0x0000000000565972 in _PyEval_EvalCodeWithName ()
#14 0x00000000005f1d85 in _PyFunction_Vectorcall ()
#15 0x000000000056c475 in _PyEval_EvalFrameDefault ()
#16 0x00000000005f1b8b in _PyFunction_Vectorcall ()
#17 0x00000000005677c7 in _PyEval_EvalFrameDefault ()
#18 0x00000000005f1b8b in _PyFunction_Vectorcall ()
#19 0x000000000056769f in _PyEval_EvalFrameDefault ()
#20 0x00000000005f1b8b in _PyFunction_Vectorcall ()
#21 0x000000000056769f in _PyEval_EvalFrameDefault ()
#22 0x00000000005f1b8b in _PyFunction_Vectorcall ()
#23 0x000000000056769f in _PyEval_EvalFrameDefault ()
#24 0x00000000005f1b8b in _PyFunction_Vectorcall ()
#25 0x00000000005eef71 in ?? ()
#26 0x00000000005ef3d4 in _PyObject_CallMethodIdObjArgs ()
#27 0x000000000054f39d in PyImport_ImportModuleLevelObject ()
#28 0x0000000000569b5a in _PyEval_EvalFrameDefault ()
#29 0x0000000000565972 in _PyEval_EvalCodeWithName ()
#30 0x0000000000686053 in PyEval_EvalCode ()
#31 0x00000000005fc080 in ?? ()
#32 0x00000000005c078c in ?? ()
#33 0x00000000005f1669 in PyVectorcall_Call ()
#34 0x000000000056d299 in _PyEval_EvalFrameDefault ()
#35 0x0000000000565972 in _PyEval_EvalCodeWithName ()
#36 0x00000000005f1d85 in _PyFunction_Vectorcall ()
#37 0x000000000056c475 in _PyEval_EvalFrameDefault ()
#38 0x00000000005f1b8b in _PyFunction_Vectorcall ()
#39 0x00000000005677c7 in _PyEval_EvalFrameDefault ()
#40 0x00000000005f1b8b in _PyFunction_Vectorcall ()
#41 0x000000000056769f in _PyEval_EvalFrameDefault ()
#42 0x00000000005f1b8b in _PyFunction_Vectorcall ()
#43 0x000000000056769f in _PyEval_EvalFrameDefault ()
#44 0x00000000005f1b8b in _PyFunction_Vectorcall ()
#45 0x00000000005eef71 in ?? ()
#46 0x00000000005ef3d4 in _PyObject_CallMethodIdObjArgs ()
#47 0x000000000054f39d in PyImport_ImportModuleLevelObject ()
#48 0x0000000000569b5a in _PyEval_EvalFrameDefault ()
#49 0x0000000000565972 in _PyEval_EvalCodeWithName ()
#50 0x0000000000686053 in PyEval_EvalCode ()
#51 0x00000000006753d1 in ?? ()
#52 0x000000000067544f in ?? ()
#53 0x0000000000675507 in PyRun_FileExFlags ()
#54 0x000000000067758a in PyRun_SimpleFileExFlags ()
#55 0x00000000006ae99e in Py_RunMain ()
#56 0x00000000006aed29 in Py_BytesMain ()
#57 0x00007fb016d570b3 in __libc_start_main (main=0x4ebd20 <main>, argc=2, argv=0x7ffc19126b28, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffc19126b18) at ../csu/libc-start.c:308
#58 0x00000000005f62ee in _start () 

It looks like and issue with boost_python but I am not able to analyse this further.

[smomapz]

I found out that the Segmentation Fault only appears when installing into the python virtual environment. When installing with /usr/bin/python3 setup.py install mapnik can be imported.

Any ideas why it's not working in venvs?

[Maxxen]

I cannot get this to work even outside of venv. I've also tried to compile boost from scratch, both using version 1.67 and 1.71 but still receive segfault with the exact same backtrace as soon as I try to import mapnik.

I see that you're using a Dockerfile, would you mind sharing it?

[smomapz]

The example above is pretty much everything you need to install python, except for not using the venv. Imho the rest of the Dockerfile is irrelevant and I don't wan't to share it since its corporate intern.

Does that not work for you?

[Maxxen]

Thanks for replying again after such a long time. No worries, I got it working eventually.

For the record, in addition to the setup smomapz described I also attempted to compile mapnik+boost+proj+geos from scratch, as I assumed that I needed to recompile mapnik as well in order to create bindings for Python 3.9. I can only assume that something went wrong during that process and that my segfault occured due to a faulty mapnik or dependency installation. If I were to guess, something down the compile chain probably picked up system Python2.7 headers instead, causing a mismatch. Turns out I only really had to recompile boost-python against the Python3.9 headers to change the python version, after that it simply worked to install and link against libmapnik3.0/-dev from the ubuntu package repo (focal).