Segfault when partially exporting class with virtual methods #2132
Description
Environment: Ubuntu 18.04, GCC 7.4.0, Python 3.6.9, pybind11 2.4.3
Suppose we want to wrap a shared library libfoo.so, exporting a base class Foo (i.e., with a vtable, in this case just a virtual destructor).
Exporting the whole class: OK
If we export the whole class (i.e., add visibility("default") to the whole class), then everything works as expected:
CMakeLists.txt
cmake_minimum_required(VERSION 3.1.0)
add_subdirectory(pybind11)
add_library(foo SHARED foo.cpp)
set_target_properties(foo PROPERTIES CXX_VISIBILITY_PRESET hidden)
pybind11_add_module(pyfoo pyfoo.cpp)
target_link_libraries(pyfoo PRIVATE foo)
foo.h
#define FOO_API __attribute__((visibility("default")))
class FOO_API Foo { public: virtual ~Foo(); };
foo.cpp
#include "foo.h"
Foo::~Foo() {}
pyfoo.cpp
#include <pybind11/pybind11.h>
#include "foo.h"
PYBIND11_MODULE(pyfoo, m) {
pybind11::class_<Foo>(m, "Foo")
.def(pybind11::init<>());
}
test.py
import pyfoo
f = pyfoo.Foo()
print("Test succeeded")
Build and run:
$ mkdir build && cd build && cmake .. && make
$ PYTHONPATH=. python3 ../test.py
Test succeeded
Partially exporting the class: Segfault
However, my understanding is that to minimize the number of exported symbols, it is better practice to selectively export a desired selection of member functions, rather than the whole class. For a concrete example, see SdfData as part of Pixar's USD.
In our minimal example, this would look like the following (note that the FOO_API is now declared specifically for the destructor, not the whole class):
foo.h
#define FOO_API __attribute__((visibility("default")))
class Foo { public: FOO_API virtual ~Foo(); };
Build and run:
$ make
$ PYTHONPATH=. python3 ../test.py
Segmentation fault (core dumped)
(Note that it doesn't matter if the virtual method is the destructor or another method. The mere presence of a virtual method causes a segfault when the whole class isn't exported, even if such virtual method isn't wrapped. There is no segfault if we remove virtual from the above example)
Questions
Is this a bug?
If not, is there any workaround/method to be able to partially export a base class with virtual methods? The issue seems to be that there are missing exported typeinfo/vtable from the shared library: is there a way to explicitly export those without having to export the whole class?
Thanks!
More debug info
When exporting the whole class:
$ nm -o -C -D *.so | grep "Foo"
libfoo.so:0000000000000818 T Foo::~Foo()
libfoo.so:00000000000007fa T Foo::~Foo()
libfoo.so:00000000000007fa T Foo::~Foo()
libfoo.so:0000000000200df8 V typeinfo for Foo
libfoo.so:000000000000084d V typeinfo name for Foo
libfoo.so:0000000000200dd8 V vtable for Foo
pyfoo.cpython-36m-x86_64-linux-gnu.so: U typeinfo for Foo
pyfoo.cpython-36m-x86_64-linux-gnu.so: U vtable for Foo
When exporting the destructor only:
$ nm -o -C -D *.so | grep "Foo"
libfoo.so:0000000000000794 T Foo::~Foo()
libfoo.so:000000000000077a T Foo::~Foo()
libfoo.so:000000000000077a T Foo::~Foo()
pyfoo.cpython-36m-x86_64-linux-gnu.so: U typeinfo for Foo
pyfoo.cpython-36m-x86_64-linux-gnu.so: U vtable for Foo
Stacktrace (the segfault is with the import pyfoo call)
$ PYTHONPATH=. gdb python3
(gdb) r ../test.py
Starting program: /usr/bin/python3 ../test.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff42b72a9 in GlobalError::PushToStack() () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0
(gdb) bt
#0 0x00007ffff42b72a9 in GlobalError::PushToStack() () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0
#1 0x00007ffff433b082 in pkgInitConfig(Configuration&) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0
#2 0x00007ffff45d8ed8 in ?? () from /usr/lib/python3/dist-packages/apt_pkg.cpython-36m-x86_64-linux-gnu.so
#3 0x000000000050a8af in ?? ()
#4 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#5 0x0000000000508245 in ?? ()
#6 0x00000000005167b9 in ?? ()
#7 0x00000000005678ee in PyCFunction_Call ()
#8 0x000000000051171e in _PyEval_EvalFrameDefault ()
#9 0x0000000000508245 in ?? ()
#10 0x000000000050a080 in ?? ()
#11 0x000000000050aa7d in ?? ()
#12 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#13 0x0000000000509d48 in ?? ()
#14 0x000000000050aa7d in ?? ()
#15 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#16 0x0000000000509d48 in ?? ()
#17 0x000000000050aa7d in ?? ()
#18 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#19 0x0000000000509d48 in ?? ()
#20 0x000000000050aa7d in ?? ()
#21 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#22 0x0000000000509455 in _PyFunction_FastCallDict ()
#23 0x00000000005a55a1 in _PyObject_FastCallDict ()
#24 0x00000000005a65de in _PyObject_CallMethodIdObjArgs ()
#25 0x00000000004f729d in PyImport_ImportModuleLevelObject ()
#26 0x000000000050e4f1 in _PyEval_EvalFrameDefault ()
#27 0x0000000000508245 in ?? ()
#28 0x00000000005167b9 in ?? ()
#29 0x00000000005678ee in PyCFunction_Call ()
#30 0x000000000051171e in _PyEval_EvalFrameDefault ()
#31 0x0000000000508245 in ?? ()
#32 0x000000000050a080 in ?? ()
#33 0x000000000050aa7d in ?? ()
#34 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#35 0x0000000000509d48 in ?? ()
#36 0x000000000050aa7d in ?? ()
#37 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#38 0x0000000000509d48 in ?? ()
#39 0x000000000050aa7d in ?? ()
#40 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#41 0x0000000000509d48 in ?? ()
#42 0x000000000050aa7d in ?? ()
#43 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#44 0x0000000000509455 in _PyFunction_FastCallDict ()
#45 0x00000000005a55a1 in _PyObject_FastCallDict ()
#46 0x00000000005a65de in _PyObject_CallMethodIdObjArgs ()
#47 0x00000000004f729d in PyImport_ImportModuleLevelObject ()
#48 0x000000000050e4f1 in _PyEval_EvalFrameDefault ()
#49 0x0000000000508245 in ?? ()
#50 0x00000000005167b9 in ?? ()
#51 0x00000000005678ee in PyCFunction_Call ()
#52 0x000000000051171e in _PyEval_EvalFrameDefault ()
#53 0x0000000000508245 in ?? ()
#54 0x000000000050a080 in ?? ()
#55 0x000000000050aa7d in ?? ()
#56 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#57 0x0000000000509d48 in ?? ()
#58 0x000000000050aa7d in ?? ()
#59 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#60 0x0000000000509d48 in ?? ()
#61 0x000000000050aa7d in ?? ()
#62 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#63 0x0000000000509d48 in ?? ()
#64 0x000000000050aa7d in ?? ()
#65 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#66 0x0000000000509455 in _PyFunction_FastCallDict ()
#67 0x00000000005a55a1 in _PyObject_FastCallDict ()
#68 0x00000000005a65de in _PyObject_CallMethodIdObjArgs ()
#69 0x00000000004f729d in PyImport_ImportModuleLevelObject ()
#70 0x000000000050e4f1 in _PyEval_EvalFrameDefault ()
#71 0x0000000000508245 in ?? ()
#72 0x00000000005167b9 in ?? ()
#73 0x00000000005678ee in PyCFunction_Call ()
#74 0x000000000051171e in _PyEval_EvalFrameDefault ()
#75 0x0000000000508245 in ?? ()
#76 0x000000000050a080 in ?? ()
#77 0x000000000050aa7d in ?? ()
#78 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#79 0x0000000000509d48 in ?? ()
---Type <return> to continue, or q <return> to quit---
#80 0x000000000050aa7d in ?? ()
#81 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#82 0x0000000000509d48 in ?? ()
#83 0x000000000050aa7d in ?? ()
#84 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#85 0x0000000000509d48 in ?? ()
#86 0x000000000050aa7d in ?? ()
#87 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#88 0x0000000000509455 in _PyFunction_FastCallDict ()
#89 0x00000000005a55a1 in _PyObject_FastCallDict ()
#90 0x00000000005a65de in _PyObject_CallMethodIdObjArgs ()
#91 0x00000000004f729d in PyImport_ImportModuleLevelObject ()
#92 0x000000000050e4f1 in _PyEval_EvalFrameDefault ()
#93 0x0000000000508245 in ?? ()
#94 0x00000000005167b9 in ?? ()
#95 0x00000000005678ee in PyCFunction_Call ()
#96 0x000000000051171e in _PyEval_EvalFrameDefault ()
#97 0x0000000000508245 in ?? ()
#98 0x000000000050a080 in ?? ()
#99 0x000000000050aa7d in ?? ()
#100 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#101 0x0000000000509d48 in ?? ()
#102 0x000000000050aa7d in ?? ()
#103 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#104 0x0000000000509d48 in ?? ()
#105 0x000000000050aa7d in ?? ()
#106 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#107 0x0000000000509d48 in ?? ()
#108 0x000000000050aa7d in ?? ()
#109 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#110 0x0000000000509455 in _PyFunction_FastCallDict ()
#111 0x00000000005a55a1 in _PyObject_FastCallDict ()
#112 0x00000000005a65de in _PyObject_CallMethodIdObjArgs ()
#113 0x00000000004f729d in PyImport_ImportModuleLevelObject ()
#114 0x0000000000514804 in ?? ()
#115 0x00000000005678b3 in PyCFunction_Call ()
#116 0x000000000051171e in _PyEval_EvalFrameDefault ()
#117 0x0000000000508245 in ?? ()
#118 0x000000000050a080 in ?? ()
#119 0x000000000050aa7d in ?? ()
#120 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#121 0x0000000000509d48 in ?? ()
#122 0x000000000050aa7d in ?? ()
#123 0x000000000050c5b9 in _PyEval_EvalFrameDefault ()
#124 0x0000000000509455 in _PyFunction_FastCallDict ()
#125 0x00000000005a55a1 in _PyObject_FastCallDict ()
#126 0x00000000005a65de in _PyObject_CallMethodIdObjArgs ()
#127 0x00000000004f729d in PyImport_ImportModuleLevelObject ()
#128 0x000000000050e4f1 in _PyEval_EvalFrameDefault ()
#129 0x0000000000509455 in _PyFunction_FastCallDict ()
#130 0x00000000005a55a1 in _PyObject_FastCallDict ()
#131 0x00000000006386cb in PyErr_PrintEx ()
#132 0x0000000000638ab3 in PyRun_SimpleFileExFlags ()
#133 0x0000000000639631 in Py_Main ()
#134 0x00000000004b0f40 in main ()