Skip to content

Commit

Permalink
python bindings: Load all modules with RTLD_GLOBAL
Browse files Browse the repository at this point in the history
libdnf's python bindings are implemented by a set of C++ shared objects
generated by swig.  Some generated code is duplicated between modules,
in particular the SwigPyIterator class templates, which use exceptions
of type swig::stop_iteration to signal an end-of-iteration condition.
The modules do not depend on each other and thus belong to different
DAGs from the perspective of the runtime linker.

It turns out that this stop_iteration exception can be thrown between
modules.  This happens at least during dnf startup with python 3.9:

cli.py(935):         subst.update_from_etc(from_root, varsdir=conf._get_value('varsdir'))
 --- modulename: config, funcname: _get_value
config.py(102):         method = getattr(self._config, name, None)
config.py(103):         if method is None:
config.py(105):         return method().getValue()
 --- modulename: conf, funcname: varsdir
conf.py(1183):         return _conf.ConfigMain_varsdir(self)
 --- modulename: conf, funcname: getValue
conf.py(512):         return _conf.OptionStringList_getValue(self)
 --- modulename: substitutions, funcname: update_from_etc
substitutions.py(47):         for vars_path in varsdir:
 --- modulename: module, funcname: __iter__
module.py(557):         return self.iterator()
 --- modulename: module, funcname: iterator
module.py(555):         return _module.VectorString_iterator(self)
 --- modulename: transaction, funcname: __next__
transaction.py(94):         return _transaction.SwigPyIterator___next__(self)

In particular, the module and transaction modules are somehow both
involved: module returns the iterator, and transaction advances the
iterator.  Both modules contain the same iterator code, so I'm not sure
why it works this way.  The behaviour is sensitive to import order; for
example, if transaction is imported before module, then the code above
ends up using module's implementation of SwigPyItreator___next__.

In any case, the use of swig::stop_iteration is broken in the above
scenario since the exception is thrown by module with module.so's copy
of the swig::stop_iteration type info, and caught by transaction.so
using transaction.so's copy of the type info, resulting in an uncaught
exception.

Work around the problem by loading all modules with RTLD_GLOBAL to
ensure that RTTI is unique.  This is required when throwing exceptions
across DSO boundaries, see https://gcc.gnu.org/faq.html#dso for example.
  • Loading branch information
markjdb authored and jan-kolarik committed Sep 6, 2023
1 parent 39098f3 commit 8a8548d
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion bindings/python/__init__.py
Expand Up @@ -6,11 +6,14 @@
import sys, os
sys.setdlopenflags(os.RTLD_NOW | os.RTLD_GLOBAL)
from . import error
sys.setdlopenflags(os.RTLD_NOW)

# Other modules also need to be loaded with RTLD_GLOBAL to preserve uniqueness
# of RTTI. There are code paths where an exception thrown in one module is
# supposed to be caught in another.
from . import common_types
from . import conf
from . import module
from . import repo
from . import transaction
from . import utils
sys.setdlopenflags(os.RTLD_NOW)

0 comments on commit 8a8548d

Please sign in to comment.