Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
ldso: Teach dynamic linking to handle library -> library dependencies
Currently Emscripten allows to create shared libraries (DSO) and link them to main module. However a shared library itself cannot be linked to another shared library. The lack of support for DSO -> DSO linking becomes problematic in cases when there are several shared libraries that all need to use another should-be shared functionality, while linking that should-be shared functionality to main module is not an option for size reasons. My particular use-case is SciPy support for Pyodide: pyodide/pyodide#211, pyodide/pyodide#240 where several of `*.so` scipy modules need to link to LAPACK. If we link to LAPACK statically from all those `*.so` - it just blows up compiled size pyodide/pyodide#211 (comment) and if we link in LAPACK statically to main module, the main module size is also increased ~2x, which is not an option, since LAPACK functionality is not needed by every Pyodide user. This way we are here to add support for DSO -> DSO linking: 1. similarly to how it is already working for main module -> side module linking, when building a side module it is now possible to specify -s RUNTIME_LINKED_LIBS=[...] with list of shared libraries that side module needs to link to. 2. to store that information, for asm.js, similarly to how it is currently handled for main module (which always has js part), we transform RUNTIME_LINKED_LIBS to libModule.dynamicLibraries = [...] (see src/preamble_sharedlib.js) 3. for wasm module, in order to store the information about to which libraries a module links, we could in theory use "module" attribute in wasm imports. However currently emscripten almost always uses just "env" for that "module" attribute, e.g. (import "env" "abortStackOverflow" (func $fimport$0 (param i32))) (import "env" "_ffunc1" (func $fimport$1)) ... and this way we have to embed the information about required libraries for the dynamic linker somewhere else. What I came up with is to extend "dylink" section with information about which shared libraries a shared library needs. This is similar to DT_NEEDED entries in ELF. (see tools/shared.py) 4. then, the dynamic linker (loadDynamicLibrary) is reworked to handle that information: - for asm.js, after loading a libModule, we check libModule.dynamicLibraries and post-load them recursively. (it would be better to load needed modules before the libModule, but for js we currently have to first eval whole libModule's code to be able to read .dynamicLibraries) - for wasm the needed libraries are loaded before the wasm module in question is instantiated. (see changes to loadWebAssemblyModule for details)
- Loading branch information