ldso: Teach dynamic linking to handle library -> library dependencies

Currently Emscripten allows to create shared libraries (DSO) and link
them to main module. However a shared library itself cannot be linked to
another shared library.

The lack of support for DSO -> DSO linking becomes problematic in cases when
there are several shared libraries that all need to use another should-be
shared functionality, while linking that should-be shared functionality to main
module is not an option for size reasons. My particular use-case is SciPy
support for Pyodide:

        pyodide/pyodide#211,
        pyodide/pyodide#240

where several of `*.so` scipy modules need to link to LAPACK. If we link to
LAPACK statically from all those `*.so` - it just blows up compiled size

        pyodide/pyodide#211 (comment)

and if we link in LAPACK statically to main module, the main module size is
also increased ~2x, which is not an option, since LAPACK functionality is not
needed by every Pyodide user.

This way we are here to add support for DSO -> DSO linking:

1. similarly to how it is already working for main module -> side module
   linking, when building a side module it is now possible to specify

        -s RUNTIME_LINKED_LIBS=[...]

   with list of shared libraries that side module needs to link to.

2. to store that information, for asm.js, similarly to how it is currently
   handled for main module (which always has js part), we transform
   RUNTIME_LINKED_LIBS to

        libModule.dynamicLibraries = [...]

   (see src/preamble_sharedlib.js)

3. for wasm module, in order to store the information about to which libraries
   a module links, we could in theory use "module" attribute in wasm imports.
   However currently emscripten almost always uses just "env" for that "module"
   attribute, e.g.

        (import "env" "abortStackOverflow" (func $fimport$0 (param i32)))
        (import "env" "_ffunc1" (func $fimport$1))
        ...

   and this way we have to embed the information about required libraries for
   the dynamic linker somewhere else.

   What I came up with is to extend "dylink" section with information about
   which shared libraries a shared library needs. This is similar to DT_NEEDED
   entries in ELF.

   (see tools/shared.py)

4. then, the dynamic linker (loadDynamicLibrary) is reworked to handle that information:

   - for asm.js, after loading a libModule, we check libModule.dynamicLibraries
     and post-load them recursively. (it would be better to load needed modules
     before the libModule, but for js we currently have to first eval whole
     libModule's code to be able to read .dynamicLibraries)

   - for wasm the needed libraries are loaded before the wasm module in
     question is instantiated.

     (see changes to loadWebAssemblyModule for details)

emcc.py

@@ -2602,7 +2602,7 @@ def do_binaryen(target, asm_target, options, memfile, wasm_binary_target,
     shared.Building.eval_ctors(final, wasm_binary_target, binaryen_bin, debug_info=debug_info)
   # after generating the wasm, do some final operations
   if shared.Settings.SIDE_MODULE:
-    wso = shared.WebAssembly.make_shared_library(final, wasm_binary_target)
+    wso = shared.WebAssembly.make_shared_library(final, wasm_binary_target, shared.Settings.RUNTIME_LINKED_LIBS)
     # replace the wasm binary output with the dynamic library. TODO: use a specific suffix for such files?
     shutil.move(wso, wasm_binary_target)
     if not shared.Settings.WASM_BACKEND and not DEBUG:

src/library_browser.js

@@ -244,7 +244,7 @@ var LibraryBrowser = {
         // promises to run in series.
         this['asyncWasmLoadPromise'] = this['asyncWasmLoadPromise'].then(
           function() {
-            return loadWebAssemblyModule(byteArray, true);
+            return loadWebAssemblyModule(byteArray, {loadAsync: true, nodelete: true});
           }).then(
             function(module) {
               Module['preloadedWasm'][name] = module;

src/preamble_sharedlib.js

@@ -4,5 +4,19 @@
 // Runtime essentials
 //========================================
 
+{{{
+(function() {
+  // add in RUNTIME_LINKED_LIBS, if provided
+  //
+  // for side module we only set Module.dynamicLibraries - and loading them
+  // will be handled by dynamic linker runtime in the main module.
+  if (RUNTIME_LINKED_LIBS.length > 0) {
+    return "if (!Module['dynamicLibraries']) Module['dynamicLibraries'] = [];\n" +
+           "Module['dynamicLibraries'] = " + JSON.stringify(RUNTIME_LINKED_LIBS) + ".concat(Module['dynamicLibraries']);\n";
+  }
+  return '';
+})()
+}}}
+
 // === Body ===
 

src/settings.js

@@ -683,9 +683,8 @@ var MAIN_MODULE = 0;
 // Corresponds to MAIN_MODULE (also supports modes 1 and 2)
 var SIDE_MODULE = 0;
 
-// If this is a main module (MAIN_MODULE == 1), then
-// we will link these at runtime. They must have been built with
-// SIDE_MODULE == 1.
+// If this is a shared object (MAIN_MODULE == 1 || SIDE_MODULE == 1), then we
+// will link these at runtime. They must have been built with SIDE_MODULE == 1.
 var RUNTIME_LINKED_LIBS = [];
 
 // If set to 1, this is a worker library, a special kind of library that is run