Skip to content

Commit

Permalink
"Modularize" the WebAssembly build
Browse files Browse the repository at this point in the history 
By default, Emscripten creates an object called "Module" which holds
the Wasm heap, exported C-compiled functions, and helper routines.
But if all emscripten codebases did this, you wouldn't be able to load
two codebases in the same page because they would overwrite each other.

This takes advantage of the "MODULARIZE=1" option in Emscripten, to
name a function which gives a Promise back to make a module.  This is
then poked away inside the `reb` object, as `reb.m`.

Also, this goes ahead and wraps up some miscellaneous global tables and
puts them inside the `reb` object as well.
  • Loading branch information
@hostilefork
hostilefork committed Jul 6, 2020
1 parent 3183acf commit 0287faa
Show file tree
Hide file tree
Showing 4 changed files with 192 additions and 150 deletions.
11 changes: 11 additions & 0 deletions configs/emscripten.r
View file
Expand Up @@ -175,6 +175,17 @@ ldflags: compose [
{--pre-js prep/include/node-preload.js}
])

; The default build will create an emscripten module named "Module", which
; holds various emscripten state (including the memory heap) and service
; routines. If everyone built their projects like this, you would not be
; able to load more than one at a time due to the name collision. So
; we use the "Modularize" option to get a callback with a parameter that
; is the module object when it is ready. This also simplifies the loading
; process of registering notifications for being loaded and ready.
; https://emscripten.org/docs/getting_started/FAQ.html#can-i-use-multiple-emscripten-compiled-programs-on-one-web-page
;
{-s MODULARIZE=1 -s 'EXPORT_NAME="r3_module_promiser"'}

(if debug-javascript-extension [
{-s ASSERTIONS=1}
] else [
Expand Down
211 changes: 111 additions & 100 deletions extensions/javascript/load-r3.js
View file
Expand Up @@ -81,15 +81,20 @@
//
// https://medium.com/@tkssharma/javascript-module-pattern-b4b5012ada9f
//
// Two global objects are exported. One is `Module`, which is how Emscripten
// expects to get its configuration parameters (as properties of that global
// object), so the startup function must initialize it with all the necessary
// properties and callbacks.
//
// The other object is `reb` which is the container for the API. The reason
// all the APIs are in an object (e.g. `reb.Elide()` instead of `rebElide()`)
// is because Node.js doesn't allow global functions, so the only way to get
// an API would be through something like `let reb = require('rebol')`.
// Only one object is exported, the `reb` object. It is the container for
// the API. The reason all the APIs are in an object (e.g. `reb.Elide()`
// instead of `rebElide()` as needed in C) is because Node.js doesn't allow
// global functions. So the only way to get an API would be through something
// like `let reb = require('rebol')`.
//
// The `reb.m` member holds the Emscripten module object. This contains the
// WebAssembly heap and other service routines, and all of the "unwrapped"
// raw API exports--which take integer parameters as heap addresses only.
// (The reb.XXX routines are wrapped to speak in terms of types like strings
// or arrays.) By default Emscripten would make this a global variable
// called `Module`, but using the `MODULARIZE=1` option it will give us a
// factory function that passes the module as a parameter so we can place it
// wherever we like.
//
// It may look like reb.Startup() takes two parameters. But if you read all
// the way to the bottom of the file, you'll see `console` is passed in with
Expand All @@ -102,10 +107,22 @@
// off. If you have that problem, comment out the function temporarily.
//

var reb = {} // This aggregator is where we put all the Rebol APIs
// We can make an aggregator here for all the Rebol APIs, which is global for
// the browser... e.g. this is actually `window.reb`. But that doesn't do
// a Web Worker much good in threaded builds, since they have no `window`.
// Even though they share the same heap, they make their own copies of all
// the API wrappers in their own `reb` as `self.reb`.
//
var reb = {}

var Module // Emscripten expects this to be global and set up with options

// !!! This `reb.Startup` function gets overwritten by the wrapped version of
// the internal API's `reb.Startup`, which is then invoked during the load
// process. It works...but, for clarity the internal version might should be
// changed to something like `Startup_internal`, and the C version can then
// simply `#define rebStartup RL_Startup_internal` since it has no parallel
// to this loading step.
//
reb.Startup = function(console_in, config_in) { // only ONE arg, see above!


Expand Down Expand Up @@ -136,6 +153,62 @@ else
config = default_config


// The factory function for MODULARIZE=1 in Emscripten takes an object as a
// parameter for the defaults. Since we don't need the defaults once the
// actual module is loaded, we reuse the same variable for the defaults as we
// do the ultimately loaded module. For documentation on options:
//
// https://emscripten.org/docs/api_reference/module.html#affecting-execution
//
reb.m = {
//
// For errors like:
//
// "table import 1 has a larger maximum size 37c than the module's
// declared maximum 890"
//
// The total memory must be bumped up. These large sizes occur in debug
// builds with lots of assertions and symbol tables. Note that the size
// may appear smaller than the maximum in the error message, as previous
// tables (e.g. table import 0 in the case above) can consume memory.
//
// !!! Messing with this setting never seemed to help. See the emcc
// parameter ALLOW_MEMORY_GROWTH for another possibility.
//
/* TOTAL_MEMORY: 16 * 1024 * 1024, */

locateFile: function(s) {
//
// function for finding %libr3.wasm (Note: memoryInitializerPrefixURL
// for bytecode was deprecated)
//
// https://stackoverflow.com/q/46332699
//
config.info("reb.m.locateFile() asking for .wasm address of " + s)

let stem = s.substr(0, s.indexOf('.'))
let suffix = s.substr(s.indexOf('.'))

// Although we rename the files to add the Git Commit Hash before
// uploading them to S3, it seems that for some reason the .js hard
// codes the name the file was built under in this request. :-/
// So even if the request was for `libr3-xxxxx.js` it will be asked
// in this routine as "Where is `libr3.wasm`
//
// For the moment, sanity check to libr3. But it should be `rebol`,
// or any name you choose to build with.
//
if (stem != "libr3")
throw Error("Unknown libRebol stem: " + stem)

if (suffix == ".worker.js")
return URL.createObjectURL(workerJsBlob)

return libRebolComponentURL(suffix)
}
}


//=//// PICK BUILD BASED ON BROWSER CAPABILITIES //////////////////////////=//
//
// The JavaScript extension can be built two different ways for the browser.
Expand Down Expand Up @@ -212,17 +285,17 @@ let reb_args = "["

for (let i = 0; i < js_args.length; i++) {
let a = decodeURIComponent(js_args[i]).split("=") // makes array
if (a.length == 1) { // simple switch with no arguments, e.g. ?debug
if (a[0] == 'debug') {
is_debug = true
} else if (a[0] == 'local') {
base_dir = "./"
} else if (a[0] == 'remote') {
base_dir = "https://metaeducation.s3.amazonaws.com/travis-builds/"
} else if (a[0] == 'tracing_on') {
config.tracing_on = true
} else
reb_args += a[0] + ": true " // look like being set to true
if (a.length == 1) { // simple switch with no arguments, e.g. ?debug
if (a[0] == 'debug') {
is_debug = true
} else if (a[0] == 'local') {
base_dir = "./"
} else if (a[0] == 'remote') {
base_dir = "https://metaeducation.s3.amazonaws.com/travis-builds/"
} else if (a[0] == 'tracing_on') {
config.tracing_on = true
} else
reb_args += a[0] + ": true " // look like being set to true
}
else if (a.length = 2) { // combination key/val, e.g. "git_commit=<hash>"
if (a[0] == 'git_commit') {
Expand Down Expand Up @@ -407,61 +480,6 @@ let prefetch_worker_js_promiser = () => new Promise(
)


Module = { // Note that this is assigning a global
//
// For errors like:
//
// "table import 1 has a larger maximum size 37c than the module's
// declared maximum 890"
//
// The total memory must be bumped up. These large sizes occur in debug
// builds with lots of assertions and symbol tables. Note that the size
// may appear smaller than the maximum in the error message, as previous
// tables (e.g. table import 0 in the case above) can consume memory.
//
// !!! Messing with this setting never seemed to help. See the emcc
// parameter ALLOW_MEMORY_GROWTH for another possibility.
//
/* TOTAL_MEMORY: 16 * 1024 * 1024, */

locateFile: function(s) {
//
// function for finding %libr3.wasm (Note: memoryInitializerPrefixURL
// for bytecode was deprecated)
//
// https://stackoverflow.com/q/46332699
//
config.info("Module.locateFile() asking for .wasm address of " + s)

let stem = s.substr(0, s.indexOf('.'))
let suffix = s.substr(s.indexOf('.'))

// Although we rename the files to add the Git Commit Hash before
// uploading them to S3, it seems that for some reason the .js hard
// codes the name the file was built under in this request. :-/
// So even if the request was for `libr3-xxxxx.js` it will be asked
// in this routine as "Where is `libr3.wasm`
//
// For the moment, sanity check to libr3. But it should be `rebol`,
// or any name you choose to build with.
//
if (stem != "libr3")
throw Error("Unknown libRebol stem: " + stem)

if (suffix == ".worker.js")
return URL.createObjectURL(workerJsBlob)

return libRebolComponentURL(suffix)
},

// This is a callback that happens sometime after you load the emscripten
// library (%libr3.js in this case). It's turned into a promise instead
// of a callback. Sanity check it's not used prior by making it a string.
//
onRuntimeInitialized: "<mutated from a callback into a Promise>"
}


//=// CONVERTING CALLBACKS TO PROMISES /////////////////////////////////////=//
//
// https://stackoverflow.com/a/22519785
Expand Down Expand Up @@ -490,16 +508,6 @@ if (document.readyState == "loading") {
dom_content_loaded_promise = Promise.resolve()
}

let runtime_init_promise = new Promise(function(resolve, reject) {
//
// The load of %libr3.js will at some point will trigger a call to
// onRuntimeInitialized(). We set it up so that when it does, it will
// resolve this promise (used to trigger a .then() step).
//
Module.onRuntimeInitialized = resolve
})


let load_rebol_scripts = function(defer) {
let scripts = document.querySelectorAll("script[type='text/rebol']")
let promise = Promise.resolve(null)
Expand Down Expand Up @@ -552,23 +560,25 @@ let load_rebol_scripts = function(defer) {
//=//// MAIN PROMISE CHAIN ////////////////////////////////////////////////=//

return assign_git_commit_promiser(os_id) // sets git_commit
.then(function () {
.then(() => {
config.log("prefetching worker...")
return prefetch_worker_js_promiser() // no-op in the non-pthread build
return prefetch_worker_js_promiser() // no-op in the non-pthread build
})
.then(function() {
.then(() => {

load_js_promiser(libRebolComponentURL(".js")) // needs git_commit
return load_js_promiser(libRebolComponentURL(".js")) // needs git_commit

}).then(function() {
}).then(() => { // we now know r3_module_promiser is available

config.info('Loading/Running ' + libRebolComponentURL(".js") + '...')
if (use_asyncify)
config.warn("The Asyncify build is biggers/slower, be patient...")
config.warn("The Asyncify build is bigger/slower, be patient...")

return r3_module_promiser(reb.m) // at first, `reb.m` is defaults...

return runtime_init_promise
}).then(module => { // "Modularized" emscripten passes us the module

}).then(function() { // emscripten's onRuntimeInitialized() has no args
reb.m = module // overwrite the defaults with the instantiated module

config.info('Executing Rebol boot code...')

Expand Down Expand Up @@ -622,12 +632,13 @@ return assign_git_commit_promiser(os_id) // sets git_commit
"for-each collation builtin-extensions",
"[load-extension collation]"
)
}).then(()=>load_rebol_scripts(false))
}).then(() => load_rebol_scripts(false))
.then(dom_content_loaded_promise)
.then(()=>load_rebol_scripts(true))
.then(()=>{
let code = me.innerText.trim()
if (code) eval(code)
.then(() => load_rebol_scripts(true))
.then(() => {
let code = me.innerText.trim()
if (code)
eval(code)
})

//=//// END ANONYMOUS CLOSURE USED AS MODULE //////////////////////////////=//
Expand Down
12 changes: 8 additions & 4 deletions extensions/javascript/mod-javascript.c
View file
Expand Up @@ -414,7 +414,7 @@ EXTERN_C intptr_t RL_rebPromise(REBFLGS flags, void *p, va_list *vaptr)

#ifdef USE_ASYNCIFY
EM_ASM(
{ setTimeout(function() { _RL_rebIdle_internal(); }, 0); }
{ setTimeout(function() { reb.m._RL_rebIdle_internal(); }, 0); }
); // note `_RL` (leading underscore means no cwrap)
#else
pthread_mutex_lock(&PG_Promise_Mutex);
Expand Down Expand Up @@ -880,7 +880,7 @@ REB_R JavaScript_Dispatcher(REBFRM *f)
MAIN_THREAD_EM_ASM( // blocking call
{
reb.RunNative_internal($0, $1); // `;` is necessary here
_RL_rebTakeAwaitLock_internal();
reb.m._RL_rebTakeAwaitLock_internal();
},
native_id, // => $0
frame_id // => $1
Expand Down Expand Up @@ -1185,12 +1185,16 @@ REBNATIVE(js_eval_p)
MAIN_THREAD_EM_ASM(
{ eval(UTF8ToString($0)) },
utf8
);
else
); // !!! ...should be an else clause here...
// !!! However, there's an emscripten bug, so use two `if`s instead
// https://github.com/emscripten-core/emscripten/issues/11539
//
if (not REF(local))
MAIN_THREAD_EM_ASM(
{ (1,eval)(UTF8ToString($0)) },
utf8
);

return Init_Void(D_OUT);
}

Expand Down

0 comments on commit 0287faa

Please sign in to comment.