Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose method to allow synchronous initialization #2924

Merged
merged 2 commits into from Jun 8, 2022

Conversation

d3lm
Copy link
Contributor

@d3lm d3lm commented Jun 2, 2022

This PR exposes helpers that will allow to synchronously initialize the WASM module. The helpers are only exported when compiling for web. But we could also export them for no-modules.

What I have done is to extract some of the functionality from init() into helper functions which are now exported.

This fixes #2698.

Todos

  • Add documentation
  • Do we only want to expose the helpers or maybe export a single function initSync()?

Background

At StackBlitz we are building WebContainer, a Node.js runtime for the browser, and we could benefit from this addition a lot because it would allow us to lazy load WASM modules on demand. Some features are split into separate WASM modules to improve load performance and those features are often not frequently used. By exposing these helpers we could synchronously instantiate modules in a worker like this then:

import { getImports, initMemory, finalizeInit } from 'my_wasm_module.js';

const imports = getImports();

initMemory(imports[, maybe_memory]);

const bytes = "<sync_fetch_bytes>";

const module = new WebAssembly.Module(bytes);
const instance = new WebAssembly.Instance(module, imports);

const wasm = finalizeInit(instance, module);

ℹ️ The above only really works in a Web Worker but that's ok IMO as the main thread disallows sync APIs for good reasons.

What do you think @alexcrichton? Is that what you were thinking of in #2698?

@alexcrichton
Copy link
Contributor

@alexcrichton alexcrichton commented Jun 6, 2022

This seems reasonable enough to me, thanks for the PR! Can you add some documentation for this as well?

@d3lm
Copy link
Contributor Author

@d3lm d3lm commented Jun 6, 2022

Yep, will add some documentation.

@d3lm
Copy link
Contributor Author

@d3lm d3lm commented Jun 6, 2022

@alexcrichton Do we want to expose an initSync that wraps the internals? Or do we only want to expose those internals?

@alexcrichton
Copy link
Contributor

@alexcrichton alexcrichton commented Jun 6, 2022

Honestly this is so far outside my expertise I don't really have an opinion. It's not "obviously wrong" but I know very little about all the myriad embeddings that JS can be run in, so I have no idea how to gauge which is better to provide. I'm basically leaving such a judgement call to you.

@d3lm d3lm force-pushed the feat/expose-init-helpers branch from ec3f22f to f72b9c0 Compare Jun 8, 2022
@@ -469,7 +469,8 @@ impl<'a> Context<'a> {
OutputMode::Web => {
self.imports_post.push_str("let wasm;\n");
init = self.gen_init(needs_manual_start, Some(&mut imports))?;
footer.push_str("export default init;\n");
footer.push_str("export { initSync }\n");
footer.push_str("export default init;");
Copy link
Contributor Author

@d3lm d3lm Jun 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ I have noticed there was two newlines at the end of the generated JS so I removed one newline.

@d3lm d3lm force-pushed the feat/expose-init-helpers branch from f72b9c0 to 5c9a270 Compare Jun 8, 2022
@d3lm
Copy link
Contributor Author

@d3lm d3lm commented Jun 8, 2022

Alright, I have added an example and a guide as well for this.

I have also decided not expose the internal helpers and instead expose a new initSync to make it easier to use. Exposing the helpers itself gives more flexibility but I think it's not really needed at this point. In order to lazy load a WASM module you'd have to either fetch the bytes snychronously or take the router via a postMessage and use a SAB to block and wait for the bytes to be written into the shared memory. Sharing the same WebAssembly.Module across multiple workers wouldn't work because it can't be serialized into shared memory. For now, each worker has to compile the module again. The only thing I have added is the additional (optional) maybe_memory parameter in case it's compiled for shared memory.

imports_init.push_str("imports.");
imports_init.push_str(module_name);
imports_init.push_str(" = {};\n");
Copy link
Contributor Author

@d3lm d3lm Jun 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ I have noticed that if a module has no imports and we don't initialize the imports with the module_name (wbg) that it will break when it tries to initialize the memory, because it gets assigned to imports.wbg.memory.

{post_instantiate}
{start}
return wasm;
}}

function initSync(bytes{init_memory_arg}) {{
Copy link
Contributor Author

@d3lm d3lm Jun 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ This function gets exported to synchronously initialize the module.

@d3lm
Copy link
Contributor Author

@d3lm d3lm commented Jun 8, 2022

@alexcrichton This PR is ready for review.

@alexcrichton alexcrichton merged commit 3822e67 into rustwasm:main Jun 8, 2022
21 checks passed
@alexcrichton
Copy link
Contributor

@alexcrichton alexcrichton commented Jun 8, 2022

Thanks!

@d3lm
Copy link
Contributor Author

@d3lm d3lm commented Jun 8, 2022

@alexcrichton Nice! When do you think an we release this as a new version? So that the wasm-bindgen book is also updated with the example etc.

@d3lm d3lm deleted the feat/expose-init-helpers branch Jun 8, 2022
@d3lm d3lm changed the title Expose helpers to allow synchronous initialization Expose method to allow synchronous initialization Jun 8, 2022
@alexcrichton
Copy link
Contributor

@alexcrichton alexcrichton commented Jun 9, 2022

I'll look into doing a release next week

@alexcrichton
Copy link
Contributor

@alexcrichton alexcrichton commented Jun 14, 2022

Ok I'm doing a publish at #2935

expenses pushed a commit to expenses/wasm-bindgen that referenced this issue Jun 14, 2022
* Expose helpers to allow synchronous initialization

* fixup: fix tests
@d3lm
Copy link
Contributor Author

@d3lm d3lm commented Jun 14, 2022

Thanks a lot!

@d3lm d3lm mentioned this pull request Jun 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants