Use WebAssembly.instantiateStreaming instead of WebAssembly.compileStreaming

[linclark]

Unfortunately there are performance issues with WebAssembly.compileStreaming on iOS. On iOS devices, there is a limited amount of faster memory. Because of this, the engine doesn't know which kind of memory to compile for until instantiate is called.

If I understand correctly, for the JSC engine compileStreaming will just do a memcopy of the .wasm. It's only when instantiate is called that the Memory object is created. Then the engine knows whether it will be using the fast memory, or just a normal malloc'd Memory that needs bounds checks. At this point, it can generate the code appropriately.

Because of this, we plan to recommend that all bundlers use instantiateStreaming instead. Does this cause any issues for webpack?

[sokra]

yes there are multiple issues with that:

---

Pre-knowledge:

webpack tries to treat WASM like ESM.
We try to apply all rules/assumptions that apply to ESM to WASM too.
We assume that in future the WASM JS API may be superseeded by a integration of WASM into the ESM module graph.

This means the imports section in the WASM file is resolved like import statements in ESM and the export section is treated like export in ESM.

WASM modules also have a start section, which is executed when the WASM is instantiated.

In the WASM JS API imports are passed via importsObject to the instantiate method.

---

The ESM spec specifies multiple phases. One phase is the ModuleEvaluation. In this phase all modules are evaulated in the well defined order. This phase is synchronous. All modules evaluate in the same "tick".

When WASM are in the module graph this means:

1. the start section is executed in the same "tick"
2. all dependencies of the WASM are executed in the same "tick"
3. the ESM importing the WASM is executed in the same "tick"

This behavior is not possible when using the Promise-version of instantiate. A Promise always delays it's fulfillment into another "tick".

It's only possible when using a sync version of instantiate (WebAssembly.Instance).

Note: Technically there could be a WASM without start section and without dependencies. In this case this doesn't apply. But we can't assume this is always the case.

---

webpack wants to download and compile the WASM in parallel to downloading the JS code. Using instantiateStreaming won't allow this (when the WASM has dependencies), because instantiate requires passing a importsObject. Creating the importsObject requires evaluating all dependencies/imports of the WASM, so these dependencies need to be downloaded before starting to download the WASM.

When using compileStreaming + new WebAssembly.Instance parallel download and compile is possible, because compileStreaming doesn't require an importsObject. The importsObject can be created when WASM and JS download has finished.

---

I also want to quote the WebAssembly spec. It states that compilation happens in a background thread in compile and instantiation happens on the main thread.

It's not clearly stated but in my optinion JSC's behavior is not spec-comform.

---

Other notes:

WASM also lacks the ability to use live bindings of imported identifiers. Instead the importsObject is spec'ed to be copied. This could have weird problems with circular dependencies and WASM.

It would be great to support getters in the importsObject and the be able to get the exports before the start section is executed.

[TheLarkInn]

cc @jsbastien @flagxor @rossberg I wanted to surface this to you all in hopes maybe we can bring this up for the next WASM CG Meeting if discussions are involved. Primarily around the above concerns.

[jfbastien]

This is worth discussing at the next CG meeting. @TheLarkInn, would you mind adding to the agenda (once it's posted... we just had the meeting today so I haven't created the agenda yet)?

To answer a few points:

1. I think treating WebAssembly like ECMAScript modules is the right thing to do. The JSC implementation uses exactly the same C++ code under the hood, so our behavior should match up very well. Same as ESM, export are handled when the JS object is created (but before execution returns), import is handled at "link" time, and start is called during "evaluate" (after tables and memories have been initialized). That's what I think WebAssembly should standardize as ESM integration.
2. IIUC the main issue you're encountering is that you can't inject yourself between object creation and link, correct? ESM just do it by construction, so even circular imports work.
3. I assume that having the import object as an extra indirection layer is a no-go (especially if you link wasm-wasm with i64)?

Interesting points on the spec. It's not final and I don't think this wording is accurate. It cannot mandate parallelism (unless parallel == 1 is OK). It does not mandate that all compilation be performed (disallowing any future compilation), and cannot either because that would prevent having an interpreter or tiering (so Chakra and SpiderMonkey would then not conform either).

[kmiller68]

@sokra For what it's worth, the original design of Wasm actually differs from what is in the initial spec. See the pull request on the design: WebAssembly/design#1018. I'm not sure where this got lost in the transition to the formal spec but I'll file an issue.

[TheLarkInn]

@kmiller68, so are you saying both methods can be used in paralell?

[lukewagner]

To discourage devs from using the sync WebAssembly.Instance and Module ctors (which, do sync compilation on various engines), Chrome currently throws when these constructors are called with modules bigger than an internal limit (4kb, last I saw). So that, combined with the fact that it's the worst-case load-time behavior for JSC, means sync new Instance should not be used by bundlers.

The ESM spec specifies multiple phases. One phase is the ModuleEvaluation. In this phase all modules are evaulated in the well defined order. This phase is synchronous. All modules evaluate in the same "tick".

I wasn't aware of this spec requirement, but you seem to be quite right!

So given these two constraints, could WebPack:

1. rewrite the .wasm to remove the start section and add the start function as an export
2. call WebAssembly.instantiateStreaming() during the instantiation phase
3. call the previously-start function synchronously during the evaluation phase

To break most cycles involving functions, I would imagine instantiateStreaming being passed shim functions that call the real imported function via a mutable var that gets filled in later.

The main problems with this are:

1. direct .wasm->.wasm imports (and avoiding serializing instantiateStreaming requrests)
2. non-function (i.e., memory, table, global) wasm imports (for which the above shim-calling-a-var trick won't work)

While there are future JS API extensions we've discussed that could address some of these cases, I think it might be ok for a bundler to simply reject these cases (i.e., issue a build error) for now:

• the point of treating a .wasm as an ESM is to use it from JS; wasm-to-wasm linkage is achieved more efficiently via static linking (lld) or LTO; dynamic linking isn't well-supported atm
• as long as dynamic linking isn't well supported, .wasm modules either define their own memory/table or import them directly from a generated glue ESM; there isn't inter-.wasm Memory/Table sharing

However, if these cases are or become important, I think they could be made to work, with some effort; happy to talk more about that.

[sokra]

rewrite the .wasm to remove the start section and add the start function as an export

I also thought about that, but that only works great when WASM has no dependencies.

shim functions
direct .wasm->.wasm imports

Shim function would really defeat future performance improvements in JS engines. I think wasm-to-wasm calls will get optimized in engines soon. The benefit of not going through the JS stack is big (afaik at least for v8). (cc @mstarzinger)

non-function

This would break a lot of good stuff in WASM. No memory sharing, no tables...

---

In my opinion this is not a good way to go. I guess I'll stay with the sync Instance call for now. (Maybe using instantiateStreaming for WASM without start and only function imports from JS.) I hope browsers will allow sync Instance > 4kb. I mean they can still do off-thread compilation in compile and off-thread lazy optimization after instantiation... This works for JS why should it be a problem for WASM.

[lukewagner]

I also thought about that, but that only works great when WASM has no dependencies.

Are there any problems in addition to what is already being discussed?

Shim function would really defeat future performance improvements in JS engines. I think wasm-to-wasm calls will get optimized in engines soon. The benefit of not going through the JS stack is big (afaik at least for v8). (cc @mstarzinger)

When I mentioned shim functions, this was for JS↔wasm imports; and even then they weren't always necessary; just to break cycles. I gave some reasons above why I don't expect wasm↔wasm imports to be common at the bundler stage; have you seen any indications otherwise?

By the time wasm↔wasm imports have become common, we can have increased the expressiveness of JS API (in ways already proposed, e.g., allowing instantiate to accept Promises for imported values) or simply have native ESM-wasm integration.

non-function

This would break a lot of good stuff in WASM. No memory sharing, no tables...

Similarly, until dynamic linking starts being used, every module (or .wasm+.js glue pair) will have its own memory/table, so I don't think we need to worry about multiple .wasm's sharing the same memory/table in the initial support.

That being said, if you wanted to do extra work and support this, it would be possible with another simple .wasm rewrite: convert the module that defines the memory/table to instead import and then create the memory/table in the JS glue code so they can be imported.

---

I guess I'll stay with the sync Instance call for now. (Maybe using instantiateStreaming for WASM without start and only function imports from JS.) I hope browsers will allow sync Instance > 4kb.

Then in practice WebPack wasm integration won't work on Chrome and will have worst-case load-time performance on Safari; that seems pretty unattractive. I think it's important not to let the perfect be the enemy of the good here: we're talking about very simple configurations of .wasm modules for the first few years that could easily and optimally use today's instantiateStreaming.