Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WASM=0 + SINGLE_FILE=1 not a stand-alone file #9380

Closed
Beuc opened this issue Sep 4, 2019 · 14 comments · Fixed by #9407

Comments

@Beuc
Copy link
Contributor

commented Sep 4, 2019

Hi,

I'm trying to make zee.js https://github.com/kripken/zee.js work with the "upstream" branch.

This project is compiled so it can be loaded synchronously on start-up, possibly to load other compressed files such as index.wasm.gz with a custom loader.

EMCC=emcc -O2 -s WASM=0 -s SINGLE_FILE=1 -s EXPORTED_FUNCTIONS="['_gzopen', '_gzread', '_gzwrite', '_gzclose']" --pre-js pre.js --post-js post.js

zee.js now raises:

uncaught exception: could not load memory initializer http://localhost:8000/libz.raw.js.mem
Error: ENOENT: no such file or directory, open '.../zee.js/libz.raw.js.mem'

Is SINGLE_FILE support incomplete in "upstream"?

@sbc100

This comment has been minimized.

Copy link
Collaborator

commented Sep 4, 2019

No its more likely due to the fact that upstream + WASM=0 (i.e. wasm2js) is fairly new still and probably hasn't been tested with SINGLE_FILE.

What is you reason for needed WASM=0 here?

@sbc100

This comment has been minimized.

Copy link
Collaborator

commented Sep 4, 2019

Oh I see you are using WASM=0 to allow for the synchronous startup. Is the wasm file to large for WASM_ASYNC_COMPILATION=0?

@Beuc

This comment has been minimized.

Copy link
Contributor Author

commented Sep 4, 2019

Yes, I get 188kB hence far above Chromium's 4kB limit :/

@sbc100

This comment has been minimized.

Copy link
Collaborator

commented Sep 4, 2019

I think we can fix this bug.. however I don't think wasm2js is intended as way to work around wasm synchronous compilation limits (I could be wrong). Ideally you probably want to re-factor your code to allow for asynchronous compilation. I understand this can sometimes be tricky.. but it is probably work that is worth doing.

@Beuc

This comment has been minimized.

Copy link
Contributor Author

commented Sep 4, 2019

I'm not even sure such a re-factoring is possible TBH, short of patching Emscripten / Emscripten's boilerplate - the context is loading a compressed main.wasm.gz (on a CDN where HTTP compression can't be configured). I currently have to work within Module.instantiateWasm, it's tricky enough as it is.
(now if Emscripten provides fallback JS compression natively, I'm all ears ;))

@sbc100

This comment has been minimized.

Copy link
Collaborator

commented Sep 4, 2019

Can't you do it like this?:

  1. Async fetch the decompressor
  2. Async fetch main.wasm.gz
  3. Run decompress on main.wasm.gz to create blobURL (or just arraybuffer)
  4. Async instantiate main.wasm from the blob url / arraybuffer?

Where does the requirement for synchronous instantiate come from? Is it hard to override the emscripten URL with the blob URL from (3)?

@Beuc

This comment has been minimized.

Copy link
Contributor Author

commented Sep 4, 2019

Well I already have this much rewrite:
https://github.com/renpy/renpyweb/blob/2094ec0866401bbe6bdbf0e0a52b6216692354cc/renpy-pre.js#L26
(note that I also handle properly-configured web servers with compression and proper content-type)
I find the init process particularly tricky.

Maybe I overlooked a trivial fix, but if I can save some sanity by being able to generate a synchronous zee.js, that would be nice.

That's only one example, I'm pretty sure other users of zee.js would appreciate running it like they currently do.

@sbc100

This comment has been minimized.

Copy link
Collaborator

commented Sep 4, 2019

I'm not aware of zee.js or its users. I'll try to find time to take look at this use case. Hopefully we can find a way to avoid needing to use wasm2js on platforms that have wasm, but obviously people are free to do as they please.

@Beuc

This comment has been minimized.

Copy link
Contributor Author

commented Sep 5, 2019

You can ask @kripken as zee.js is his project :)

@kripken

This comment has been minimized.

Copy link
Member

commented Sep 6, 2019

Ah, looks like the relevant test was disabled, so this was missed, sorry...

Fix in #9407!

@buu700

This comment has been minimized.

Copy link
Contributor

commented Sep 8, 2019

Awesome, thanks @kripken! I was just about to submit a report for this issue as well.

@Beuc

This comment has been minimized.

Copy link
Contributor Author

commented Sep 10, 2019

On a related note, it seems that:
-s WASM=1 -s SINGLE_FILE=1
works in Chrome, even with code > 4kB.

Was I just repeatedly lucky, or is this really synchronous?

@kripken

This comment has been minimized.

Copy link
Member

commented Sep 11, 2019

No, see other issue - SINGLE_FILE doesn't mean synchronous. The extra WASM_ASYNC_COMPILATION=0 flag must be used for that (and it won't work in chrome for large files, sadly, yeah...).

kripken added a commit that referenced this issue Sep 11, 2019
Fix SINGLE_FILE with wasm2js (#9407)
The relevant test was disabled internally, which is why I missed it when I triaged all the disabled tests (I was looking for decorators etc.).

Fix is pretty simple, the logic for when to use a mem init file was not aware of SINGLE_FILE. (There may be an even simpler way to rewrite that code, but let's wait until after fastcomp is removed for any major refactoring.)

Fixes #9380
@Beuc

This comment has been minimized.

Copy link
Contributor Author

commented Sep 11, 2019

Thanks for clarifying.

(Maybe there's some ordering constraint that made my custom (async) wasm.gz loader always run after zee's.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.