Embedding JavaScript source code in a QuickJS interpreter module vs. ahead-of-time compilation #1298
Replies: 5 comments 24 replies
|
If I understand correctly you want to ship a .wasm module that contains both the QuickJS interpreter and the Javascript source and runs it? How does js2wasm fit into this picture? When compiling .js with js2wasm the resulting .wasm module neither will need to ship the .js nor QuickJS. |
|
The concept doesn't fit into this picture. In a sense. Unless once you get to all of the features you are going to implement the resulting WASM binary increases in final file size. It's just something that works. And we get the whole QuickJS NG engine. API's. After Then there's Micro QuickJS, that I havn't yet been able to compile to WASM, yet. The picture, as I see it, is the entire landscape of JavaScript compiled to WASM - by any means. That's the sector of the industry. The umbrella. I compare, test, vet, see if I can break them all, or find existing breakage the maintainer might not have observed - 'cause they can't thing of all uses cases and thus can't test their gear in cases they havn't thought of. |
|
I get that you are comparing "gears" and usually this work with QuickJS NG. I also see that if AOT compiled .js uses JS builtins or requires interpretation of runtime generated code module sizes will grow for AOT as well. Is there anything beyond that I should read from your post? |
|
I'm saying it's 6 of one, half dozen of the other. The more you add, the closer you get to writing an entire engine. "AOT" compilation works. So does embedding an engine. These are just different tools in the programming toolbox. It would be interesting to see everything in V8 - ECMAScript and WebAssembly-wise - ahead of time compiled, on demand. I think each approach winds up being basically the same. Either way you have to have a symbol to represent an instruction. |
|
Right, with the distinction that with AOT compilation modules don't need to ship code for features that are not used - and also no interpreter or compiler code or runtime. For example, if:
Also no parsing, interpretation, compilation is needed on the JS side, so no interpreter or speculative JIT optimization compiler tiers need to be shipped alongside. With the AOT approach compilation has already taken place once on the side of the producer, which decreases compilation complexity at runtime and results in real gains like shipping less code and less work at runtime. |
|
The issue title is confusing to me and reads unrelated to this repo. Based on our discussion I would suggest to rename it from: Embed JavaScript in qjs.c before compilation to WASM with WASI-SDK To: Embedding JavaScript in a QuickJS interpreter module vs. ahead-of-time compilation |
|
It's not an Issue, it's a Discussion. I changed the title anyway. |
|
I got |
|
I just invested a couple days trying to get Bellard and Gordon's Micro QuickJS compiled to WASM with wasi-sdk. My goal, of course, was to create a Native Messaging host embedded into There's no built in
# rm -f *.o mqjs mqjs_stdlib mqjs_stdlib.h mquickjs_atom.h
# 1. Re-compile the host compiler table utility tool natively
gcc -m32 -Wall -O2 -DCONFIG_M32 -o mqjs_stdlib mqjs_stdlib.c mquickjs_build.c -I.
# 2. Write out your layout headers (Will execute flawlessly now)
./mqjs_stdlib -a > mquickjs_atom.h
./mqjs_stdlib > mqjs_stdlib.h
# 3. Cross-compile to WebAssembly using WASI-SDK
make CC="$WASI_SDK_PATH/bin/clang --target=wasm32-wasip1 --sysroot=$WASI_SDK_PATH/share/wasi-sysroot" \
AR="$WASI_SDK_PATH/bin/ar" \
STRIP="$WASI_SDK_PATH/bin/llvm-strip" \
CFLAGS="-O2 -D_WASI_EMULATED_SIGNAL -I." \
LDFLAGS="-Wl,--allow-undefined -lwasi-emulated-signal" \
mqjs
# ...
// 1. In MicroQuickJS, parameters are plain JSValue types.
// There is NO JSValueConst or explicit Free operations.
// 1. Ensure the signature exactly matches what 'JSCFunction' demands under the hood.
// JSValue in this engine is an unsigned int typedef.
JSValue js_print(JSContext *ctx, JSValue *this_val, int argc, JSValue *argv) {
if (argc > 0) {
JSCStringBuf buf;
// Pass the explicit 3-argument signature requested by your mquickjs.h header
const char *str = JS_ToCString(ctx, argv[0], &buf);
if (str) {
printf("%s\n", str);
fflush(stdout); // Keep WASI from buffering the stream output
}
}
return JS_UNDEFINED;
}
int main(int argc, char **argv) {
size_t mem_size = 65536;
uint8_t *mem_buf = malloc(mem_size);
// RESTORE the original js_stdlib link so the engine populates
// the global definitions table using your mqjs_stdlib.h edits!
JSContext *ctx = JS_NewContext(mem_buf, mem_size, &js_stdlib);
if (!ctx) {
printf("Failed to create JS context!\n");
return -1;
}
const char *script = "print('Hello World');";
// Pass 0 for flags to execute standard global evaluation blocks
JSValue result = JS_Eval(ctx, script, strlen(script), "<input>", 0);
// Check if the script encountered an internal processing exception
if (JS_IsException(result)) {
printf("JavaScript Exception Triggered!\n");
}
fflush(stdout);
free(mem_buf);
return 0;
}That spits out a WASM binary around 481.4 KiB. I've gotten it down to 239.5 KiB (that throws when run by
Fair enough. I'm personally highly skeptical of popularity. I'm thinking I can get close to what is being done here coming from the opposite direction with something like Micro QuickJS. I took quite a bit of refinement to cobble together a few Native Messaging hosts; with the |
Back in QuickJS NG workd I'm trying to figure out the most efficient way to write that JavaScript as a const char* str =
"import * as std from 'qjs:std';\n"
//"import * as os from 'qjs:os';\n"
"globalThis.std = std;\n"
//"globalThis.os = os;\n"
"function _(){let t=new Uint32Array(1);std.in.read(t.buffer,0,4);let "
"s=new Uint8Array(t[0]),w=std.in.read(s.buffer,0,s.length);return "
"s}function g(t){if(t.length<=1048576){let n=new "
"Uint8Array(4);n[0]=t.length>>0&255,n[1]=t.length>>8&255,n[2]=t.length>>"
"16&255,n[3]=t.length>>24&255;let f=new "
"Uint8Array(4+t.length);f.set(n,0),f.set(t,4),std.out.write(f.buffer,0,f."
"length),std.out.flush();return}let i=0;while(i<t.length){let "
"n,f=i+1048576-8;if(f>=t.length)n=t.length;else "
"if(n=t.indexOf(44,f),n===-1)n=t.length;let "
"l=t.subarray(i,n),u=l[0],a=l[l.length-1],c=null,o=null;if(u===91&&a!=="
"93)o=93;else if(u===44){if(c=91,a!==93)o=93}let "
"d=l.length,y=4,C=c!==null,E=o!==null,A=0;if(u===44)A=1,d-=1;let "
"O=4+(C?1:0)+d+(E?1:0),e=new "
"Uint8Array(O),h=O-4;e[0]=h>>0&255,e[1]=h>>8&255,e[2]=h>>16&255,e[3]=h>>"
"24&255;let r=4;if(C)e[r]=c,r++;else "
"if(u===44)e[r]=91,r++;if(e.set(l.subarray(A),r),r+=d,E)e[r]=o;std.out."
"write(e.buffer,0,e.length),std.out.flush(),std.gc(),i=n}}function "
"M(){while(!0){let t=_();g(t)}}try{M()}catch(t){std.exit(0)}";
eval_buf(ctx, str, strlen(str), "<input>", JS_EVAL_TYPE_MODULE);Software nation-state: Ultra-nationalists and neo-liberalsI'm thinking of doing a little prose write-up: "Software nation-state: Ultra-nationalists and neo-liberals". What I'm thinking of doing is comparing that issue where we patiently hammered out technical details cf. to BCA folks banning me for daring to express shock at a 13.4 MiB WASM binary spit out by ComponentizeJS. |
|
@guest271314 I would highly advise against it and would ask you to respect I do not want my work to be drawn into heated discussions with others. I see your good intentions to verify claims and give honest feedback. If you want my advice, I think the point where people become offended is if in an emotional reaction examples are overgeneralized and incorrect conclusions and intentions are projected. |
|
Well, I already published the gist. |
|
It's not a heated discussion, it's an observation. Comparative analysis of an independent browser/ECMAScript hacker. |
|
Or, are you really saying delete it? Get you and your work, and the work we did in that issue out of it? Is that what you are saying? Cuts too deep? |
|
Get it off of your WebAssembly voard on Discord? |
|
The best I can do for you is delete it off of your WebAssembly board. That's done. I'm not a delete my opinion type of guy. |
Oh, it can be even more discriminatory that skin color or politics in the software world. Try just mentioning "LLM" anywhere in Zig world and see how fast your post gets scrubbed https://ziggit.dev/t/flagged-post-removed-by-staff/16001. |
|
Dissent is not allowed... |
|
You're free to express your opinion, but I don't want this project to be part of an article pitting open source projects against each other titled "Software nation-state: Ultra-nationalists and neo-liberals". This is hurtful to the goals of this project and I have to draw a line here and close this issue since it was offtopic anyways and it's getting out of hand. |
|
Yeah, well, ping me for further testing if you are so inclined. There's no pitting. There is only observation of the landscape. |
Uh oh!
There was an error while loading. Please reload this page.
See quickjs-ng/quickjs#1527.
#!/usr/bin/env -S wasmtime /home/user/native-messaging-webassembly/qjs-wasi-opt.wasmAll reactions