Use case: Web support using Emscripten

[kripken]

(Background: This reddit post where @andrewrk suggested I file this issue.)

I think it would be useful for Zig to allow compiling the same code both natively and to the Web.

Zig already supports the Web to some extent with the wasi and freestanding wasm targets. The difference with Emscripten is in API support: Emscripten allows common native APIs to be compiled to the Web, such as SDL2, OpenGL, OpenAL, pthreads, and so forth. A very common use case for Emscripten is to port a game engine, and that's why Unity and many others use it. As game engines begin to be written in Zig, allowing the same there as for C++ would be nice I think. (Of course, this isn't limited to games: Photoshop, Figma, AutoCAD, etc. also use Emscripten.)

What this means concretely: this gist has an example of a C program that uses GLES3 and GLFW to render, and relies on Asyncify to handle the Web event loop. That exact same program can be compiled with gcc or clang natively, or with emcc to the Web - that's the big benefit here.

The gist also has an example of a Zig program that I managed to do the same with, with some hacks. With small (I hope) improvements to Zig I think the hacks could be avoided, namely to allow setting the Emscripten target triple, and to get the full Zig stdlib compiling with that triple (hopefully simple, as the differences between the Emscripten wasm triple and the WASI wasm triple are very small). Emscripten itself would do all the rest, when the user runs it on the object files emitted from Zig (see details in that gist; there may also be more interesting Zig build system integration opportunities, but I don't know enough about that).

Why is it useful to compile the exact same code both natively and to the Web? You don't have to, of course - you can have separate codebases for the two platforms. For example, you could add a new rendering backend to your game engine to use WebGL, you can rewrite all your event handling logic to be async like the Web requires, rethink how you handle files, etc., and all the other stuff that is different on the Web. Such rewrites may have benefits, sometimes. But it makes porting harder, it makes debugging harder, and you may end up handling the mismatches between native and the Web in a sub-optimal way. Reusing the large effort put into Emscripten here can be useful.

[kubkon]

I'm happy to look into this as I've got a running example of a cross between C++ that requires both threading and exceptions, and pure Zig code. Ideal for testing the emscripten target triple.

[KioriSun]

Hey @kripken thanks for taking the time to help us!
Tried the gist and didn't work, both c and zig examples.
They run, but the screen stays black.

Interesting non-essential observations:
The initial obj gen by zig is around 5 times larger than the clang one, yet the final wasm is only around 5 kb larger.
The js and html emitted are seemingly the same size.
When editing the zig version ZLS(I think) complains about the "//" at the beginning of the shader string.

I will try again later on with my own code, just in case.
I was already going to test gles2 anyways, for webgl 1 support, which I think is more widely available.

Let me know if there is anything you'd like me to test.

I have latest emcc and zig.

[kripken]

@KioriSun Oh, sorry, looks like I forgot to write that you need to manually add _run_zig() in the HTML, or do that in the dev console. That is, the zig code can't use the automatic startup code because that isn't integrated between zig and emcc. Does that fix things for you? If not, is there an error in the dev console?

[KioriSun]

_run_zig()

Thanks it works. I was wondering about the -s STANDALONE_WASM -s EXPORTED_FUNCTIONS=_run_zig --no-entry part of the code and why it was necessary, how it would be used.
I'm not an expert on zig internals, but hopefully that integration can be reached.

A hint for everyone else that might give this a shot:
Another thing that I did that is important is i didn't just run the files but served them from a file server.
(this prevents cors issues and all, that modern wasm requires)

I couldn't get the _run_zig() automated, tried adding it to all sorts of onload functions, didn't work. only running from the console so far. Now that it's at least running form the console, I wanna try other apis. But I assume they'll run fine.

About automating the run function, for the time being, couldn't a properly placed EM_JS call solve that?
I'm not sure when/where the run function exists, as stated I tried calling it in other contexts and it either didn't exist or was undefined.

[kripken]

hopefully that integration can be reached.

Yes, exactly - that's one of the things that I think could be worked on, as part of this topic.

About automating the run function, for the time being, couldn't a properly placed EM_JS call solve that?

Hmm, no, because EM_JS happens after startup.

But a generic solution to this could be to add this to the HTML:

Module.onRuntimeInitialized = function() {
  _run_zig();
};

That uses one of the emscripten JS startup hooks. But this is kind of a hack that we should fix with proper integration in the Zig and emcc startup code, as mentioned before.

[KioriSun]

Module.onRuntimeInitialized = function() {
  _run_zig();
};

This is exactly what i was looking for in terms of lifetimes. But it didn't work for me. Maybe I'm missing something. I tried adding it into a script tag, body onload, and adding it directly into the zig.js that emcc builds. Nothing seems to work only running from the console.

If EM_JS happens after "startup", it also happens before or after "onRuntimeInitialized"? Because this is what I thought might work as a in code hack because if the objs are already there, maybe the function can be added, maybe it can't.

[floooh]

I can 'contribute' a little real-world example here for testing:

https://github.com/floooh/pacman.zig

WASM version here:

https://floooh.github.io/pacman.zig/pacman.html

Build instructions:

https://github.com/floooh/pacman.zig#experimental-web-support

The project is a mix of cross-platform C headers which makes use of Emscripten APIs and EM_JS() feature, and platform-agnostic Zig code for the gameplay stuff.

To make the build work I had to compile the Zig parts with wasm32-freestanding instead of wasm32-emscripten, otherwise I would get build errors in Zig standard library. Here's the "hack":

https://github.com/floooh/pacman.zig/blob/549a73ecd6f5c9bfe4f8150b08e4b43f02eae331/build.zig#L88-L91

To reproduce the problem just comment out this line:

https://github.com/floooh/pacman.zig/blob/549a73ecd6f5c9bfe4f8150b08e4b43f02eae331/build.zig#L89

...trying to build then produces this error:

/opt/homebrew/Cellar/zig/0.9.0/lib/zig/std/start.zig:318:17: error: unsupported arch
        else => @compileError("unsupported arch"),
                ^
/opt/homebrew/Cellar/zig/0.9.0/lib/zig/std/os.zig:134:24: error: container 'std.os.system' has no member called 'fd_t'
pub const fd_t = system.fd_t;
                       ^
/opt/homebrew/Cellar/zig/0.9.0/lib/zig/std/fs/file.zig:31:26: note: referenced here
    pub const Handle = os.fd_t;
                         ^
/opt/homebrew/Cellar/zig/0.9.0/lib/zig/std/fs/file.zig:15:13: note: referenced here
    handle: Handle,
            ^
/opt/homebrew/Cellar/zig/0.9.0/lib/zig/std/debug.zig:290:34: note: referenced here
                const stderr = io.getStdErr().writer();

Cheers!

[floooh]

PS: as I learned from this thread (https://groups.google.com/g/emscripten-discuss/c/fWPEAulgslM), building the C code with wasm32-emscripten is required to make the "EM_JS() magic" work. The Javascript source string is placed into a global variable which gets some special treatment by Clang only if the wasm32-emscripten triple is set. The string is then extracted from the object files by the Emscripten linker and placed into the Javascript file that's produced along with the .wasm and .html output.

I think the biggest motivation to better support wasm32-emscripten in Zig is that this could pave the road for an EM_JS()-like feature directly in Zig :)

...ideally at some later point it would be great if such special Emscripten linker features could be somehow integrated into the Zig toolchain, or if this is 'out of scope" maybe a better way to integrate a slim EMSDK (just the linker, sysroot and JS shims) with a Zig build.

[kripken]

@KioriSun

But it didn't work for me. Maybe I'm missing something. I tried adding it into a script tag, body onload, and adding it directly into the zig.js that emcc builds. Nothing seems to work only running from the console.

It's possible that adding that hook in the wrong place will not work. Here is what works for me locally, this is near the end of the HTML file:

        };
      };
      Module.onRuntimeInitialized = function() { _run_zig() }; // this is the new line
    </script>
    <script async type="text/javascript" src="a.js"></script>

[KioriSun]

Module.onRuntimeInitialized = function() { _run_zig() };

Thanks a lot! I found the issue. All of the above works if you do ; Module.onRuntimeInitialized = function() { _run_zig() };.
Mind the ;. It seems that at the end of the js script in the html, emscripten doesn't emit a last, trailing, ";", and that's why it wasn't working before.

[RReverser]

All of the above works if you do ; Module.onRuntimeInitialized = function() { _run_zig() };.
Mind the ;. It seems that at the end of the js script in the html, emscripten doesn't emit a last, trailing, ";", and that's why it wasn't working before.

That's odd, it really shouldn't matter.

[Luukdegram]

The next steps for this (if this were to be accepted), would be to provide proper std support for the wasm32-emscripten target triple, similar to how we provide support for WASI. This also means that the startup code of Zig must be made aware of Emscripten, so functions such as _run_zig() are no longer necessary. The Emscripten target could then also possibly become tier 2 supported. I think something like a slim EMSDK serves better as a third-party library that utilizes Zig's extensive build system so users can easily integrate with it.

[JosiasAurel]

I am using RayLib with Zig and I can compile my program with zig build-exe main.zig -lc -lraylib but the web build fails to run properly.

Here is the code for main.zig

const std = @import("std");
const c = @cImport({
    @cInclude("raylib.h");
});

export fn main() void {
    const screenWidth = 800;
    const screenHeight = 480;

    const windowTitle = "funLab";

    c.InitWindow(screenWidth, screenHeight, windowTitle);

    while (!c.WindowShouldClose()) {
        _update();
    }

    c.CloseWindow();
}

fn _update() void {
    const Msg = "Zig x C x RayLib";
    c.BeginDrawing();
    c.ClearBackground(c.RAYWHITE);
    c.DrawText(Msg, 200, 240, 50, c.LIGHTGRAY);
    c.EndDrawing();
}

Building object file from zig

zig build-obj -isystem ~/.mybin/emsdk/upstream/emscripten/cache/sysroot/include src/main.zig -I. -L. -target wasm32-wasi

Finish building with emscripten

emcc main.o -o index.html -s FULL_ES3 -s USE_GLFW=3 -s ASYNCIFY -s STANDALONE_WASM -s -s EXPORTED_FUNCTIONS=_main -lraylib -I. -L. --no-entry -O3 -Wall -sASSERTIONS --shell-file minshell.html -sSAFE_HEAP

I get an unreachable code executed on the web build.

Here is the minshell.html from official raylib source but I added Module.onRuntimeInitialized = funciton() { _main(); } within script tags at the end of the file.

references:

• https://gist.github.com/kripken/58c0e640227fe5bac9e7b30100a2a1d3#emscripten-as-a-linker-for-zig-and-c

[floooh]

(deleted my previous comment because it was full of false information, see end of the comment for current workarounds)

Current state in pacman.zig with the final zig-0.11.0 version is:

Trying to build Zig code with wasm32-emscripten into a library still has this stdlib error:

/Users/floh/bin/zig/lib/std/os.zig:153:24: error: struct 'os.system__struct_2831' has no member named 'fd_t'
pub const fd_t = system.fd_t;
                 ~~~~~~^~~~~
/Users/floh/bin/zig/lib/std/os.zig:73:13: note: struct declared here
    else => struct {},
            ^~~~~~~~~
referenced by:
    Handle: /Users/floh/bin/zig/lib/std/fs/file.zig:31:26
    File: /Users/floh/bin/zig/lib/std/fs/file.zig:15:13
    remaining reference traces hidden; use '-freference-trace' to see all reference traces

Trying to build Zig code which has a pub fn main() function with wasm32-emscripten now throws an unsupported arch error in start.zig (which is probably kind of expected, because it's not a WASI target)

install transitive failure
└─ run emsdk/upstream/emscripten/cache/sysroot/../../emcc transitive failure
   └─ install game transitive failure
      └─ zig build-lib game Debug wasm32-emscripten 2 errors
/Users/floh/bin/zig/lib/std/start.zig:324:21: error: unsupported arch
            else => @compileError("unsupported arch"),
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/floh/bin/zig/lib/std/os.zig:153:24: error: struct 'os.system__struct_2839' has no member named 'fd_t'
pub const fd_t = system.fd_t;
                 ~~~~~~^~~~~
/Users/floh/bin/zig/lib/std/os.zig:73:13: note: struct declared here
    else => struct {},

What I'm doing now in pacman.zig for the 'web browser target':

• build the Zig code into a library with target wasm32-freestanding to avoid the above errors
• build the C code into a library with target wasm32-emscripten (required for the Emscripten EM_JS magic to work)
• don't link with Zig, instead invoke emcc in a build step to build a C file with Emscripten compatible entry point, and link the above to libraries, this then results in the usual .html+.js+.wasm file triplet

PS: for reference: https://github.com/floooh/pacman.zig/blob/a2ce623da2a564beea5aea4512c48e9e1d260fc7/build.zig#L48-L134

[permutationlock]

I was able to get emscripten builds working for my own projects with guidance from the gist linked above and the build files from projects such as raylib-zig. From there, I ended up testing out some additions to the standard library that would add basic support for the wasm32-emscripten target, see this commit in my fork.

I copied the supported portions of os/linux.zig and c/linux.zig to their emscripten counterparts, modifying as necessary to match the emscripten C headers. I also added definitions for most of the API defined in emscripten.h (emscripten has a ton of other functionality, but not all of it is easy to translate, e.g. the EM_JS macros). Finally, I added a comptime block to export emscripten compatible versions of the __stack_chk symbols that are required when compiling in safe optimization modes (I don't know if this is the best solution, but it satisfied the linker and correctly reported when I tested smashing the stack).

To build a native project for the web you should just need to usezig build-obj -target wasm32-emscripten -lc and then link with emcc (or do the equivalent using the zig build system). When linking zig objects built with a safe optimization mode, emcc requires the flag -sUSE_OFFSET_CONVERTER or else runtime errors will occur related to __builtin_return_address.

[andrewrk]

It's been possible to target emscripten as the "OS" of zig for a while now. Any improvements to the standard library to support this OS are welcome.

[floooh]

Btw, confirmed that this is working now. Nice :)

[silbinarywolf]

Not sure where I should note my Emscripten findings but I'm gonna dump them here for now so others can discover them.
This is for Zig version: 0.12.0-dev.2701+d18f52197

• Put this in your main.zig to force use of the C allocator to avoid what seems to be memory collisions when using C libraries like SDL. My guess is because you're not allocating with Emscriptens malloc, you end up stomping over things allocated with C libraries.

// Force allocator to use c allocator for emscripten
pub const os = if ( builtin.os.tag != .wasi) std.os else struct {
    pub const heap = struct {
        pub const page_allocator = std.heap.c_allocator;
    };
};

• Updated various linkWithEmscripten settings to help with debugging problems and fix memory errors that mostly just give an "unreachable" error in console when using Zig. Based off of: https://github.com/Not-Nik/raylib-zig/blob/183347adaf8fb6dddf25ece3c8a157c8bab212c5/build.zig#L322

emcc_command.addArgs(&[_][]const u8{
    "-g", // adds debugging information to build, ie. get symbol names for functions where crash occurred, you need "strip = false" in your build
    "-o",
    emccOutputDir ++ emccOutputFile,
    "-sFULL-ES3=1",
    "-sUSE_GLFW=3",
    "-sINITIAL_MEMORY=512Mb", // increased default heap memory
    "-sSTACK_SIZE=1000000", // 1mb, raised to what Windows has to prevent a crash in my app
    // Debug options
    "-sSAFE_HEAP=1",
    "-sASSERTIONS=1", // note(jae): ASSERTIONS=2 crashes due to not rounding down mouse position for SDL2, https://github.com/emscripten-core/emscripten/issues/19655
    "-sSTACK_OVERFLOW_CHECK=1",
    "-sUSE_OFFSET_CONVERTER=1", // not sure if this is necessary anymore for Zig builds.
    "-sMALLOC='dlmalloc'",
    "-sASYNCIFY",
    "-sASYNCIFY_STACK_SIZE=5120000", // I increased this randomly to stop problems, not sure how necessary this change was.
    "-O0", // "-O3", //-Og = debug
    "--emrun",
});

• Updated the following in compileForEmscripten to include .strip = false, this is needed so Emscripten won't hit DWARF file opcode errors: https://github.com/Not-Nik/raylib-zig/blob/183347adaf8fb6dddf25ece3c8a157c8bab212c5/build.zig#L264
• Updating the local current working directory for emscripten build and update how you embed local filesystem resources.

link_step.addArg("--embed-file");
link_step.addArg("src/assets@/wasm_data"); // "src/assets" is a real folder in the code repo, "wasm_data" is the directory it maps to when embedded.

// belongs in your main.zig

pub const std_options: std.Options = .{ .wasiCwd = if (builtin.os.tag == .wasi) defaultWasiCwd else std.fs.defaultWasiCwd };
var default_wasi_dir = if (builtin.os.tag == .wasi) std.fs.defaultWasiCwd() else void;
pub fn defaultWasiCwd() std.os.wasi.fd_t {
    // Expect the first preopen to be current working directory.
    return default_wasi_dir;
}

pub fn main() !void {
    if (builtin.os.tag == .wasi) {
        const dir = try std.fs.cwd().openDir("/wasm_data", .{});
        default_wasi_dir = dir.fd;
    }
}

NOTE: If you're wondering why I did this, it's because even if you do the following when mapping your embed file, to be able to see files in emscripten/web builds, you'd need to prefix each file open with /, ie. std.fs.cwd().openFile("/my_file.png", .{ .mode = .read_only }); which doesn't work on both native desktop and Emscripten.

link_step.addArg("--embed-file");
link_step.addArg("src/assets@");