Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stage2: Upstream zwld (partly) into wasm backend #10229

Merged
merged 3 commits into from
Nov 27, 2021

Conversation

Luukdegram
Copy link
Member

@Luukdegram Luukdegram commented Nov 26, 2021

This upstreams a lot of logic from https://github.com/Luukdegram/zwld.
For now, I have chosen a hybrid method of linking, where easy-to-track and small computations are done using incremental style but hard-to-track linking is delayed until flushModule(). This has the benefits of having more information available during the linking process, and not having to worry too much about updating all indexes constantly. Besides, this will allow us to merge the remaining parts of zwld (once it gains the missing features), with almost no effort, meaning we will then be able to link both stage1 as well as stage2 code with external object files.

Prior to this PR, we were using a userland implementation of an offset table where we placed it inside wasm's memory. After this PR, we will generate and resolve relocations properly. We now have all the information available to also produce object files from the wasm backend, rather than only executables (once the emitting parts are implemented).

cc @kubkon

@andrewrk
Copy link
Member

andrewrk commented Nov 26, 2021

Nice work!

The CI failure is due to compile errors from trying to build the compiler on 32-bit systems. You should be able to reproduce (even in zwld) by throwing a -Dtarget=arm-linux-musleabihf on there.

+ /workspace/_debug/staging/bin/zig build -Dtarget=arm-linux-musleabihf
./src/link/Wasm.zig:501:32: error: expected type 'usize', found 'u64'
        if (!std.mem.isAligned(initial_memory, page_size)) {
                               ^
./src/link/Wasm.zig:501:32: note: unsigned 32-bit int cannot represent all possible unsigned 64-bit values
        if (!std.mem.isAligned(initial_memory, page_size)) {
                               ^

- Converts previous `DeclBlock` into `Atom`'s to also make them compatible when
the rest of zlwd gets upstreamed and we can link with other object files.
- Resolves function signatures and removes any duplicates, saving us a lot of
potential bytes for larger projects.
- We now create symbols for each decl of the respective type
- We can now (but not implemented yet) perform proper relocations.
- Having symbols and segment_info allows us to create an object file
for wasm.
We now resolve relocations for globals, memory addresses and function indexes.
Besides above, we now also emit imported functions correctly and create a
corresponding undefined symbol for it, where as we create a defined symbol
for all other cases.

TODO: Make incrememental compilation work again with new linker infrastructure
This contains a few additions:
- Proper stack pointer calculation keeping alignment in mind.
- Setting up memory layout (including user flags).
- Export or import memory
- Handle 'easy' linker tasks during incremental compilation, while offloading
heavy-tracking/computation tasks to `flush()`
- This architecture allows us to easily integrate with the rest of 'zwld' to
implement linking stage2 code with external object files.
@Luukdegram
Copy link
Member Author

Luukdegram commented Nov 27, 2021

Nice work!

The CI failure is due to compile errors from trying to build the compiler on 32-bit systems. You should be able to reproduce (even in zwld) by throwing a -Dtarget=arm-linux-musleabihf on there.

+ /workspace/_debug/staging/bin/zig build -Dtarget=arm-linux-musleabihf
./src/link/Wasm.zig:501:32: error: expected type 'usize', found 'u64'
        if (!std.mem.isAligned(initial_memory, page_size)) {
                               ^
./src/link/Wasm.zig:501:32: note: unsigned 32-bit int cannot represent all possible unsigned 64-bit values
        if (!std.mem.isAligned(initial_memory, page_size)) {
                               ^

Thanks for the pointer on the target, saved me a lot of time :). Fixed it in 6e88df4

@andrewrk andrewrk merged commit aa61e03 into ziglang:master Nov 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants