Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add WASI based WebAssembly support #5407

Merged
merged 20 commits into from
Jan 19, 2022

Conversation

kateinoigakukun
Copy link
Member

@kateinoigakukun kateinoigakukun commented Jan 7, 2022

WebAssembly / WASI port of Ruby

How to cross-build

Requirement

Steps

  1. Download a prebuilt WASI SDK package from WASI SDK release page.
  2. Set WASI_SDK_PATH environment variable to the root directory of the WASI SDK package.
$ export WASI_SDK_PATH=/path/to/wasi-sdk-X.Y
  1. Download a prebuilt binaryen from Binaryen release page
  2. Set PATH environment variable to lookup binaryen tools
$ export PATH=path/to/binaryen:$PATH
  1. Configure
  • You can select which extensions you want to build.
  • If you got Out of bounds memory access while running the produced ruby, you may need to increase the maximum size of stack.
$ ./configure LDFLAGS="-Xlinker -zstack-size=16777216" \
  --host wasm32-unknown-wasi \
  --with-destdir=./ruby-wasm32-wasi \
  --with-static-linked-ext \
  --with-ext=ripper,monitor
  1. Make
$ make install

Now you have a WASI compatible ruby binary. You can run it by any WebAssembly runtime like wasmtime, wasmer, Node.js, or browser with WASI polyfill.

Note: it may take a long time (~20 sec) for the first time for JIT compilation

$ wasmtime ruby-wasm32-wasi/usr/local/bin/ruby --mapdir /::./ruby-wasm32-wasi/ -- -e 'puts RUBY_PLATFORM'
wasm32-wasi

Current Limitation

  • No Thread support for now.
  • Spawning a new process is not supported. e.g. Kernel.spawn and Kernel.system

@shyouhei
Copy link
Member

shyouhei commented Jan 7, 2022

@ioquatix You might want to look at the coroutine implementation.

@nobu nobu added the Feature label Jan 15, 2022
@kateinoigakukun kateinoigakukun marked this pull request as ready for review January 15, 2022 11:47
@kateinoigakukun kateinoigakukun force-pushed the wasm-wasi-upstream-pr branch 2 times, most recently from 3e3f6d7 to f8407ef Compare January 15, 2022 17:53
wasm/machine.c Outdated Show resolved Hide resolved
process.c Outdated Show resolved Hide resolved
ruby.c Outdated Show resolved Hide resolved
dir.c Outdated Show resolved Hide resolved
dir.c Outdated Show resolved Hide resolved
@KernelErr
Copy link

Nice work! Tested on WasmEdge. However, the required version of Binaryen seems should be 91 or others. For latest 105 version, there is an unsatisfiable import: asyncify::stop_unwind.

image

@kateinoigakukun kateinoigakukun force-pushed the wasm-wasi-upstream-pr branch 7 times, most recently from d88368f to 74ed76e Compare January 18, 2022 10:59
clang does not yet support stack-protector for wasm
These flags are very wasi-libc version specific, so updating wasi-libc
may break the build. But supporting multiple wasi-libc versions in ruby
doesn't have much benefit because wasi-libc is not installed in most
systems.
configure.ac: setup build tools and register objects

main.c: wrap main with rb_wasm_rt_start to handle asyncify unwinds

tool/m4/ruby_wasm_tools.m4: setup default command based on WASI_SDK_PATH
environment variable. checks wasm-opt which is used for asyncify.

tool/wasm-clangw wasm/wasm-opt: a clang wrapper which replaces real
wasm-opt with do-nothing wasm-opt to avoid misoptimization before
asyncify. asyncify is performed at POSTLINK, but clang linker driver
tries to run optimization by wasm-opt unconditionally. inlining pass
at wasm level breaks asyncify's assumption, so should not optimize
before POSTLIK.

wasm/GNUmakefile.in: wasm specific rules to compile objects
set the default coroutine_type as asyncify when wasi
WASI currently does not yet support signal
WebAssembly has function local infinite registers and stack values, but
there is no way to scan the values in a call stack for now.
This implementation uses Asyncify to spilling out wasm locals into
linear memory.
This implementation does nothing around preemptive context switching
because there is no native thread.
Add a hook point to initialize extra extension libraries. The default
hook function is replaced when linking a strong `Init_extra_exts`
symbol. A builder can insert an object file that defines Init_extra_exts
by XLDFLAGS.
WASI doesn't support spawning a new process for now.
RB_WAITFD_PRI uses POLLPRI for other platforms, but wasi-libc doesn't
have POLLPRI for now.
@supechicken
Copy link

supechicken commented Mar 4, 2022

Hello, I am trying to follow the build instruction in the comment, but I get the following error while running make install:

linking ruby
wasm-ld: error: ext/pty/pty.a(pty.o): undefined symbol: rb_fork_async_signal_safe
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)

Output of ${WASI_SDK_PATH}/bin/clang --version (tried building from source and using prebuilt binaries):

clang version 14.0.0 (https://github.com/llvm/llvm-project 80e2c587498a7b2bf14dde47a33a058da6e88a9a)
Target: wasm32-unknown-wasi
Thread model: posix
InstalledDir: /usr/local/wasi/wasi-sdk/build/install/opt/bin/

Output of ./wasm-opt --version:

wasm-opt version 91 (version_91)

Does there any steps I have missed? Thanks!

@kateinoigakukun
Copy link
Member Author

@supechicken It seems you're trying to build an unsupported ext pty for wasi. You need to select only supported exts by --with-ext. The list of supported exts is here https://github.com/kateinoigakukun/ruby.wasm/blob/75c4007d229786ae638510bcf39cafa08a932c23/Rakefile#L14

@eat4toast

This comment was marked as off-topic.

@hsbt
Copy link
Member

hsbt commented Mar 8, 2022

@supechicken @eat4toast Can you file your question to https://bugs.ruby-lang.org/ ? This tracker is not support space.

@eat4toast
Copy link

@supechicken @eat4toast Can you file your question to https://bugs.ruby-lang.org/ ? This tracker is not support space.

Sorry, I will move the question.
Thank you.

@junaruga
Copy link
Member

I think the instruction document below to run Ruby with WebAssembly support, created in this PR is useful for someone to try to do. So, let me put the link here.
https://github.com/ruby/ruby/blob/master/wasm/README.md

Because this PR URL is referred from the release note.
https://www.ruby-lang.org/en/news/2022/09/09/ruby-3-2-0-preview2-released/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
9 participants