Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support RubyGems / Bundler #358

Merged
merged 44 commits into from Jan 4, 2024
Merged

Support RubyGems / Bundler #358

merged 44 commits into from Jan 4, 2024

Conversation

kateinoigakukun
Copy link
Member

@kateinoigakukun kateinoigakukun commented Jan 2, 2024

Demo

$ bundle init
Writing new Gemfile to /tmp/tmp.ZeVCP05HyB/Gemfile
$ bundle add ruby_wasm
$ bundle add rainbow
$ bundle exec rbwasm build -o ruby.wasm
(snip)
INFO: Packaging gem: rainbow-3.1.1
INFO: Packaging setup.rb: bundle/setup.rb
INFO: Size: 51.17 MB

$ cat <<EOS > app.rb
require "/bundle/setup"
require "rainbow"

puts Rainbow("Hello RubyGems").green + " with " + Rainbow("WebAssembly").underline.bright + "!"
EOS

$ wasmtime run --dir ./::/ ruby.wasm app.rb
Screenshot 2024-01-04 at 15 01 45

What's Added

  • exe/rbwasm: CLI tool to build .wasm file packaging gems.
    • It does the following steps:
      1. Configure CRuby to make rbconfig.rb
      2. Build extension libraries in gems specified in Gemfile
      3. Build CRuby and static-link extension libraries built in step b
      4. Package gem files by wasi-vfs.
    • The gem packaging scheme is similar to bundle install --standalone
      • rbwasm generates bundle/setup.rb as well as bundle install --standalone
      • But extension libraries are cross-compiled
  • ext/ruby_wasm: An extension library to use wasi-vfs and wizer from Ruby.

What's Changed

  • js library is now gemified: Moved under packages/gems/js
  • *-js build profiles, which additionally link js extension, are now generalized as a RubyGems application.
    • packages/npm-packages/ruby-wasm-wasi/Gemfile specifies gems to package

TODO

  • Allow to package user app source files: Introduce rbwasm pack command as an alias of wasi-vfs pack #361
  • Support dynamic linking for extension libraries.
    • Currently rbwasm uses a custom ext library builder instead of RubyGems' one since RubyGems' builder always expects dynamic linking and mkmf does not support make install for static linking.
      With dynamic linking, I hope we can use RubyGems' ext builder.
    • Some gems with complex ext library like nokogiri does feature detection using ruby's C header files but they are not ready before make install of CRuby build. This problem will be solved by dynamic linking, which allows building ext library after make install of CRuby build.
  • Publish pre-built ext/ruby_wasm extension as fat-gems
  • Include pre-built rubies/ruby-3.3-wasm32-unknown-wasi-full.tar.gz in gem package
    • Users can use it without building CRuby if the given Gemfile does not include any extension library

I've tried two approaches to implement the Component Model for Ruby.
The first one is the *binary-editing* approach and the second one is
*extension-library* approach.

The basic idea of binary-editing* is to inject import/export sections
into a preview1 compatible .wasm binary and turn it into a Component by
`wasm-tool component new` command.

The approach was good for binary reusability. The binary before
injection can be used as a standalone WASI preview1 binary. Therefore
user can use a pre-built .wasm binary and just inject the import/export
sections to turn it into a Component.

However, considering the future dynamic linking support, the approach
introduces a bunch of complexity. In the dynamic linking era, an
extension library will be able to provide a WIT interface and a Ruby
core interpreter should not import or export anything except for WASI.
The binary editing approach needs to edit the Ruby core interpreter
binary every time when WIT world changes, so the build process would
be different from the one of dynamically linked extension library.

I decided to go to the extension library approach for mainainability.

The static linking build is just for a compatibility with the existing
Wasm runtimes, so I don't want to pay much effort for maintining it. The
extention library approach uses almost the same build process for static
linking and dynamic linking, so it's easier to maintain and we can focus
on the dynamic linking support.
@kateinoigakukun kateinoigakukun force-pushed the katei/support-gem branch 2 times, most recently from 5a84e7d to ff26de2 Compare January 3, 2024 00:45
Since there is no guarantee that the host system and the builder
container use the same Ruby ABI
@kateinoigakukun kateinoigakukun force-pushed the katei/support-gem branch 3 times, most recently from c630a67 to e9f5f6c Compare January 3, 2024 18:50
to use Bundler in rbwasm command
@kateinoigakukun kateinoigakukun changed the title [WIP] Support RubyGems / Bundler Support RubyGems / Bundler Jan 4, 2024
@kateinoigakukun kateinoigakukun linked an issue Jan 4, 2024 that may be closed by this pull request
@kateinoigakukun kateinoigakukun marked this pull request as ready for review January 4, 2024 06:08
@kateinoigakukun kateinoigakukun merged commit c1be033 into main Jan 4, 2024
20 checks passed
@kateinoigakukun kateinoigakukun deleted the katei/support-gem branch January 4, 2024 06:09
@rubyFeedback
Copy link

Could we also add to the FAQ a few examples for gems that are supported? From the above I now know that rainbow gem is supported. I'd like a few more - does not have to mention EVERY gem, but, say, half a dozen (aka 6) gems that we could experiment with would be nice.

Also the example used bundler. Can we use "gem" too? If so, how? All my gems use a .gemspec file; I do not really use bundler myself so I am curious if I can avoid bundler (I am aware it is bundled via gem itself nowadays, but I still don't want to have to maintain any Gemfile from bundler, I am too happy with .gemspec since many years).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Rubygems and bundler
2 participants