Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support --disable-static option on TruffleRuby #2191

flavorjones opened this issue Feb 10, 2021 · 2 comments

support --disable-static option on TruffleRuby #2191

flavorjones opened this issue Feb 10, 2021 · 2 comments


Copy link

flavorjones commented Feb 10, 2021

See for more context.

TruffleRuby, by default, sets the --use-system-libraries option (via these lines in mkmf.rb). Because of how Nokogiri's extconf.rb is currently implemented, this setting takes precedence over other options like --disable-static which would allow users to compile the packaged, patched versions of libxml2/libxslt and use them as shared libraries (instead of the system libraries).

I've proposed, in, to change Nokogiri's extconf.rb to essentially follow this logic:

static_p = enable_config("static", true) # defaults to true, overridden to false by `--disable-static`
if defined?(::TruffleRuby)
  if static_p
    # use system libraries - sane default to make installation easy
    # use packaged libraries, built as shared objects - use the patched packaged versions, which might be hard for some systems to compile

Essentially, this means that the environment variable NOKOGIRI_USE_SYSTEM_LIBRARIES would be presumed to be the default behavior on TruffleRuby; but allow it to be overridden.

Copy link

eregon commented Feb 17, 2021

I've made a small change locally to allow overriding NOKOGIRI_USE_SYSTEM_LIBRARIES with command-line arguments (PR incoming: #2193).

With that, it's possible to install nokogiri in all 3 variants. I'm using the latest TruffleRuby nightly build.

Current default, using system libraries:

$ time gem install --local pkg/nokogiri-1.11.1.gem    
Building native extensions. This could take a while...
Successfully installed nokogiri-1.11.1
1 gem installed
 34.24s user 2.19s system 424% cpu 8.579 total

Compiling the vendored libxml2/libxslt as shared libraries:

$ time gem install --local pkg/nokogiri-1.11.1.gem -- --use-system-libraries=false --disable-static 
Building native extensions with: '--use-system-libraries=false --disable-static'
This could take a while...
Successfully installed nokogiri-1.11.1
1 gem installed
99.45s user 12.71s system 281% cpu 39.789 total

Compiling the vendored libxml2/libxslt as static libraries (the default on CRuby):

$ time gem install --local pkg/nokogiri-1.11.1.gem -- --use-system-libraries=false                  
Building native extensions with: '--use-system-libraries=false'
This could take a while...
Successfully installed nokogiri-1.11.1
1 gem installed
136.03s user 13.37s system 223% cpu 1:06.99 total

The latter 2 include the libxml2 patches from nokogiri since they use the vendored version.
They are also compiled as bitcode (in addition to native code) and run on Sulong.

Running on Sulong might actually fix #1882, but the drawback is then libxml2/libxslt need to warm up on Sulong, which is slower than native code, and for large C libraries like those it's probably a better experience to run them natively when possible, and the flexibility from Sulong is not needed.

I think this is something that is probably best to solve on TruffleRuby or Sulong's side, where one could specify that some library dependencies should be run natively and not on Sulong, even if they have bitcode.

Regarding defaults, it seems vendored libxml2/libxslt as shared libaries is strictly better on TruffleRuby than vendored libxml2/libxslt as static libraries:

  • Faster to compile
  • Flexibility to run the vendored libraries as native code (this is very difficult with static libraries, but should be easy with shared libraries)
  • Clear dependencies as seen with ldd/otool (minor, but nice for debugging).

Between shared libraries and system libraries it's less clear.

  • --use-system-libraries installs faster (9s vs 40s, but 40s isn't really long either)
  • but --use-system-libraries requires to have the libraries installed on the system which is inconvenient in CI, notably it's an issue on GitHub Actions and TravisCI which don't have them installed by default. On GitHub Actions, the libraries are automatically installed with ruby/setup-ruby and bundler-cache: true, but that takes an extra 13-15s.
  • and --use-system-libraries has extra bugs and doesn't match the default behavior on CRuby
  • however --use-system-libraries is probably slightly better for warmup until we have support in TruffleRuby/Sulong to always execute libxml2/libxslt as native. Early testing seems to show that running libxml2/libxslt is fast enough and warmup is OK, at least in TruffleRuby --native mode (the default).

Copy link
Member Author

Closing this issue in favor of the work at #2193

eregon added a commit to eregon/nokogiri that referenced this issue Mar 9, 2021
* Shared libraries are more flexible and compile faster, see
  sparklemotion#2191 (comment)
* Static libraries can still be chosen (e.g., for testing) with:
  gem install nokogiri -- --use-system-libraries=false --enable-static

Co-authored-by: Benoit Daloze <>
This was referenced Mar 11, 2021
This was referenced Mar 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet

No branches or pull requests

2 participants