New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build ruby with shared library #35

Open
mloughran opened this Issue Sep 13, 2011 · 22 comments

Comments

Projects
None yet
@mloughran

mloughran commented Sep 13, 2011

I'm not sure whether there is a good reason for this, but ruby-build does not build a ruby shared library.

This caused an issue for me when trying to build the zookeeper gem (ruby 1.9.2-p290), which fails to compile.

I solved the issue by recompiling ruby with

./configure --prefix=$HOME/.rbenv/versions/1.9.2-p290 --enable-shared

RVM appears to do this by default - can ruby-build do the same?

Thanks!

@sstephenson

This comment has been minimized.

Show comment
Hide comment
@sstephenson

sstephenson Sep 13, 2011

Contributor

That seems like a good default.

Contributor

sstephenson commented Sep 13, 2011

That seems like a good default.

@sstephenson

This comment has been minimized.

Show comment
Hide comment
@sstephenson

sstephenson Sep 14, 2011

Contributor

I just added support for passing options to configure with the CONFIGURE_OPTS variable. You can use this to set --enable-shared until I've had time to test the option with all the various definitions.

$ CONFIGURE_OPTS=--enable-shared ruby-build 1.9.2-p290 …
Contributor

sstephenson commented Sep 14, 2011

I just added support for passing options to configure with the CONFIGURE_OPTS variable. You can use this to set --enable-shared until I've had time to test the option with all the various definitions.

$ CONFIGURE_OPTS=--enable-shared ruby-build 1.9.2-p290 …
@mloughran

This comment has been minimized.

Show comment
Hide comment
@mloughran

mloughran commented Sep 14, 2011

Thanks!

@lilith

This comment has been minimized.

Show comment
Hide comment
@lilith

lilith Dec 12, 2013

diff_match_patch_native also requires the shared library. Upvote for --enable-shared by default?

lilith commented Dec 12, 2013

diff_match_patch_native also requires the shared library. Upvote for --enable-shared by default?

@mislav

This comment has been minimized.

Show comment
Hide comment
@mislav

mislav Dec 12, 2013

Member

From my limited knowledge, it seems that --enable-shared can't hurt to always have enabled, so I'm reopening this until we decide on it.

@nathanaeljones You didn't argue for your case well enough. What is diff_match_patch_native and why do you need it?

Member

mislav commented Dec 12, 2013

From my limited knowledge, it seems that --enable-shared can't hurt to always have enabled, so I'm reopening this until we decide on it.

@nathanaeljones You didn't argue for your case well enough. What is diff_match_patch_native and why do you need it?

@mislav mislav reopened this Dec 12, 2013

@lilith

This comment has been minimized.

Show comment
Hide comment
@lilith

lilith Dec 12, 2013

It's the only reasonably fast way to get accurate diffs between large strings in ruby. The native ruby version takes upwards of 30 seconds on a 33kb file.

Any gem that depends on Rice will require --enable-shared... It's pretty important for C++ interop.

lilith commented Dec 12, 2013

It's the only reasonably fast way to get accurate diffs between large strings in ruby. The native ruby version takes upwards of 30 seconds on a 33kb file.

Any gem that depends on Rice will require --enable-shared... It's pretty important for C++ interop.

@lilith

This comment has been minimized.

Show comment
Hide comment
@lilith

lilith Feb 9, 2014

Over the last two months we've wasted more than 80 hours due to this bug. CONFIGURE_OPTS does not work reliably; often it takes 4-5 re-installs before it works, with no clear difference as to why. Identical ~/.bash_profile and a clean shell state doesn't help. All 8 machines are brand-new MBPs with Mavericks. Another fails to build the shared libraries no matter how many attempts are made. I've compared the version numbers of all the build tools used during the process, and they're all the same.

This is probably the most costly bug we've encountered in the last 5 years. C++ and Ruby interop is important, no? Shared libs are built reliably, by default, with RVM, and despite its multitude of flaws, that one detail turned out more important than anything else in the rvm/rbenv balance.

lilith commented Feb 9, 2014

Over the last two months we've wasted more than 80 hours due to this bug. CONFIGURE_OPTS does not work reliably; often it takes 4-5 re-installs before it works, with no clear difference as to why. Identical ~/.bash_profile and a clean shell state doesn't help. All 8 machines are brand-new MBPs with Mavericks. Another fails to build the shared libraries no matter how many attempts are made. I've compared the version numbers of all the build tools used during the process, and they're all the same.

This is probably the most costly bug we've encountered in the last 5 years. C++ and Ruby interop is important, no? Shared libs are built reliably, by default, with RVM, and despite its multitude of flaws, that one detail turned out more important than anything else in the rvm/rbenv balance.

@mislav

This comment has been minimized.

Show comment
Hide comment
@mislav

mislav Feb 9, 2014

Member

I'm sorry that you lost a lot of time on this. Any idea why setting CONFIGURE_OPTS would not have effect 4 of out 5 times? It smells like it might be a bug in Ruby's build system.

Do you have a Ruby script I could run on a freshly compiled Ruby to test whether it was successfully built with --enable-shared?

Member

mislav commented Feb 9, 2014

I'm sorry that you lost a lot of time on this. Any idea why setting CONFIGURE_OPTS would not have effect 4 of out 5 times? It smells like it might be a bug in Ruby's build system.

Do you have a Ruby script I could run on a freshly compiled Ruby to test whether it was successfully built with --enable-shared?

@lilith

This comment has been minimized.

Show comment
Hide comment
@lilith

lilith Feb 10, 2014

I haven't found an easier way to test for --enable-shared than gem install rice. If there's an easier way to test for the presence of the .so files, let me know and I'll update our docs.

I don't have any solid leads on the inconsistent behavior. I lined up 6 of the laptops last week and did the full uninstall/reinstall on all of them. Multiple times. There were a few errors, but I excluded all those that I found.

For example, CONFIGURE_OPTS=--enable-shared rbenv install 2.0.0-p353 doesn't work, but CONFIGURE_OPTS="--enable-shared" rbenv install 2.0.0-p353 does (eventually). But not if you put them on two separate lines, or if you copy/paste from GitHub (I'm guessing whitespace weirdness, perhaps non-breaking spaces or something?).

The ruby build never fails - it always succeeds. The flag just isn't getting to the actual compilation step.

My uneducated guess is that somehow the bash variable is going out of scope too early, and not making it to ruby-build. IMHO, this should be a default, not an unreliable flag.

lilith commented Feb 10, 2014

I haven't found an easier way to test for --enable-shared than gem install rice. If there's an easier way to test for the presence of the .so files, let me know and I'll update our docs.

I don't have any solid leads on the inconsistent behavior. I lined up 6 of the laptops last week and did the full uninstall/reinstall on all of them. Multiple times. There were a few errors, but I excluded all those that I found.

For example, CONFIGURE_OPTS=--enable-shared rbenv install 2.0.0-p353 doesn't work, but CONFIGURE_OPTS="--enable-shared" rbenv install 2.0.0-p353 does (eventually). But not if you put them on two separate lines, or if you copy/paste from GitHub (I'm guessing whitespace weirdness, perhaps non-breaking spaces or something?).

The ruby build never fails - it always succeeds. The flag just isn't getting to the actual compilation step.

My uneducated guess is that somehow the bash variable is going out of scope too early, and not making it to ruby-build. IMHO, this should be a default, not an unreliable flag.

@DanielHeath

This comment has been minimized.

Show comment
Hide comment
@DanielHeath

DanielHeath May 27, 2014

Ouch. Just switched to chruby + ruby-build from RVM last week.

+1, no downside to building libruby & it's used by (e.g.) mapnik_ruby (via rice).

DanielHeath commented May 27, 2014

Ouch. Just switched to chruby + ruby-build from RVM last week.

+1, no downside to building libruby & it's used by (e.g.) mapnik_ruby (via rice).

@kiranos

This comment has been minimized.

Show comment
Hide comment
@kiranos

kiranos May 27, 2014

why not add this as default? I fail to see the reason. Everyone seems to be in favor of rbenv adding this as default?

kiranos commented May 27, 2014

why not add this as default? I fail to see the reason. Everyone seems to be in favor of rbenv adding this as default?

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Oct 29, 2015

For those with the same issue:
CONFIGURE_OPTS=--enable-shared rbenv install 2.2.3

ghost commented Oct 29, 2015

For those with the same issue:
CONFIGURE_OPTS=--enable-shared rbenv install 2.2.3

@mislav

This comment has been minimized.

Show comment
Hide comment
@mislav

mislav Oct 29, 2015

Member

What exactly requires the shared library and would there be any downsides to enabling it for all builds?

Member

mislav commented Oct 29, 2015

What exactly requires the shared library and would there be any downsides to enabling it for all builds?

@jeremy

This comment has been minimized.

Show comment
Hide comment
@jeremy

jeremy Oct 29, 2015

Member

Ruby doesn't build as a shared lib by default. Most things link fine against the static lib. Generally, those who need a shared lib know. We could expose a toplevel option to make it easier/more discoverable, though.

Rice does explicitly state "Also Rice requires a Ruby built with –enable-shared and will not install properly against a Ruby with only static libraries." but it doesn't give any guidance on how to do that with rbenv, rvm, etc.

A little more background: https://www.ruby-forum.com/topic/4422189

Also, can be more specific than blanket CONFIGURE_OPTS:

RUBY_CONFIGURE_OPTS=--enable_shared rbenv install 2.2.3
Member

jeremy commented Oct 29, 2015

Ruby doesn't build as a shared lib by default. Most things link fine against the static lib. Generally, those who need a shared lib know. We could expose a toplevel option to make it easier/more discoverable, though.

Rice does explicitly state "Also Rice requires a Ruby built with –enable-shared and will not install properly against a Ruby with only static libraries." but it doesn't give any guidance on how to do that with rbenv, rvm, etc.

A little more background: https://www.ruby-forum.com/topic/4422189

Also, can be more specific than blanket CONFIGURE_OPTS:

RUBY_CONFIGURE_OPTS=--enable_shared rbenv install 2.2.3
@mislav

This comment has been minimized.

Show comment
Hide comment
@mislav

mislav Oct 29, 2015

Member

Is there a short code snippet one could use in ruby -e "..." to test if the current Ruby has shared lib support?

Member

mislav commented Oct 29, 2015

Is there a short code snippet one could use in ruby -e "..." to test if the current Ruby has shared lib support?

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Oct 30, 2015

Shared libraries are libraries that are loaded by programs when they start. When a shared library is installed properly, all programs that start afterwards automatically use the new shared library. So basically if you have a program that needs a shared library (in this case ruby libruby.so.2.2.0 or similar) the program will fail (to start hopefully) if it doesn't exist. if you want to know if you have the shared library you can do a simple search for the file libruby.so*, but as jeremy indicated, you don't need it, and those that do will know.

ghost commented Oct 30, 2015

Shared libraries are libraries that are loaded by programs when they start. When a shared library is installed properly, all programs that start afterwards automatically use the new shared library. So basically if you have a program that needs a shared library (in this case ruby libruby.so.2.2.0 or similar) the program will fail (to start hopefully) if it doesn't exist. if you want to know if you have the shared library you can do a simple search for the file libruby.so*, but as jeremy indicated, you don't need it, and those that do will know.

@rickypai

This comment has been minimized.

Show comment
Hide comment
@rickypai

rickypai Dec 28, 2015

seems like you can test it with RbConfig::CONFIG["ENABLE_SHARED"] == "no"

https://github.com/jasonroelofs/rice/blob/master/extconf.rb#L21

rickypai commented Dec 28, 2015

seems like you can test it with RbConfig::CONFIG["ENABLE_SHARED"] == "no"

https://github.com/jasonroelofs/rice/blob/master/extconf.rb#L21

@cmol

This comment has been minimized.

Show comment
Hide comment
@cmol

cmol Feb 7, 2016

Or ruby -e "require 'rbconfig'; puts RbConfig::CONFIG['ENABLE_SHARED']" for the lazy.

I just came across the same issue with the cwiid gem, that didn't compile until ruby was installed with --enable-shared. My "system" ruby comes with --enable-shared as well so I cannot see why this shouldn't be default for ruby-build.

cmol commented Feb 7, 2016

Or ruby -e "require 'rbconfig'; puts RbConfig::CONFIG['ENABLE_SHARED']" for the lazy.

I just came across the same issue with the cwiid gem, that didn't compile until ruby was installed with --enable-shared. My "system" ruby comes with --enable-shared as well so I cannot see why this shouldn't be default for ruby-build.

@Ruin0x11

This comment has been minimized.

Show comment
Hide comment
@Ruin0x11

Ruin0x11 Feb 9, 2016

I wasted several hours trying to install qtbindings before realizing --enable-shared wasn't set by default. I also don't see any reason not to set it by default.

Ruin0x11 commented Feb 9, 2016

I wasted several hours trying to install qtbindings before realizing --enable-shared wasn't set by default. I also don't see any reason not to set it by default.

@lilith

This comment has been minimized.

Show comment
Hide comment
@lilith

lilith Mar 10, 2016

Can this please be re-opened? Lack of shared by default breaks so many applications and gems.

lilith commented Mar 10, 2016

Can this please be re-opened? Lack of shared by default breaks so many applications and gems.

@mislav

This comment has been minimized.

Show comment
Hide comment
@mislav

mislav Mar 13, 2016

Member

@nathanaeljones This issue was open all along.

If someone wants to contribute a workaround that does --enable-shared by default for MRI, that would be welcome. I didn't realize this affected so many people, and so far we are not aware of a potential downside of doing that by default.

Member

mislav commented Mar 13, 2016

@nathanaeljones This issue was open all along.

If someone wants to contribute a workaround that does --enable-shared by default for MRI, that would be welcome. I didn't realize this affected so many people, and so far we are not aware of a potential downside of doing that by default.

@rubypanther

This comment has been minimized.

Show comment
Hide comment
@rubypanther

rubypanther Mar 13, 2016

Contributor

Is the most desirable way to add this by adding an enable_shared package target?
For example right now ruby-2.3.0 has this line in the definition file:
install_package "ruby-2.3.0" "https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.0.tar.bz2#ec7579eaba2e4c402a089dbc86c98e5f1f62507880fd800b9b34ca30166bfa5e" ldflags_dirs standard verify_openssl
and I've got a fork that adds a build_package_enable_shared function in the ruby-build script, and then the package options just change to ldflags_dirs enable_shared standard verify_openssl
That way it can be controlled from the definitions file. Good, bad, comments?

edit to add: Here is what I came up with, as you can see it is a minimalist approach without any safety checks: rubypanther@4f7e4c7

Contributor

rubypanther commented Mar 13, 2016

Is the most desirable way to add this by adding an enable_shared package target?
For example right now ruby-2.3.0 has this line in the definition file:
install_package "ruby-2.3.0" "https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.0.tar.bz2#ec7579eaba2e4c402a089dbc86c98e5f1f62507880fd800b9b34ca30166bfa5e" ldflags_dirs standard verify_openssl
and I've got a fork that adds a build_package_enable_shared function in the ruby-build script, and then the package options just change to ldflags_dirs enable_shared standard verify_openssl
That way it can be controlled from the definitions file. Good, bad, comments?

edit to add: Here is what I came up with, as you can see it is a minimalist approach without any safety checks: rubypanther@4f7e4c7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment