ruby-build built ruby is slow #335

Closed
derekprior opened this Issue Mar 27, 2013 · 18 comments

Comments

Projects
None yet
4 participants
@derekprior
Contributor

derekprior commented Mar 27, 2013

I have a new 15" Retina MacBook Pro (Early 2013) that I have installed numerous rubies on via rbenv and ruby-build. These rubies work just fine, but they are very slow. To illustrate, I created the following ruby script:

require 'benchmark'
array = (1..1000000).map { rand }
puts Benchmark.measure { array.dup.sort }

I then ran the benchmark on multiple rubies:

# System Ruby (1.8.7)
➜  time /usr/bin/ruby bm.rb
  1.470000   0.010000   1.480000 (  1.478272)
/usr/bin/ruby bm.rb  1.71s user 0.02s system 99% cpu 1.740 total

# rbenv/ruby-build installed ruby 2.0.0
➜  time ~/.rbenv/versions/2.0.0-p0/bin/ruby bm.rb
  2.770000   0.010000   2.780000 (  2.746690)
~/.rbenv/versions/2.0.0-p0/bin/ruby bm.rb  3.14s user 0.03s system 99% cpu 3.173 total

# homebrew installed ruby 2.0.0
➜  time /usr/local/bin/ruby bm.rb
  0.990000   0.010000   1.000000 (  0.989383)
/usr/local/bin/ruby bm.rb  1.15s user 0.03s system 99% cpu 1.185 total

That is, Ruby 2.0 built via ruby-build is more than 2.5 times slower than the same ruby built with homebrew and ialmost twice as slow as the system installed ruby 1.8.7.

GCC is present and accounted for:

➜  which gcc
/usr/bin/gcc

➜  gcc --version
i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)

I'm really interested in figuring out what's going on here, but I don't know what to look at next. Has anyone seen this before? What can I look at next?

@derekprior

This comment has been minimized.

Show comment
Hide comment
@derekprior

derekprior Mar 28, 2013

Contributor

Thanks for the suggestion, but that's not it. The homebrew-built ruby doesn't set any CFLAGS at all and it runs just fine. I manually built a working ruby with good performance using the following configure statement:

./configure --prefix=/Users/derek/.rbenv/versions/2.0.0-p0 --enable-shared --disable-tcltk-framework --with-out-ext=tcl --with-out-ext=tk --disable-install-doc --with-openssl-dir=/usr/local/opt/openssl

This differs from the configure statement ruby-build runs (even with verbose output, I can't actually see the configure statement, but I extracted this from the resulting Makefile):

configure_args =  '--prefix=/Users/derek/.rbenv/versions/2.0' '--with-openssl-dir=/Users/derek/.rbenv/versions/2.0/openssl' 'CFLAGS=  -Wno-error=shorten-64-to-32' 'LDFLAGS=-L'\''/Users/derek/.rbenv/versions/2.0/lib'\'' ' 'CPPFLAGS=-I'\''/Users/derek/.rbenv/versions/2.0/include'\'' '

I decided to try RVM and found the rubies it builds are also performant. Here are the configure_args its makefile records:

configure_args =  '--prefix=/Users/derek/.rvm/rubies/ruby-2.0.0-p0' '--with-opt-dir=/usr/local/opt/libyaml:/usr/local/opt/readline:/usr/local/opt/libxml2:/usr/local/opt/libxslt:/usr/local/opt/libksba:/usr/local/opt/openssl:/usr/local/opt/sqlite' '--disable-install-doc' '--without-tcl' '--without-tk' '--enable-shared' 'CC=/usr/local/bin/gcc-4.2'

I'd prefer to use rbenv and ruby-build. but I haven't been able to get a performant ruby built with it on this machine. Some things I tried:

  • forcing CC=/usr/local/bin/gcc-4.2 when running ruby-build
  • running brew link openssl before ruby-build so ruby-build doesn't build its own openssl
  • commenting out the clang workaround to eliminate the CFLAGS discrepancy.

I want to figure this out. I'm willing to try more things but each test is relatively time consuming and I don't know enough to make an educated guess at what might be the next best thing to try. I don't know what most of these flags do.

Contributor

derekprior commented Mar 28, 2013

Thanks for the suggestion, but that's not it. The homebrew-built ruby doesn't set any CFLAGS at all and it runs just fine. I manually built a working ruby with good performance using the following configure statement:

./configure --prefix=/Users/derek/.rbenv/versions/2.0.0-p0 --enable-shared --disable-tcltk-framework --with-out-ext=tcl --with-out-ext=tk --disable-install-doc --with-openssl-dir=/usr/local/opt/openssl

This differs from the configure statement ruby-build runs (even with verbose output, I can't actually see the configure statement, but I extracted this from the resulting Makefile):

configure_args =  '--prefix=/Users/derek/.rbenv/versions/2.0' '--with-openssl-dir=/Users/derek/.rbenv/versions/2.0/openssl' 'CFLAGS=  -Wno-error=shorten-64-to-32' 'LDFLAGS=-L'\''/Users/derek/.rbenv/versions/2.0/lib'\'' ' 'CPPFLAGS=-I'\''/Users/derek/.rbenv/versions/2.0/include'\'' '

I decided to try RVM and found the rubies it builds are also performant. Here are the configure_args its makefile records:

configure_args =  '--prefix=/Users/derek/.rvm/rubies/ruby-2.0.0-p0' '--with-opt-dir=/usr/local/opt/libyaml:/usr/local/opt/readline:/usr/local/opt/libxml2:/usr/local/opt/libxslt:/usr/local/opt/libksba:/usr/local/opt/openssl:/usr/local/opt/sqlite' '--disable-install-doc' '--without-tcl' '--without-tk' '--enable-shared' 'CC=/usr/local/bin/gcc-4.2'

I'd prefer to use rbenv and ruby-build. but I haven't been able to get a performant ruby built with it on this machine. Some things I tried:

  • forcing CC=/usr/local/bin/gcc-4.2 when running ruby-build
  • running brew link openssl before ruby-build so ruby-build doesn't build its own openssl
  • commenting out the clang workaround to eliminate the CFLAGS discrepancy.

I want to figure this out. I'm willing to try more things but each test is relatively time consuming and I don't know enough to make an educated guess at what might be the next best thing to try. I don't know what most of these flags do.

@jeremy

This comment has been minimized.

Show comment
Hide comment
@jeremy

jeremy Mar 28, 2013

Member

Hey @derekprior - this is a Ruby issue, unfortunately. Check out https://bugs.ruby-lang.org/issues/7984

Member

jeremy commented Mar 28, 2013

Hey @derekprior - this is a Ruby issue, unfortunately. Check out https://bugs.ruby-lang.org/issues/7984

@derekprior

This comment has been minimized.

Show comment
Hide comment
@derekprior

derekprior Mar 28, 2013

Contributor

From reading that thread it doesn't seem 100% clear that is the case that it's a bug with ruby.

I've seen this behavior on my laptop with 2.0 and 1.9.3 (haven't tested anything older). Certainly there exists some combination of dependencies and flags that result in performant rubies because I can get them from rvm and homebrew.

I have an additional mac (a 2011 iMac) for which rbenv and ruby-build-built rubies work great. It's running the same version of OS X, the same dotfile configuration, and the same versions of most homebrew formula.

At any rate, at least the ruby thread seems to indicate that I'm not alone. Thanks for the tip.

Contributor

derekprior commented Mar 28, 2013

From reading that thread it doesn't seem 100% clear that is the case that it's a bug with ruby.

I've seen this behavior on my laptop with 2.0 and 1.9.3 (haven't tested anything older). Certainly there exists some combination of dependencies and flags that result in performant rubies because I can get them from rvm and homebrew.

I have an additional mac (a 2011 iMac) for which rbenv and ruby-build-built rubies work great. It's running the same version of OS X, the same dotfile configuration, and the same versions of most homebrew formula.

At any rate, at least the ruby thread seems to indicate that I'm not alone. Thanks for the tip.

@derekprior

This comment has been minimized.

Show comment
Hide comment
@derekprior

derekprior Mar 29, 2013

Contributor

I hand-built ruby with the following process...

wget http://ftp.ruby-lang.org/pub/ruby/2.0/ruby-2.0.0-p0.tar.gz
tar -xzvf ruby-2.0.0-p0.tar.gz
cd ruby-2.0.0-p0
./configure --prefix=/Users/derek/.rubies/2.0.0-p0 —with-openssl-dir=/usr/local/opt/openssl
make
make install

and re-ran the benchamk I posted above:

➜  time ~/.rubies/2.0.0-p0/bin/ruby bm.rb
  0.910000   0.010000   0.920000 (  0.906132)
~/.rubies/2.0.0-p0/bin/ruby bm.rb  1.06s user 0.03s system 99% cpu 1.092 total

That's a 3x improvement.

Contributor

derekprior commented Mar 29, 2013

I hand-built ruby with the following process...

wget http://ftp.ruby-lang.org/pub/ruby/2.0/ruby-2.0.0-p0.tar.gz
tar -xzvf ruby-2.0.0-p0.tar.gz
cd ruby-2.0.0-p0
./configure --prefix=/Users/derek/.rubies/2.0.0-p0 —with-openssl-dir=/usr/local/opt/openssl
make
make install

and re-ran the benchamk I posted above:

➜  time ~/.rubies/2.0.0-p0/bin/ruby bm.rb
  0.910000   0.010000   0.920000 (  0.906132)
~/.rubies/2.0.0-p0/bin/ruby bm.rb  1.06s user 0.03s system 99% cpu 1.092 total

That's a 3x improvement.

@jeremy

This comment has been minimized.

Show comment
Hide comment
@jeremy

jeremy Apr 2, 2013

Member

Nice, @derekprior. Could you try tweaking the OpenSSL configure options to suss out what's responsible?

Member

jeremy commented Apr 2, 2013

Nice, @derekprior. Could you try tweaking the OpenSSL configure options to suss out what's responsible?

@derekprior

This comment has been minimized.

Show comment
Hide comment
@derekprior

derekprior Apr 4, 2013

Contributor

@jeremy it doesn't seem to be related to openssl. I just tried the following:

$ brew link openssl --force
$ ruby-build 2.0.0-p0 /Users/derek/.rubies/ruby-build-2.0.0-p0/

This built Ruby 2.0.0 and skipped building openssl, opting to use the version homebrew already installed. The resulting ruby is still slow.

Contributor

derekprior commented Apr 4, 2013

@jeremy it doesn't seem to be related to openssl. I just tried the following:

$ brew link openssl --force
$ ruby-build 2.0.0-p0 /Users/derek/.rubies/ruby-build-2.0.0-p0/

This built Ruby 2.0.0 and skipped building openssl, opting to use the version homebrew already installed. The resulting ruby is still slow.

@jeremy

This comment has been minimized.

Show comment
Hide comment
@jeremy

jeremy Apr 4, 2013

Member

@derekprior Turns out that Ruby uses optimization level -O3 by default for GCC, but none for Clang.

Try building with RUBY_CFLAGS="-O3 -march=native" rbenv install 2.0.0-p0 to work around that.

Member

jeremy commented Apr 4, 2013

@derekprior Turns out that Ruby uses optimization level -O3 by default for GCC, but none for Clang.

Try building with RUBY_CFLAGS="-O3 -march=native" rbenv install 2.0.0-p0 to work around that.

@derekprior

This comment has been minimized.

Show comment
Hide comment
@derekprior

derekprior Apr 4, 2013

Contributor

If that were the issue, why would it be different when I build manually? Would ruby-build choose a different compiler than I run by default from my shell?

Also, the build fails with those flags:

RUBY_CFLAGS="-O3 -march=native" ruby-build 2.0.0-p0 /Users/derek/.rubies/ruby-build-2.0.0-p0
Downloading openssl-1.0.1e.tar.gz...
-> https://www.openssl.org/source/openssl-1.0.1e.tar.gz
Installing openssl-1.0.1e...
Installed openssl-1.0.1e to /Users/derek/.rubies/ruby-build-2.0.0-p0

Downloading ruby-2.0.0-p0.tar.gz...
-> http://ftp.ruby-lang.org/pub/ruby/2.0/ruby-2.0.0-p0.tar.gz
Installing ruby-2.0.0-p0...

BUILD FAILED

Inspect or clean up the working tree at /var/folders/j3/bvx89xp114qdsp_ccnvj4_340000gn/T/ruby-build.20130404130743.68652
Results logged to /var/folders/j3/bvx89xp114qdsp_ccnvj4_340000gn/T/ruby-build.20130404130743.68652.log

Last 10 log lines:
configure: WARNING: unrecognized options: --with-openssl-dir
checking build system type... x86_64-apple-darwin12.3.0
checking host system type... x86_64-apple-darwin12.3.0
checking target system type... x86_64-apple-darwin12.3.0
checking for gcc-4.2... gcc-4.2
checking for gcc... (cached) gcc-4.2
checking whether the C compiler works... no
configure: error: in `/var/folders/j3/bvx89xp114qdsp_ccnvj4_340000gn/T/ruby-build.20130404130743.68652/ruby-2.0.0-p0':
configure: error: C compiler cannot create executables
See `config.log' for more details
Contributor

derekprior commented Apr 4, 2013

If that were the issue, why would it be different when I build manually? Would ruby-build choose a different compiler than I run by default from my shell?

Also, the build fails with those flags:

RUBY_CFLAGS="-O3 -march=native" ruby-build 2.0.0-p0 /Users/derek/.rubies/ruby-build-2.0.0-p0
Downloading openssl-1.0.1e.tar.gz...
-> https://www.openssl.org/source/openssl-1.0.1e.tar.gz
Installing openssl-1.0.1e...
Installed openssl-1.0.1e to /Users/derek/.rubies/ruby-build-2.0.0-p0

Downloading ruby-2.0.0-p0.tar.gz...
-> http://ftp.ruby-lang.org/pub/ruby/2.0/ruby-2.0.0-p0.tar.gz
Installing ruby-2.0.0-p0...

BUILD FAILED

Inspect or clean up the working tree at /var/folders/j3/bvx89xp114qdsp_ccnvj4_340000gn/T/ruby-build.20130404130743.68652
Results logged to /var/folders/j3/bvx89xp114qdsp_ccnvj4_340000gn/T/ruby-build.20130404130743.68652.log

Last 10 log lines:
configure: WARNING: unrecognized options: --with-openssl-dir
checking build system type... x86_64-apple-darwin12.3.0
checking host system type... x86_64-apple-darwin12.3.0
checking target system type... x86_64-apple-darwin12.3.0
checking for gcc-4.2... gcc-4.2
checking for gcc... (cached) gcc-4.2
checking whether the C compiler works... no
configure: error: in `/var/folders/j3/bvx89xp114qdsp_ccnvj4_340000gn/T/ruby-build.20130404130743.68652/ruby-2.0.0-p0':
configure: error: C compiler cannot create executables
See `config.log' for more details
@jeremy

This comment has been minimized.

Show comment
Hide comment
@jeremy

jeremy Apr 4, 2013

Member
If that were the issue, why would it be different when I build manually?

Because if ruby-build sets any CFLAGS, then Ruby's -O3 default doesn't kick in.

Also, the build fails with those flags:

Sorry about that—-march=native is clang-only. Try RUBY_CFLAGS="-O3" alone.

Member

jeremy commented Apr 4, 2013

If that were the issue, why would it be different when I build manually?

Because if ruby-build sets any CFLAGS, then Ruby's -O3 default doesn't kick in.

Also, the build fails with those flags:

Sorry about that—-march=native is clang-only. Try RUBY_CFLAGS="-O3" alone.

@derekprior

This comment has been minimized.

Show comment
Hide comment
@derekprior

derekprior Apr 5, 2013

Contributor

@jeremy: that was it.

Contributor

derekprior commented Apr 5, 2013

@jeremy: that was it.

@jeremy

This comment has been minimized.

Show comment
Hide comment
@jeremy

jeremy Apr 5, 2013

Member

@derekprior excellent!

We'll see about setting opts flags by default for all builds.

Member

jeremy commented Apr 5, 2013

@derekprior excellent!

We'll see about setting opts flags by default for all builds.

@jeremy jeremy closed this Apr 5, 2013

@dunric

This comment has been minimized.

Show comment
Hide comment
@dunric

dunric Apr 19, 2013

For the God's sake, why ruby-build just don't leave out default compiler flags as autodetected with autoconfigure shipped with Ruby sources ? At least as an option. I almost did pulled out my hair to track ruby-build compiles with suboptimal cflags.

dunric commented Apr 19, 2013

For the God's sake, why ruby-build just don't leave out default compiler flags as autodetected with autoconfigure shipped with Ruby sources ? At least as an option. I almost did pulled out my hair to track ruby-build compiles with suboptimal cflags.

@jeremy

This comment has been minimized.

Show comment
Hide comment
@jeremy

jeremy Apr 19, 2013

Member

@dunric Glad your hair is still intact. It is a pain. We solved a clang issue by setting a -W flag, but that clears Ruby's default flags. One step forward; two steps back! Please do dig in and lend a hand 😁

Member

jeremy commented Apr 19, 2013

@dunric Glad your hair is still intact. It is a pain. We solved a clang issue by setting a -W flag, but that clears Ruby's default flags. One step forward; two steps back! Please do dig in and lend a hand 😁

@derekprior

This comment has been minimized.

Show comment
Hide comment
@derekprior

derekprior Apr 19, 2013

Contributor

@jeremy I'd argue that this issue should remain open so folks looking to contribute know it exists.

Contributor

derekprior commented Apr 19, 2013

@jeremy I'd argue that this issue should remain open so folks looking to contribute know it exists.

@jeremy

This comment has been minimized.

Show comment
Hide comment
@jeremy

jeremy Apr 19, 2013

Member

@derekprior good idea!

Member

jeremy commented Apr 19, 2013

@derekprior good idea!

@jeremy jeremy reopened this Apr 19, 2013

@jeremy

This comment has been minimized.

Show comment
Hide comment
@jeremy

jeremy Apr 26, 2013

Member

Fix @ #351

Member

jeremy commented Apr 26, 2013

Fix @ #351

@jeremy jeremy closed this Apr 26, 2013

@derekprior

This comment has been minimized.

Show comment
Hide comment
@derekprior

derekprior Apr 26, 2013

Contributor

That's awesome. Thanks a bunch, @jeremy.

On Apr 26, 2013, at 6:15 PM, Jeremy Kemper notifications@github.com wrote:

Fix @ #351


Reply to this email directly or view it on GitHub.

Contributor

derekprior commented Apr 26, 2013

That's awesome. Thanks a bunch, @jeremy.

On Apr 26, 2013, at 6:15 PM, Jeremy Kemper notifications@github.com wrote:

Fix @ #351


Reply to this email directly or view it on GitHub.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment