compiled extensions go to wrong directory when using gem install

[twk3]

If you are installing the gem using bundle install or gem install, and need to compile the native extensions (arm for example), the extensions will compile, but will be placed in src/ruby/bin/grpc/grpc_c.so instead of src/ruby/lib/grpc/grpc_c.so

which results in the following error when you use the gem:

LoadError: cannot load such file -- /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/grpc-1.1.2/src/ruby/lib/grpc/grpc_c

This happens because rubygems uses the first require_path as one of the extension destinations: https://github.com/rubygems/rubygems/blob/af3c7bed24106cb86ad56210fcb90696d00c31d5/lib/rubygems/ext/builder.rb#L155

And in this gem, the bin directory is first:
https://github.com/grpc/grpc/blob/v1.1.2/grpc.gemspec#L27

[apolcyn]

Haven't seen this before. What's the command, ruby version, and ruby platform?

[twk3]

@apolcyn

The error happens as soon as you do a require 'grpc' (I get the error if I do a bundle exec irb, then require 'grpc')

ruby version: ruby 2.3.3p222 (2016-11-21 revision 56859) [armv6l-linux-eabihf]
platform: armv6l-linux
rubygems: 2.6.10

grpc installed using bundle install

This is on raspbian jessie, we have tried on a pi2 pi3 and scaleway's arm servers.

[jacobvosmaer]

What I don't understand is why this only breaks on our ARM build. What @twk3 writes sounds like a platform=ruby problem, not an ARM problem. And then why do we not have this problem when using one of the platform-specific gem files from rubygems.org?

[jacobvosmaer]

The contents of data.tar.gz in https://rubygems.org/gems/grpc/versions/1.1.2-x86_64-linux and https://rubygems.org/gems/grpc/versions/1.1.2 (platform ruby) are identical except for the following files not included in platform ruby:

./src/ruby/lib/grpc/2.0/grpc_c.so
./src/ruby/lib/grpc/2.1/grpc_c.so
./src/ruby/lib/grpc/2.2/grpc_c.so
./src/ruby/lib/grpc/2.3/grpc_c.so
./src/ruby/lib/grpc/2.4/grpc_c.so
./src/ruby/lib/grpc/grpc_c.so

[twk3]

It looks like when the gem is build and packaged using rake, rake-compiler hardcodes using the lib directory: https://github.com/rake-compiler/rake-compiler/blob/master/lib/rake/baseextensiontask.rb#L42
So that is why the pre-compiled versions of the gem work.

But when building using rubygems, (as a result of a gem install) rubygems uses the first require path as the lib directory.

[jacobvosmaer]

I am able to reproduce the problem on ubuntu 16.04 with:

apt-get install build-essential cmake pkg-config ruby2.3-dev
gem install grpc --platform=ruby
ruby -rgrpc -e 'puts "OK"'

The last command errors out with LoadError.

[jacobvosmaer]

Putting bin before lib in gemspec require_paths seems to have been introduced in 17b1659

[jacobvosmaer]

Using the steps in #9998 (comment) I also reproduced this on grpc 1.0.0.

[Op3n4M3]

@apolcyn this looks like the general case you were concerned may exist when looking into #9861.

I ran into this today when testing 1.2.0.pre1 gem. It would be nice to see the fix required in 1.2.0 release on 3/20 and cherry-picked into 1.1.x for anyone not able to update to 1.2.x immediately after release.

[jacobvosmaer]

It seems the fix made it into 1.2.2. Yay! https://github.com/grpc/grpc/blob/v1.2.2/grpc.gemspec