Unable to install nio4r gem via Vagrant plugin

[just3ws]

Hi,

TL;DR The nio4r gem builds fine on my Ruby 1.9.3 but fails to build from Vagrant's Ruby interpreter.

There seems to be an issue with Xcode >=4.6.3 on Mountain Lion being unable to compile some gems with native code using the Vagrant >= 1.3.4 embedded Ruby, in this case nio4r.

This is a continuation of an issue started over on the vagrant-berkshelf repository.

berkshelf/vagrant-berkshelf#89

One other vagrant-berkshelf user, @zepheiryan dug into the issue a little deeper, from the thread on the vagrant-berkshelf issue.

I'm having the same problem with about the same environment, except I have a full XCode install.

Looking at the files the error messages point to, I see the conflict is that OS X defines structs timeval and timespec, but the parameters to tell Ruby that those aren't missing aren't provided in nio4r's extconf.rb (it should define HAVE_STRUCT_TIMESPEC and HAVE_STRUCT_TIMEVAL based on the presence of sys/_structs.h, perhaps in a platform-dependent manner; it does not).

I "solved" this by temporarily commenting out the struct definitions of Vagrant's embedded Ruby's missing.h. I also had to make sure my ARCHFLAGS was unset for my environment because the embedded Ruby does not provide headers for x86_64-darwin12.3.0, only universal. The install succeeded after making those changes.

I have no qualms about doing this locally just to get this working since Ruby development is low on my activities list, but I'm guessing as a real solution, this is an issue to bring up with the nio4r team. I'll file an issue there if that makes sense to others.

I've also submitted an error directly on the nio4r (the gem throwing the exception) but since this was because Vagrant requires using a custom build of Ruby I'd also submit the issue here.

socketry/nio4r#35

Full output of the error trace below:

unknown28cfe9162235 :: ~/src/vgrnt ‹master*› % vagrant plugin install vagrant-berkshelf 1 ↵
Installing the 'vagrant-berkshelf' plugin. This can take a few minutes...
/Applications/Vagrant/embedded/lib/ruby/1.9.1/rubygems/installer.rb:562:in `rescue in block in build_extensions': ERROR: Failed to build gem native extension. (Gem::Installer::ExtensionBuildError)

    /Applications/Vagrant/embedded/bin/ruby extconf.rb

checking for rb_thread_blocking_region()... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.

Provided configuration options:
--with-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/Applications/Vagrant/embedded/bin/ruby
/Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:381:in try_do': The compiler failed to generate an executable file. (RuntimeError) You have to install development tools first. from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:455:inblock in try_link0'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/tmpdir.rb:83:in mktmpdir' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:452:intry_link0'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:476:in try_link' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:619:intry_func'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:894:in block in have_func' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:790:inblock in checking_for'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:284:in block (2 levels) in postpone' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:254:inopen'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:284:in block in postpone' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:254:inopen'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:280:in postpone' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:789:inchecking_for'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:893:in have_func' from extconf.rb:3:in

'

Gem files will remain installed in /Users/Michael/.vagrant.d/gems/gems/nio4r-0.5.0 for inspection.
Results logged to /Users/Michael/.vagrant.d/gems/gems/nio4r-0.5.0/ext/nio4r/gem_make.out
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/rubygems/installer.rb:540:in block in build_extensions' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/rubygems/installer.rb:515:ineach'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/rubygems/installer.rb:515:in build_extensions' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/rubygems/installer.rb:180:ininstall'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/rubygems/dependency_installer.rb:297:in block in install' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/rubygems/dependency_installer.rb:270:ineach'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/rubygems/dependency_installer.rb:270:in each_with_index' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/rubygems/dependency_installer.rb:270:ininstall'
from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/plugins/commands/plugin/action/install_gem.rb:49:in block in call' from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/plugins/commands/plugin/gem_helper.rb:42:inblock in with_environment'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/rubygems/user_interaction.rb:40:in use_ui' from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/plugins/commands/plugin/gem_helper.rb:41:inwith_environment'
from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/plugins/commands/plugin/action/install_gem.rb:39:in call' from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/lib/vagrant/action/warden.rb:34:incall'
from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/plugins/commands/plugin/action/bundler_check.rb:20:in call' from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/lib/vagrant/action/warden.rb:34:incall'
from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/lib/vagrant/action/builder.rb:116:in call' from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/lib/vagrant/action/runner.rb:61:inblock in run'
from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/lib/vagrant/util/busy.rb:19:in busy' from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/lib/vagrant/action/runner.rb:61:inrun'
from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/plugins/commands/plugin/command/base.rb:17:in action' from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/plugins/commands/plugin/command/install.rb:27:inexecute'
from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/plugins/commands/plugin/command/root.rb:52:in execute' from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/lib/vagrant/cli.rb:38:inexecute'
from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/lib/vagrant/environment.rb:478:in cli' from /Applications/Vagrant/embedded/gems/gems/vagrant-1.3.5/bin/vagrant:96:in<top (required)>'
from /Applications/Vagrant/bin/../embedded/gems/bin/vagrant:23:in load' from /Applications/Vagrant/bin/../embedded/gems/bin/vagrant:23:in

'

[merqlove]

Same here.
Xcode 5.0.1,
OSX10.9

vagrant.d/gems/gems/nio4r-0.5.0/ext/nio4r/gem_make.out

/Applications/Vagrant/embedded/bin/ruby extconf.rb
checking for rb_thread_blocking_region()... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.

Provided configuration options:
--with-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/Applications/Vagrant/embedded/bin/ruby
/Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:381:in try_do': The compiler failed to generate an executable file. (RuntimeError) You have to install development tools first. from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:455:inblock in try_link0'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/tmpdir.rb:83:in mktmpdir' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:452:intry_link0'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:476:in try_link' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:619:intry_func'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:894:in block in have_func' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:790:inblock in checking_for'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:284:in block (2 levels) in postpone' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:254:inopen'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:284:in block in postpone' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:254:inopen'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:280:in postpone' from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:789:inchecking_for'
from /Applications/Vagrant/embedded/lib/ruby/1.9.1/mkmf.rb:893:in have_func' from extconf.rb:3:in

'

[merqlove]

"gcc -o conftest -I/Applications/Vagrant/embedded/include/ruby-1.9.1/x86_64-darwin12.3.0 -I/Applications/Vagrant/embedded/include/ruby-1.9.1/ruby/backward -I/Applications/Vagrant/embedded/include/ruby-1.9.1 -I. -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -I/vagrant-installer/staging/embedded/include -I/vagrant-installer/staging/embedded/include -L/vagrant-installer/staging/embedded/lib -fno-common -pipe conftest.c -L. -L/Applications/Vagrant/embedded/lib -L/vagrant-installer/staging/embedded/lib -L. -I/vagrant-installer/staging/embedded/include -L/vagrant-installer/staging/embedded/lib -Wl,-rpath,@loader_path/../lib -Wl,-rpath,@executable_path/../lib -L/vagrant-installer/staging/embedded/lib -arch x86_64 -lruby.1.9.1 -lpthread -ldl -lobjc "
In file included from conftest.c:1:
In file included from /Applications/Vagrant/embedded/include/ruby-1.9.1/ruby.h:32:
/Applications/Vagrant/embedded/include/ruby-1.9.1/ruby/ruby.h:24:10: fatal error: 'ruby/config.h' file not found
#include "ruby/config.h"
^
1 error generated.
checked program was:
/* begin _/
1: #include "ruby.h"
2:
3: int main(int argc, char *_argv)
4: {
5: return 0;
6: }
/* end */

[merqlove]

It's Vagrant bug. After duplicate this config.h to main ruby include folder make file was created :)

[merqlove]

Now another new Vagrant bug.
So on this i will stop my debugging.
I think librarian-chef is great time saver for now, until this bugs will be fixed:)

have_header: checking for port.h... -------------------- no

"gcc -I/Applications/Vagrant/embedded/include/ruby-1.9.1/x86_64-darwin12.3.0 -I/Applications/Vagrant/embedded/include/ruby-1.9.1/ruby/backward -I/Applications/Vagrant/embedded/include/ruby-1.9.1 -I. -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -I/vagrant-installer/staging/embedded/include -I/vagrant-installer/staging/embedded/include -L/vagrant-installer/staging/embedded/lib -fno-common -pipe -arch x86_64 -c conftest.c"
clang: warning: argument unused during compilation: '-L/vagrant-installer/staging/embedded/lib'
conftest.c:3:10: fatal error: 'port.h' file not found
#include <port.h>
^
1 error generated.
checked program was:
/* begin /
1: #include "ruby.h"
2:
3: #include <port.h>
/ end */

[merqlove]

I think i found solution.
Install is fine if i run "vagrant plugin install vagrant-berkshelf" with sudo.
After i just fix rights and that's it.

Tried to install nio4r via /Applications/Vagrant/embedded/bin/gem. Installing fine.
Maybe some with make file of nio4r?

[merqlove]

sudo vagrant plugin install vagrant-berkshelf && sudo chown -R "$USER":staff ~/.vagrant.d/*

[emil2k]

I was experiencing same issue with vagrant-salt plugin, and @merqlove's solution worked for me.

[merqlove]

Also second solution is clone plugin repo & do steps like this:

$ bundle
$ bundle exec thor gem:build
$ vagrant plugin install --plugin-prerelease --plugin-source https://rubygems.org pkg/vagrant-berkshelf-1.4.0.dev.gem

[zepheiryan]

The problem is that the embedded Ruby in Vagrant doesn't have a matching arch config.h for x86_64 and the latest Darwin. I assume using sudo merely changes the environment sufficiently to find and use an appropriate config.h, not that its traditional utility re: authentication has anything to do with it. Given the now inaccurate title for this issue, perhaps re-titling it or opening a new one focusing on the root problem would be more likely to attract a solution?

[jbpadgett]

I was getting a similar error trying to install vagrant-chef-zero as a Vagrant plugin.

/Applications/Vagrant/embedded/lib/ruby/1.9.1/rubygems/installer.rb:562:in rescue in block in build_extensions': ERROR: Failed to build gem native extension. (Gem::Installer::ExtensionBuildError) /Applications/Vagrant/embedded/bin/ruby extconf.rb creating Makefile make make: *** No rule to make target/Applications/Vagrant/embedded/include/ruby-1.9.1/x86_64-darwin12.3.0/ruby/config.h', needed by `yajl.o'. Stop.

Thanks, the sudo/chown solve worked for me with vagrant-chef-zero as well for now until the bug is fixed.

sudo vagrant plugin install vagrant-chef-zero && sudo chown -R "$USER":staff ~/.vagrant.d/*

I am using OSX 10.8.5
XCode 5.01
Vagrant 1.3.5
VirtualBox 4.2.18

[rrichards]

merqlove's solution using sudo worked for me as well:

vagrant 1.3.5
osx 10.9

an interesting side-note. I was unable to install the vagrant-aws gem as well. After doing the sudo trick i was able to install the vagrant-aws gem normally. w00t.

[mitchellh]

Hm, the problem seems to be that it needs some permissions in the Vagrant application folder. I'd prefer this not to be the case, so I wonder if there is a workaround that I can build into Vagrant itself.

[vickrum]

I have a similar problem with Vagrant's nio4r install.

$ vagrant plugin install vagrant-butcher
Installing the 'vagrant-butcher' plugin. This can take a few minutes...
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing nio4r (1.0.0), and Bundler cannot continue.
Make sure that `gem install nio4r -v '1.0.0'` succeeds before bundling.

Doing it manually requires sudo:
sudo /Applications/Vagrant/embedded/bin/gem install nio4r -v '1.0.0' -V)`

But that hangs on checking for rb_thread_blocking_region().... After interrupting the install, I see this in mkmf.log.

"clang -o conftest -I/Applications/Vagrant/embedded/include/ruby-2.0.0/universal-darwin12.5.0 -I/Applications/Vagrant/embedded/include/ruby-2.0.0/ruby/backward -I/Applications/Vagrant/embedded/include/ruby-2.0.0 -I. -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT -I/vagrant-substrate/staging/embedded/include   -I/vagrant-substrate/staging/embedded/include -fno-common -pipe conftest.c  -L. -L/Applications/Vagrant/embedded/lib -L/vagrant-substrate/staging/embedded/lib -L. -L/vagrant-substrate/staging/embedded/lib  -Wl,-rpath,@loader_path/../lib -Wl,-rpath,@executable_path/../lib -fstack-protector -L/vagrant-substrate/staging/embedded/lib    -arch x86_64 -arch i386   -lruby.2.0.0  -lpthread -ldl -lobjc "
ld: warning: directory not found for option '-L/vagrant-substrate/staging/embedded/lib'
ld: warning: directory not found for option '-L/vagrant-substrate/staging/embedded/lib'
ld: warning: directory not found for option '-L/vagrant-substrate/staging/embedded/lib'
ld: warning: directory not found for option '-L/vagrant-substrate/staging/embedded/lib'
ld: warning: directory not found for option '-L/vagrant-substrate/staging/embedded/lib'
ld: warning: directory not found for option '-L/vagrant-substrate/staging/embedded/lib'
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return 0;
6: }
/* end */

There is no /vagrant-substrate directory on my machine.

OS X 10.9
Vagrant 1.5.1

None of the suggested solutions solutions in this thread have worked unfortunately.

[vickrum]

Managed to fix it locally. It's as @zepheiryan said, so replacing config.h wasn't enough. I had to replace Vagrant's embedded Ruby entirely. I used OS X's version of Ruby 2.0 located in /System/Library/Frameworks/Ruby.framework/Versions/2.0/, which was presumably installed by XCode or as part of Mavericks (I normally use rbenv).

[mitchellh]

Berkshelf fixed this upstream