Unable to install plugins that depend on nokogiri in Vagrant 1.6+ #3769

Closed
lox opened this Issue May 13, 2014 · 169 comments

Projects

None yet
@lox
lox commented May 13, 2014

Since installing Vagrant 1.6.0, I'm unable to install landrush, or other plugins that depend on nokogiri:

I initially thought this was a landrush issue, but have reproduced with other vagrant plugins:

vagrant-landrush/landrush#69

I tested 1.6.1 and 1.6.2, both of which fail the same way. Reverting to 1.5.4 fixes the problem.

@mitchellh
Owner

Could you share the logs? The underlying installer foundation didn't change at all between 1.5 and 1.6, which makes this odd.

@lox
lox commented May 13, 2014

Which logs shall I include? The landrush bug has a trace of the output. It's certainly strange, in that I'm sure it's been working fine for several days, but suddenly today our CI system and my own installation started to have the same issue. Entirely possible it's a bug elsewhere.

@AlexeyPopov

I have same error with "vagrant plugin update" command.

Error log:

Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing nokogiri (1.6.2), and Bundler cannot continue.
Make sure that gem install nokogiri -v '1.6.2' succeeds before bundling.

@jonbuffington

Same issue here:

$ vagrant --version
Vagrant 1.6.2

$ vagrant plugin list
vagrant-login (1.0.1, system)
vagrant-share (1.0.1, system)
vagrant-vmware-fusion (2.4.1)

$ vagrant plugin update vagrant-vmware-fusion
Updating plugins: vagrant-vmware-fusion. This may take a few minutes...
Building nokogiri using packaged libraries.
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing nokogiri (1.6.2), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.2'` succeeds before bundling.
@arosenhagen

I run into this as well with vagrant 1.6.0, 1.6.1 and 1.6.2.
Downgrading to 1.5.4 helps (but well..not really ;-))

$ vagrant plugin install vagrant-omnibus
Installing the 'vagrant-omnibus' plugin. This can take a few minutes...
Building nokogiri using packaged libraries.
Building libxml2-2.8.0 for nokogiri with the following patches applied:
    - 0001-Fix-parser-local-buffers-size-problems.patch
    - 0002-Fix-entities-local-buffers-size-problems.patch
    - 0003-Fix-an-error-in-previous-commit.patch
    - 0004-Fix-potential-out-of-bound-access.patch
    - 0005-Detect-excessive-entities-expansion-upon-replacement.patch
    - 0006-Do-not-fetch-external-parsed-entities.patch
    - 0007-Enforce-XML_PARSER_EOF-state-handling-through-the-pa.patch
    - 0008-Improve-handling-of-xmlStopParser.patch
    - 0009-Fix-a-couple-of-return-without-value.patch
    - 0010-Keep-non-significant-blanks-node-in-HTML-parser.patch
    - 0011-Do-not-fetch-external-parameter-entities.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxml2.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install

However, note that nokogiri does not necessarily support all versions
of libxml2.

For example, libxml2-2.9.0 and higher are currently known to be broken
and thus unsupported by nokogiri, due to compatibility problems and
XPath optimization bugs.
************************************************************************
Building libxslt-1.1.28 for nokogiri with the following patches applied:
    - 0001-Adding-doc-update-related-to-1.1.28.patch
    - 0002-Fix-a-couple-of-places-where-f-printf-parameters-wer.patch
    - 0003-Initialize-pseudo-random-number-generator-with-curre.patch
    - 0004-EXSLT-function-str-replace-is-broken-as-is.patch
    - 0006-Fix-str-padding-to-work-with-UTF-8-strings.patch
    - 0007-Separate-function-for-predicate-matching-in-patterns.patch
    - 0008-Fix-direct-pattern-matching.patch
    - 0009-Fix-certain-patterns-with-predicates.patch
    - 0010-Fix-handling-of-UTF-8-strings-in-EXSLT-crypto-module.patch
    - 0013-Memory-leak-in-xsltCompileIdKeyPattern-error-path.patch
    - 0014-Fix-for-bug-436589.patch
    - 0015-Fix-mkdir-for-mingw.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxslt.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install
************************************************************************
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing nokogiri (1.6.2), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.2'` succeeds before bundling.
@robsonpeixoto

The same problem:

$ rm -rf ~/.vagrant.d
$ vagrant plugin update
Updating installed plugins...
Building nokogiri using packaged libraries.
Building libxml2-2.8.0 for nokogiri with the following patches applied:
    - 0001-Fix-parser-local-buffers-size-problems.patch
    - 0002-Fix-entities-local-buffers-size-problems.patch
    - 0003-Fix-an-error-in-previous-commit.patch
    - 0004-Fix-potential-out-of-bound-access.patch
    - 0005-Detect-excessive-entities-expansion-upon-replacement.patch
    - 0006-Do-not-fetch-external-parsed-entities.patch
    - 0007-Enforce-XML_PARSER_EOF-state-handling-through-the-pa.patch
    - 0008-Improve-handling-of-xmlStopParser.patch
    - 0009-Fix-a-couple-of-return-without-value.patch
    - 0010-Keep-non-significant-blanks-node-in-HTML-parser.patch
    - 0011-Do-not-fetch-external-parameter-entities.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxml2.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install

However, note that nokogiri does not necessarily support all versions
of libxml2.

For example, libxml2-2.9.0 and higher are currently known to be broken
and thus unsupported by nokogiri, due to compatibility problems and
XPath optimization bugs.
************************************************************************
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing nokogiri (1.6.2), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.2'` succeeds before bundling.
@arosenhagen

FYI: this might be caused by the recent upgrade of the nokogiri gem to 1.6.2
A quick workaround would be to pin the nokogiri gem version.
Unfortunately I don't know where the gem is required/bundled by vagrant :-/

@arosenhagen

Quick fix: add s.add_dependency(%q<nokogiri>, ["< 1.6.2"]) to ~/Applications/Vagrant/embedded/gems/specifications/vagrant-1.6.2.gemspec

@MisumiRize

Installing nokogiri gem with system installed libxml2 is temporal solution for this issue.

  1. Install libxml2 to system.
  2. Run export NOKOGIRI_USE_SYSTEM_LIBRARIES=true.
  3. Then try vagrant plugin update.
@fgrehm
Collaborator
fgrehm commented May 13, 2014

I've seen this happen when installing a pre-release version of vagrant-lxc and a user has just reported it as well.

Funny fact is that vagrant-lxc has no gem dependencies at all oO

@clarkbk
clarkbk commented May 13, 2014

Yep, I'm seeing this issue as well when attempting to install vmware_fusion /cc @josegonzalez

@clarkbk
clarkbk commented May 13, 2014

Confirmed that a downgrade works, though the export trick doesn't, and the s.add_dependency method doesn't work if you've deleted your .vagrant.d directory.

@clarkbk
clarkbk commented May 13, 2014

And now attempting to install vagrant-berkshelf in 1.5.4 also gives me the same error...

@mitchellh
Owner

Ah, I wonder if this is caused by 1.6 adding the vagrant-windows dependencies which have a Nokogiri aspect.

@mitchellh
Owner

/cc @sneal

@mitchellh mitchellh added the bug label May 13, 2014
@drpebcak

I'm hitting this issue as well with vagrant-cachier, vagrant-omnibus, AND vagrant-hostsupdater

@c0rn0
c0rn0 commented May 13, 2014

Downgrading to 1.5.4 doesn't work for me. After the downgrade when running vagrant plugin install vagrant-aws I still get the:

An error occurred while installing nokogiri (1.6.2), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.2'` succeeds before bundling.

Is there another version you guys downgraded to? Also how do you delete the current installation, by just deleting the /Applications/Vagrant directory and the /usr/bin/vagrant binary?

@clarkbk
clarkbk commented May 13, 2014

After downgrading to 1.5.4 all my plugin reinstalls were working, except berkshelf. It was giving me that nokogiri error. It ended up working fine when I explicitly declared the plugin version in the install command:

This:

$ vagrant plugin install vagrant-berkshelf --plugin-version '>=2.0.1'

Not this:

$ vagrant plugin install vagrant-berkshelf

Maybe that will help you with vagrant-aws… or maybe not.

@mitchellh
Owner

I believe this is actually caused by a recent Nokogiri update. Our build servers have recently failed to even build the Vagrant installers anymore because of this issue. I'll attempt to find a workaround.

@nodje
nodje commented May 14, 2014

Same problem here on a fresh Debian 7.5 install with Vagrant 1.6.2, trying to install vagrant-lxc

@yveslaroche

It works now on my machine with the last Nokogiri update v1.6.2.1.

1.6.2.1 / 2014-05-13

Bug fixes

Fix statically-linked libxml2 installation when using universal builds of Ruby. sparklemotion/nokogiri#1104

Patching mini_portile to address the git dependency detailed in sparklemotion/nokogiri#1102

Library load fix to address segfault reported on some systems. sparklemotion/nokogiri#1097

@ncloward

👍

@emyl emyl referenced this issue in emyl/vagrant-triggers May 14, 2014
Closed

Installation #10

@mitchellh
Owner

Fixed for me too. I think this was a Nokogiri bug.

@mitchellh mitchellh closed this May 14, 2014
@clarkbk
clarkbk commented May 14, 2014

Another 👍

@tmatilai
Collaborator

I find it a bit suboptimal that nokogiri (or other core dependencies) gets upgraded even if the plugin doesn't have any dependency on it. But maybe there's no easy and safe way around it with Bundler?

@mitchellh
Owner

@tmatilai Yeah I agree, not at the moment. I've been thinking of some optimizatoins around Bundler but I don't think what I'm thinking would fix this issue.

@geauxvirtual

I've just upgraded to Vagrant 1.6.2 and ran into the same issue with nokogiri 1.6.2.1. I am able to manually install nokogiri 1.6.2.1 successfully on my OS X laptop, but vagrant fails with

Updating installed plugins...
Building nokogiri using packaged libraries.
Building libxml2-2.8.0 for nokogiri with the following patches applied:
    - 0001-Fix-parser-local-buffers-size-problems.patch
    - 0002-Fix-entities-local-buffers-size-problems.patch
    - 0003-Fix-an-error-in-previous-commit.patch
    - 0004-Fix-potential-out-of-bound-access.patch
    - 0005-Detect-excessive-entities-expansion-upon-replacement.patch
    - 0006-Do-not-fetch-external-parsed-entities.patch
    - 0007-Enforce-XML_PARSER_EOF-state-handling-through-the-pa.patch
    - 0008-Improve-handling-of-xmlStopParser.patch
    - 0009-Fix-a-couple-of-return-without-value.patch
    - 0010-Keep-non-significant-blanks-node-in-HTML-parser.patch
    - 0011-Do-not-fetch-external-parameter-entities.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxml2.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install

However, note that nokogiri does not necessarily support all versions
of libxml2.

For example, libxml2-2.9.0 and higher are currently known to be broken
and thus unsupported by nokogiri, due to compatibility problems and
XPath optimization bugs.
************************************************************************
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing nokogiri (1.6.2.1), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.2.1'` succeeds before bundling.

Will check to see if any of the workarounds work.

@jonbuffington

@geauxvirtual I have the same issue on a system where the command line tools are not installed. I do have Xcode 5.1.1 installed.

$ vagrant plugin update vagrant-vmware-fusion
Updating plugins: vagrant-vmware-fusion. This may take a few minutes...
Building nokogiri using packaged libraries.
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing nokogiri (1.6.2.1), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.2.1'` succeeds before bundling.
@nodje
nodje commented May 15, 2014

Unfortunately, the problem still appear on a fresh Debian 7.5 fresh install and on an Ubuntu 14.04 fresh install as well:

$ vagrant plugin install vagrant-lxc --plugin-version 1.0.0.alpha.2
Installing the 'vagrant-lxc --version '1.0.0.alpha.2'' plugin. This can take a few minutes...
Building nokogiri using packaged libraries.
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing nokogiri (1.6.2.1), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.2.1'` succeeds before bundling.
@fredngo
fredngo commented May 15, 2014

The quick fix from arosenhagen worked for me on OSX 10.8.5

Quick fix: add s.add_dependency(%q, ["< 1.6.2"]) to ~/Applications/Vagrant/embedded/gems/specifications/vagrant-1.6.2.gemspec

@morrizon
Contributor

@nodje I had the same issue with a fresh installed ubuntu server 12.04. I also had installed libxml2 and libxslt as say in http://nokogiri.org/tutorials/installing_nokogiri.html

sudo apt-get install libxslt-dev libxml2-dev

When I checked the gem install command I got:

sudo /opt/vagrant/embedded/bin/gem install nokogiri -v '1.6.2.1'
...
libiconv is missing. please visit…
...

Also tried give all lib paths:

sudo /opt/vagrant/embedded/bin/gem install nokogiri -v '1.6.2.1' -- --with-xml2-lib=/usr/lib/x86_64-linux-gnu --with-xml2-include=/usr/include/libxml2 --with-xslt-lib=/usr/lib/x86_64-linux-gnu --with-xslt-include=/usr/include/libexslt --with-iconv-lib=/opt/vagrant/embedded/lib --with-iconv-include=/opt/vagrant/embedded/include
...
/opt/vagrant/embedded/lib/ruby/2.0.0/mkmf.rb:434:in `try_do': The compiler failed to generate an executable file. (RuntimeError)
You have to install development tools first.
...
Gem files will remain installed in /opt/vagrant/embedded/lib/ruby/gems/2.0.0/gems/nokogiri-1.6.2.1 for inspection.
Results logged to /opt/vagrant/embedded/lib/ruby/gems/2.0.0/gems/nokogiri-1.6.2.1/ext/nokogiri/gem_make.out

Last error gave me the answer. I needed the build tools (gcc, libc-dev, ...). I think the libiconv is integrated in libc. I installed build tools and the problem was gone.

sudo apt-get install build-essential

Now the plugins install process work like a charm :)

@nodje
nodje commented May 19, 2014

thanks @morrizon, works like a charm now!

apt-get install libxslt-dev libxml2-dev build-essential

It was my first time installing on fresh servers I guess, so I didn't realise some dependencies like these could be needed.

I'd be good to add this somewhere in the README

@blargity

This is still definitely happening for us on OS X 10.8.5.

@blargity

Ok, after about 2 days of banging our heads on the wall with it it magically worked this morning. I'm not sure what changed as we haven't done anything differently...

@thinkstylestudio

Hey @morrizon, do you know if this will fix the issue on a mac?

@maciejzgadzaj

Not really a proper solution, but downgrading Vagrant to 1.6.1 did help on OS X.

@floretan

I also had the same problem running vagrant 1.6.2 on OS X 10.9.2. A hint at a potential solution came when Homebrew complained about missing Command Line Tools. They can be installed using:

xcode-select --install

Afterwards, I was able to install the aws vagrant plugin without any issues. Apparently this is the OS X equivalent to the linux solution pointed out by @nodje.

@morrizon
Contributor

Hi @thinkstylestudio, maybe it has a similar issue on OS X. You can try to install missing tools as @floretan says. If you want to know the problem you must test the installation of Nokogiri using gem:

/Applications/Vagrant/embedded/bin/gem install nokogiri -v '1.6.2.1'
@patrickheeney patrickheeney referenced this issue in protobox/protobox May 21, 2014
Closed

Can't install protobox - nokogiri error #106

@Dzamir
Dzamir commented May 21, 2014

Same problem here, but

xcode-select --install

Always timeout

@reem
reem commented May 21, 2014

Having this problem too, but I definitely have the XCode CLI. Vagrant 1.6.2, nokigiri 1.6.2.1 and OS X 10.9.2.

@thinkstylestudio

Hi @morrizon I've run the command as you requested here is the output:
From what i can gather it's libxml2 that is simply missing. I'm a little unsure how to install that on a mac. Is it identical as linux?

Last login: Wed May 21 13:13:11 on ttys002
You have mail.
M:~ t$ /Applications/Vagrant/embedded/bin/gem install nokogiri -v '1.6.2.1'
Building native extensions.  This could take a while...
Building nokogiri using packaged libraries.
Building libxml2-2.8.0 for nokogiri with the following patches applied:
    - 0001-Fix-parser-local-buffers-size-problems.patch
    - 0002-Fix-entities-local-buffers-size-problems.patch
    - 0003-Fix-an-error-in-previous-commit.patch
    - 0004-Fix-potential-out-of-bound-access.patch
    - 0005-Detect-excessive-entities-expansion-upon-replacement.patch
    - 0006-Do-not-fetch-external-parsed-entities.patch
    - 0007-Enforce-XML_PARSER_EOF-state-handling-through-the-pa.patch
    - 0008-Improve-handling-of-xmlStopParser.patch
    - 0009-Fix-a-couple-of-return-without-value.patch
    - 0010-Keep-non-significant-blanks-node-in-HTML-parser.patch
    - 0011-Do-not-fetch-external-parameter-entities.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxml2.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install

However, note that nokogiri does not necessarily support all versions
of libxml2.

For example, libxml2-2.9.0 and higher are currently known to be broken
and thus unsupported by nokogiri, due to compatibility problems and
XPath optimization bugs.
************************************************************************
Building libxslt-1.1.28 for nokogiri with the following patches applied:
    - 0001-Adding-doc-update-related-to-1.1.28.patch
    - 0002-Fix-a-couple-of-places-where-f-printf-parameters-wer.patch
    - 0003-Initialize-pseudo-random-number-generator-with-curre.patch
    - 0004-EXSLT-function-str-replace-is-broken-as-is.patch
    - 0006-Fix-str-padding-to-work-with-UTF-8-strings.patch
    - 0007-Separate-function-for-predicate-matching-in-patterns.patch
    - 0008-Fix-direct-pattern-matching.patch
    - 0009-Fix-certain-patterns-with-predicates.patch
    - 0010-Fix-handling-of-UTF-8-strings-in-EXSLT-crypto-module.patch
    - 0013-Memory-leak-in-xsltCompileIdKeyPattern-error-path.patch
    - 0014-Fix-for-bug-436589.patch
    - 0015-Fix-mkdir-for-mingw.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxslt.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install
************************************************************************
ERROR:  Error installing nokogiri:
    ERROR: Failed to build gem native extension.

    /Applications/Vagrant/embedded/bin/ruby extconf.rb
Building nokogiri using packaged libraries.
checking for iconv.h... yes
checking for iconv_open() in iconv.h... no
checking for iconv_open() in -liconv... no
checking for libiconv_open() in iconv.h... no
checking for libiconv_open() in -liconv... yes
Building libxml2-2.8.0 for nokogiri with the following patches applied:
    - 0001-Fix-parser-local-buffers-size-problems.patch
    - 0002-Fix-entities-local-buffers-size-problems.patch
    - 0003-Fix-an-error-in-previous-commit.patch
    - 0004-Fix-potential-out-of-bound-access.patch
    - 0005-Detect-excessive-entities-expansion-upon-replacement.patch
    - 0006-Do-not-fetch-external-parsed-entities.patch
    - 0007-Enforce-XML_PARSER_EOF-state-handling-through-the-pa.patch
    - 0008-Improve-handling-of-xmlStopParser.patch
    - 0009-Fix-a-couple-of-return-without-value.patch
    - 0010-Keep-non-significant-blanks-node-in-HTML-parser.patch
    - 0011-Do-not-fetch-external-parameter-entities.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxml2.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install

However, note that nokogiri does not necessarily support all versions
of libxml2.

For example, libxml2-2.9.0 and higher are currently known to be broken
and thus unsupported by nokogiri, due to compatibility problems and
XPath optimization bugs.
************************************************************************
Extracting libxml2-2.8.0.tar.gz into tmp/x86_64-apple-darwin12.5.0/ports/libxml2/2.8.0... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxml2/0001-Fix-parser-local-buffers-size-problems.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxml2/0002-Fix-entities-local-buffers-size-problems.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxml2/0003-Fix-an-error-in-previous-commit.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxml2/0004-Fix-potential-out-of-bound-access.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxml2/0005-Detect-excessive-entities-expansion-upon-replacement.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxml2/0006-Do-not-fetch-external-parsed-entities.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxml2/0007-Enforce-XML_PARSER_EOF-state-handling-through-the-pa.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxml2/0008-Improve-handling-of-xmlStopParser.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxml2/0009-Fix-a-couple-of-return-without-value.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxml2/0010-Keep-non-significant-blanks-node-in-HTML-parser.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxml2/0011-Do-not-fetch-external-parameter-entities.patch...
Running 'patch' for libxml2 2.8.0... OK
Running 'configure' for libxml2 2.8.0... OK
Running 'compile' for libxml2 2.8.0... OK
Running 'install' for libxml2 2.8.0... OK
Activating libxml2 2.8.0 (from /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/x86_64-apple-darwin12.5.0/libxml2/2.8.0)...
Building libxslt-1.1.28 for nokogiri with the following patches applied:
    - 0001-Adding-doc-update-related-to-1.1.28.patch
    - 0002-Fix-a-couple-of-places-where-f-printf-parameters-wer.patch
    - 0003-Initialize-pseudo-random-number-generator-with-curre.patch
    - 0004-EXSLT-function-str-replace-is-broken-as-is.patch
    - 0006-Fix-str-padding-to-work-with-UTF-8-strings.patch
    - 0007-Separate-function-for-predicate-matching-in-patterns.patch
    - 0008-Fix-direct-pattern-matching.patch
    - 0009-Fix-certain-patterns-with-predicates.patch
    - 0010-Fix-handling-of-UTF-8-strings-in-EXSLT-crypto-module.patch
    - 0013-Memory-leak-in-xsltCompileIdKeyPattern-error-path.patch
    - 0014-Fix-for-bug-436589.patch
    - 0015-Fix-mkdir-for-mingw.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxslt.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install
************************************************************************
Extracting libxslt-1.1.28.tar.gz into tmp/x86_64-apple-darwin12.5.0/ports/libxslt/1.1.28... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxslt/0001-Adding-doc-update-related-to-1.1.28.patch...
Running 'patch' for libxslt 1.1.28... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxslt/0002-Fix-a-couple-of-places-where-f-printf-parameters-wer.patch...
Running 'patch' for libxslt 1.1.28... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxslt/0003-Initialize-pseudo-random-number-generator-with-curre.patch...
Running 'patch' for libxslt 1.1.28... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxslt/0004-EXSLT-function-str-replace-is-broken-as-is.patch...
Running 'patch' for libxslt 1.1.28... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxslt/0006-Fix-str-padding-to-work-with-UTF-8-strings.patch...
Running 'patch' for libxslt 1.1.28... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxslt/0007-Separate-function-for-predicate-matching-in-patterns.patch...
Running 'patch' for libxslt 1.1.28... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxslt/0008-Fix-direct-pattern-matching.patch...
Running 'patch' for libxslt 1.1.28... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxslt/0009-Fix-certain-patterns-with-predicates.patch...
Running 'patch' for libxslt 1.1.28... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxslt/0010-Fix-handling-of-UTF-8-strings-in-EXSLT-crypto-module.patch...
Running 'patch' for libxslt 1.1.28... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxslt/0013-Memory-leak-in-xsltCompileIdKeyPattern-error-path.patch...
Running 'patch' for libxslt 1.1.28... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxslt/0014-Fix-for-bug-436589.patch...
Running 'patch' for libxslt 1.1.28... OK
Running patch with /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/patches/libxslt/0015-Fix-mkdir-for-mingw.patch...
Running 'patch' for libxslt 1.1.28... OK
Running 'configure' for libxslt 1.1.28... OK
Running 'compile' for libxslt 1.1.28... OK
Running 'install' for libxslt 1.1.28... OK
Activating libxslt 1.1.28 (from /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ports/x86_64-apple-darwin12.5.0/libxslt/1.1.28)...
checking for main() in -llzma... no
checking for xmlParseDoc() in libxml/parser.h... no
checking for xmlParseDoc() in -lxml2... no
checking for xmlParseDoc() in -llibxml2... no
-----
libxml2 is missing.  please visit http://nokogiri.org/tutorials/installing_nokogiri.html for help with installing dependencies.
-----
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
    --with-opt-dir
    --with-opt-include
    --without-opt-include=${opt-dir}/include
    --with-opt-lib
    --without-opt-lib=${opt-dir}/lib
    --with-make-prog
    --without-make-prog
    --srcdir=.
    --curdir
    --ruby=/Applications/Vagrant/embedded/bin/ruby
    --help
    --clean
    --use-system-libraries
    --enable-static
    --disable-static
    --with-zlib-dir
    --without-zlib-dir
    --with-zlib-include
    --without-zlib-include=${zlib-dir}/include
    --with-zlib-lib
    --without-zlib-lib=${zlib-dir}/lib
    --enable-cross-build
    --disable-cross-build
    --with-xml2lib
    --without-xml2lib
    --with-libxml2lib
    --without-libxml2lib


Gem files will remain installed in /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1 for inspection.
Results logged to /Users/t/.rvm/gems/ruby-1.9.3-p448/gems/nokogiri-1.6.2.1/ext/nokogiri/gem_make.out
M:~ t$

@morrizon
Contributor

@thinkstylestudio the output tells you the problem:

libxml2 is missing.  please visit http://nokogiri.org/tutorials/installing_nokogiri.html for help with installing dependencies.

If you visit the link http://nokogiri.org/tutorials/installing_nokogiri.html you can read possibles solutions but you need to use macports or homebrew.

@cduv
cduv commented May 21, 2014

Got the same problem.

Enviroment:
Mac OSX last version
RVM last update
Brew updated
Vagrant 1.6.2

I installed as recommended:

xcode-select --install

I was able to install libiconv:

wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.13.1.tar.gz
tar xvfz libiconv-1.13.1.tar.gz
cd libiconv-1.13.1
./configure --prefix=/usr/local/Cellar/libiconv/1.13.1
make
sudo make install

I linked it to brew, with:

 brew link libiconv

Checked brew with:

brew update
brew doctor

I installed the gem nokogiri with:

gem install nokogiri -v '1.6.2.1' -- --with-iconv-dir=/usr/local/Cellar/libiconv/1.13.1  

I can see the gem installed with gem list and the correct version.
image

When I try to install vagrant omnibus plugin with:

vagrant plugin install vagrant-omnibus

I get the error:
image

Tried:

export NOKOGIRI_USE_SYSTEM_LIBRARIES=true.
vagrant plugin update

With the same result.

Any idea how to advance?

Thanks in advance

UPDATE:

Solved.
There was a small comment in brew doctor about the path. Added the correct path order to .bashrc and worked.

Excuses for the comment.

image

@lkwdwrd
lkwdwrd commented May 21, 2014

Finally got mine working with the fix from @fredngo Adding the dependency declaration around line 20.

Running OS X 1.8.5 with Vagrant 1.6.2, plugins now install fine.

@metaColin

I'm just going to use Vagrant 1.5.4 until this get's sorted out...

I don't know why 1.6.x is considered stable with such a crippling issue.
I don't know why this issue is closed.

@metaColin metaColin referenced this issue in Varying-Vagrant-Vagrants/VVV May 22, 2014
Closed

vagrant up fails `initialize': Permission denied #362

@knu
knu commented May 22, 2014

You guys really should read the error message and look into mkmf.log to see what's going on.

@Dzamir
Dzamir commented May 22, 2014

I'm installing vagrant because I don't want to work with Mac OS Command Line utilities

@metaColin

I agree with Dzamir.

Personally I consider this to be an open issue, because a web dev (especially one just learning vagrant after years of MAMP/XAMPP) should not need to go diving into command line error logs just to install a basic plugin. It's not a matter of laziness it's a matter of priorities, and credibility of the software.

@mitchellh
Owner

Really none of the solutions should require using system libraries, brew installing anything. Command line tools are required for any plugins because some plugins require a compiler (C extensions) and Vagrant can't ship with a full C toolchain.

Ultimately this seems to be caused by Nokogiri upstream changing some of the way they compile. It should be fully self-contained to the installer though. i.e. Install OS X, Install Vagrant, Install XCode, and it should work.

If someone can give me a solid repro or solution, I'd look into it, but on my end on the few macs we have around here things just work.

@sgerrand

@mitchellh I can reproduce this on a new MacBook Air running OS X (v 10.9.2). Looking at the output from gem install nokogiri and vagrant plugin install below, it appears that Nokogiri is emitting a non-zero exit code during the compilation step for libxml2, which Vagrant picks up during the plugin installation and interprets as a failure. Maybe it's just the warning message emitted by Nokogiri during the gem installation. I'll look into this further. More details follow.

$ vagrant plugin install vagrant-omnibus
Installing the 'vagrant-omnibus' plugin. This can take a few minutes...
Building nokogiri using packaged libraries.
Building libxml2-2.8.0 for nokogiri with the following patches applied:
        - 0001-Fix-parser-local-buffers-size-problems.patch
        - 0002-Fix-entities-local-buffers-size-problems.patch
        - 0003-Fix-an-error-in-previous-commit.patch
        - 0004-Fix-potential-out-of-bound-access.patch
        - 0005-Detect-excessive-entities-expansion-upon-replacement.patch
        - 0006-Do-not-fetch-external-parsed-entities.patch
        - 0007-Enforce-XML_PARSER_EOF-state-handling-through-the-pa.patch
        - 0008-Improve-handling-of-xmlStopParser.patch
        - 0009-Fix-a-couple-of-return-without-value.patch
        - 0010-Keep-non-significant-blanks-node-in-HTML-parser.patch
        - 0011-Do-not-fetch-external-parameter-entities.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxml2.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install

However, note that nokogiri does not necessarily support all versions
of libxml2.

For example, libxml2-2.9.0 and higher are currently known to be broken
and thus unsupported by nokogiri, due to compatibility problems and
XPath optimization bugs.
************************************************************************
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing nokogiri (1.6.2.1), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.2.1'` succeeds before bundling.
$ gem install nokogiri -v '1.6.2.1'
Building native extensions.  This could take a while...
Building nokogiri using packaged libraries.
Building libxml2-2.8.0 for nokogiri with the following patches applied:
        - 0001-Fix-parser-local-buffers-size-problems.patch
        - 0002-Fix-entities-local-buffers-size-problems.patch
        - 0003-Fix-an-error-in-previous-commit.patch
        - 0004-Fix-potential-out-of-bound-access.patch
        - 0005-Detect-excessive-entities-expansion-upon-replacement.patch
        - 0006-Do-not-fetch-external-parsed-entities.patch
        - 0007-Enforce-XML_PARSER_EOF-state-handling-through-the-pa.patch
        - 0008-Improve-handling-of-xmlStopParser.patch
        - 0009-Fix-a-couple-of-return-without-value.patch
        - 0010-Keep-non-significant-blanks-node-in-HTML-parser.patch
        - 0011-Do-not-fetch-external-parameter-entities.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxml2.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install

However, note that nokogiri does not necessarily support all versions
of libxml2.

For example, libxml2-2.9.0 and higher are currently known to be broken
and thus unsupported by nokogiri, due to compatibility problems and
XPath optimization bugs.
************************************************************************
Building libxslt-1.1.28 for nokogiri with the following patches applied:
        - 0001-Adding-doc-update-related-to-1.1.28.patch
        - 0002-Fix-a-couple-of-places-where-f-printf-parameters-wer.patch
        - 0003-Initialize-pseudo-random-number-generator-with-curre.patch
        - 0004-EXSLT-function-str-replace-is-broken-as-is.patch
        - 0006-Fix-str-padding-to-work-with-UTF-8-strings.patch
        - 0007-Separate-function-for-predicate-matching-in-patterns.patch
        - 0008-Fix-direct-pattern-matching.patch
        - 0009-Fix-certain-patterns-with-predicates.patch
        - 0010-Fix-handling-of-UTF-8-strings-in-EXSLT-crypto-module.patch
        - 0013-Memory-leak-in-xsltCompileIdKeyPattern-error-path.patch
        - 0014-Fix-for-bug-436589.patch
        - 0015-Fix-mkdir-for-mingw.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxslt.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install
************************************************************************
Successfully installed nokogiri-1.6.2.1
1 gem installed
Installing ri documentation for nokogiri-1.6.2.1...
Building YARD (yri) index for nokogiri-1.6.2.1...
Installing RDoc documentation for nokogiri-1.6.2.1...
$ vagrant plugin install vagrant-omnibus
Installing the 'vagrant-omnibus' plugin. This can take a few minutes...
Building nokogiri using packaged libraries.
Building libxml2-2.8.0 for nokogiri with the following patches applied:
        - 0001-Fix-parser-local-buffers-size-problems.patch
        - 0002-Fix-entities-local-buffers-size-problems.patch
        - 0003-Fix-an-error-in-previous-commit.patch
        - 0004-Fix-potential-out-of-bound-access.patch
        - 0005-Detect-excessive-entities-expansion-upon-replacement.patch
        - 0006-Do-not-fetch-external-parsed-entities.patch
        - 0007-Enforce-XML_PARSER_EOF-state-handling-through-the-pa.patch
        - 0008-Improve-handling-of-xmlStopParser.patch
        - 0009-Fix-a-couple-of-return-without-value.patch
        - 0010-Keep-non-significant-blanks-node-in-HTML-parser.patch
        - 0011-Do-not-fetch-external-parameter-entities.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxml2.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install

However, note that nokogiri does not necessarily support all versions
of libxml2.

For example, libxml2-2.9.0 and higher are currently known to be broken
and thus unsupported by nokogiri, due to compatibility problems and
XPath optimization bugs.
************************************************************************
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing nokogiri (1.6.2.1), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.2.1'` succeeds before bundling.
@mitchellh
Owner

Yeah, the red flag for me with regards to the Nokogiri stuff is that it worked before Nokogiri released a new version. I don't have the hard evidence yet but my gut tells me if nothing changed, then suddenly broke, it was caused by the thing that changed. In this case, it was Nokogiri.

Our solution going forward for this dep might be to just lock it down since we can't trust it to not break things in spectacular ways, unfortunately.

@sgerrand

Sounds like a good approach.

@geauxvirtual

When examining the gems included with the Vagrant install, nokogiri-1.6.1 is included. Pinning vagrant to use this included gem is a good solution.

This solution has worked on my laptop, and other laptops in our team that were running the same issue with trying to compile nokogiri-1.6.2.1.

@sklise
sklise commented May 22, 2014

@geauxvirtual could you detail how to pin Vagrant to that version of nokogiri?

@metaColin

From #3769
To pin vagrant to a specific version of nokigiri:
Quick fix: add s.add_dependency(%q, ["< 1.6.2"]) to ~/Applications/Vagrant/embedded/gems/specifications/vagrant-1.6.2.gemspec

@sklise
sklise commented May 22, 2014

Thanks @colinsteinmann

@tmatilai
Collaborator

The downside of strict pinning is that it will increase compatibility issues with vagrant-aws and other plugins which have transitive dependency on nokogiri.

That gem should really be bundled into Ruby core. It causes soo much trouble...

@josegonzalez

@tmatilai as an aside, people should maybe not use xml and start using json ;)

@lmorchard lmorchard referenced this issue in mozilla/kuma May 27, 2014
Merged

Cleans up the Vagrant setup. #2427

@jayunit100

is there a current workaround to this which has been agreed on ? Downgrading vagrant seems like a not-so-good idea to me - i love the new features with releases and would like to keep using the latest

@JacobHayes

I hacked together a script to do a full installation with @colinsteinmann's fix. It's worked for me well on 10.9. Haven't had a chance to try on 10.8, but should handle it.

EDIT: Does work on 10.8

@jallits
jallits commented May 29, 2014

The most pertinent line in @JacobHayes script which can be ran from Terminal/iTerm is the following:

sudo perl -p -i -e "s/^end$/\n  s.add_dependency\(\%q\<nokogiri\>, \[\"< 1.6.2\"\]\)\nend/" "/Applications/Vagrant/embedded/gems/specifications/vagrant-1.6.2.gemspec"

Then all the user has to do is vagrant plugin install <name> as specified in the documentation.

Of course, this makes the assumption that the User has already installed Vagrant and Command Line Tools. However, I think that is more often the case than a full blown install script.

@JacobHayes

True, but the CLT and Vagrant installs are idempotent (unless it's an upgrade/downgrade) so it's no hassle other than time. The script was made to get my coworkers and I up to date with ease, so I tried to make few assumptions. ;)

Also, for those of you trying to install vagrant-omnibus and vagrant-berkshelf, you have to install them in the correct order as vagrant-omnibus locks vagrant-berkshelf at 1.3.7 (whose troubles escape me now), so vagrant-berkshelf has to be installed after omnibus with the explicit version of 2.0.1.

@tmatilai
Collaborator

@JacobHayes As vagrant-berkshelf 2.0.1 has pre-release dependencies, you have to lock it's version to avoid it being downgraded:

vagrant plugin install vagrant-berkshelf --plugin-version '>= 2.0.1'

Dropping the >= means that the version restriction is only used when installing it, but it's not stored.

@lkwdwrd lkwdwrd referenced this issue in Varying-Vagrant-Vagrants/VVV Jun 3, 2014
Open

SSH Timeout - Cant connect via ssh no matter what #375

@glenjamin

We're seeing this issue when doing vagrant install vagrant-omnibus, which has no dependencies.

It would appear from https://github.com/mitchellh/vagrant/blob/master/lib/vagrant/bundler.rb that the embedded vagrant gems are not being added to the gem path when installing a plugin, causing the bundler to attempt to install all of these gems elsewhere when installing a plugin.

If i'm understanding correctly, nokogiri 1.6 already comes with vagrant - so installing a pure-ruby plugin should not require any compiler toolchain.

@hblanks
hblanks commented Jun 3, 2014

We've elected to just downgrade to Vagrant 1.5 until this gets fixed.

@fnordfish fnordfish referenced this issue in dotless-de/vagrant-vbguest Jun 26, 2014
Closed

vagrant plugin install vagrant-vbguest fails #126

@Tyrael
Tyrael commented Jul 22, 2014

our problems resolved after there recent nokogiri 1.6.3 release, which contains this fix affecting mac users:
sparklemotion/nokogiri#1101

@jonbuffington

@Tyrael Looks like 1.6.3.1 breaks the process:

$ vagrant plugin update vagrant-vmware-fusion
Updating plugins: vagrant-vmware-fusion. This may take a few minutes...
Building nokogiri using packaged libraries.
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing nokogiri (1.6.3.1), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.3.1'` succeeds before bundling.
@Tyrael
Tyrael commented Jul 22, 2014

from https://github.com/sparklemotion/nokogiri/commits/master it seems that there are some issues detecting the support of the compiler flag:
sparklemotion/nokogiri@3c3d34f

@nlfiedler

I second @jonbuffington 's comment, looks like this problem is back again in Vagrant 1.6.3. I'm happy to attach logs or whatever diagnostic is needed, but first I need to know where to locate those. BTW, I was able to install nokogiri 1.6.3.1 without incident (using Ruby 2.1.2).

@pierot
pierot commented Jul 23, 2014

It reappeared in 1.6.3, Ruby 2.0.0.
Using the trick described earlier

$ export NOKOGIRI_USE_SYSTEM_LIBRARIES=true
$ vagrant plugin install vagrant-vbguest

You can, again, work around this.

@jonbuffington

@pierot I am unable to update vagrant-vmware-fusion using the work around.

$ export NOKOGIRI_USE_SYSTEM_LIBRARIES=true
$ vagrant plugin update vagrant-vmware-fusion
Updating plugins: vagrant-vmware-fusion. This may take a few minutes...
Building nokogiri using system libraries.
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing nokogiri (1.6.3.1), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.3.1'` succeeds before bundling.

FWIW, I am on OS X 10.9.4 and using Xcode 5.1.1 without the command line tools installed.

$ ruby -v
ruby 2.0.0p451 (2014-02-24 revision 45167) [universal.x86_64-darwin13]
@Tyrael
Tyrael commented Jul 23, 2014

I don't think that you can install the nokogiri gem without a c compiler, so you either need the command line tools installed or install gcc or clang from some other source.
NOKOGIRI_USE_SYSTEM_LIBRARIES only controls whether nokogiri should download the libxml/libxslt/libiconv source tarballs, patch them, and compile the libs to be available for linking to nokogiri, or to use the libs already installed on the system.
if you don't wanna compile nokogiri, you can use the 1.6.1 version shipped with vagrant through adding an explicit dependency to the vagrant gemspec file as mentioned in #3769 (comment)

@sgerrand

@jonbuffington: What are the results of running gem install nokogiri -v '1.6.3.1'?

@jonbuffington

@Tyrael Right. Xcode symlinks the developer tools:

$ cc --version
Apple LLVM version 6.0 (clang-600.0.41.2) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin14.0.0
Thread model: posix

The command line tools are an optional subset package if you do not want to install the Xcode IDE.

Thanks for the reminder about editing the vagrant gemspec. I was hoping Hashicorp would have resolved this for the commercial plugin, vagrant-vmware-fusion.

@Tyrael
Tyrael commented Jul 23, 2014

@jonbuffington oh, so you are using the pre-release Yosemite? I think there is not much the vagrant devs can do about a problem in a gem which isn't really a immidiate dependency of vagrant a transient dep for some plugin.
apart from rewriting vagrant to go ofc. :P

@nlfiedler

@Tyrael 10.9 is not Yosemite, it's Mavericks, a public release of Mac OS X. That being said, the Mac is a mess of OS versions, compilers, libraries, Ruby versions, gem versions, etc. I know the pain firsthand. I wholeheartedly support rewriting this entire project in Go.

@pierot It seems that using the system libraries would require having precisely the patched version of libxml2 that nokogiri requires. I tried several flags to try to convince it to use the previously built libraries in /Applications/Vagrant/embedded, but it kept telling me I need version 2.6.21 or later (is that even accurate?).

@jonbuffington

@Tyrael Good catch. I was rushing to a meeting and pasted from the wrong system. I am using stock OS X 10.9.4 with Xcode 5.1.1:

$ cc --version
Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin13.3.0
Thread model: posix
@jonbuffington

@sgerrand I am getting the same issue as @nlfiedler

$ export NOKOGIRI_USE_SYSTEM_LIBRARIES=true
$ gem install nokogiri -v '1.6.3.1'
Fetching: mini_portile-0.6.0.gem (100%)
Successfully installed mini_portile-0.6.0
Building native extensions.  This could take a while...
Building nokogiri using system libraries.
ERROR:  Error installing nokogiri:
    ERROR: Failed to build gem native extension.

    /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby extconf.rb
Building nokogiri using system libraries.
libxml2 version 2.6.21 or later is required!
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
    --with-opt-dir
    --without-opt-dir
    --with-opt-include
    --without-opt-include=${opt-dir}/include
    --with-opt-lib
    --without-opt-lib=${opt-dir}/lib
    --with-make-prog
    --without-make-prog
    --srcdir=.
    --curdir
    --ruby=/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby
    --help
    --clean
    --use-system-libraries=true
    --with-zlib-dir
    --without-zlib-dir
    --with-zlib-include
    --without-zlib-include=${zlib-dir}/include
    --with-zlib-lib
    --without-zlib-lib=${zlib-dir}/lib
    --with-xml2-dir
    --without-xml2-dir
    --with-xml2-include
    --without-xml2-include=${xml2-dir}/include
    --with-xml2-lib
    --without-xml2-lib=${xml2-dir}/lib
    --with-libxml-2.0-config
    --without-libxml-2.0-config
    --with-pkg-config
    --without-pkg-config
    --with-xslt-dir
    --without-xslt-dir
    --with-xslt-include
    --without-xslt-include=${xslt-dir}/include
    --with-xslt-lib
    --without-xslt-lib=${xslt-dir}/lib
    --with-libxslt-config
    --without-libxslt-config
    --with-exslt-dir
    --without-exslt-dir
    --with-exslt-include
    --without-exslt-include=${exslt-dir}/include
    --with-exslt-lib
    --without-exslt-lib=${exslt-dir}/lib
    --with-libexslt-config
    --without-libexslt-config
@sgerrand

Maybe I'm missing something here, but I don't see why you're trying to use
the system level libraries during a installation of Nokogiri when you don't
have the required version of libxml2 etc installed. Nokogiri comes with the
source code for both libxslt and libxml2 - have you tried installing it
with that?

@nlfiedler

@sgerrand I have tried everything short of voodoo to work around this problem. No, it certainly doesn't make sense to use the system libraries when nokogiri comes with exactly the set it needs, but it is repeatedly suggested, so out of desperation I try it. I am reporting what I see. If you have other advice on this matter, I'm all ears.

Current status: system-wide, I have Ruby 2.1.2p95, gem 2.2.2, and nokogiri 1.6.3.1 installed, with little trouble (using Mac OS X 10.7.5, Xcode 4.6.3, gcc 4.2.1). As for Vagrant 1.6.3, a series of disappointing failures while attempting to install plugins.

BTW, could someone please reopen this issue? Or a new one? Either way, it is clearly a problem for multiple users.

@sgerrand

@sgerrand I have tried everything short of voodoo to work around this
problem. No, it certainly doesn't make sense to use the system libraries
when nokogiri comes with exactly the set it needs, but it is repeatedly
suggested, so out of desperation I try it. I am reporting what I see. If
you have other advice on this matter, I'm all ears.

Current status: system-wide, I have Ruby 2.1.2p95, gem 2.2.2, and
nokogiri 1.6.3.1 installed, with little trouble (using Mac OS X 10.7.5,
Xcode 4.6.3, gcc 4.2.1). As for Vagrant 1.6.3, a series of disappointing
failures while attempting to install plugins.

Fair enough. I don't have access to a computer running OS X v10.7, but will
see what I can find at work tomorrow.

@sneal
Collaborator
sneal commented Jul 24, 2014

I've been toying around with removing the nokogiri dependency from the winrm gem, which Vagrant uses for Windows guests. This won't fix the underlying issue, but it reduces the likelihood of plugins using incompatible Nokogiri versions.

@Tyrael
Tyrael commented Jul 24, 2014

@nlfiedler x86_64-apple-darwin14.0.0 and clang-600.0.41.2 is definitely yosemite, @jonbuffington mentioned later that he posted the cc --version from the wrong system.
@jonbuffington just to make sure: so the nokogiri install fails from vagrant plugin install and it also fails when you try installing manually via gem install, with and without NOKOGIRI_USE_SYSTEM_LIBRARIES?
For the record I'm able to install the latest nokogiri gem (letting it to compile the libraries for itself), and as far as I can tell we are using the same version of everything, the only difference is that you don't have the command line tools installed while I do have it.
Could you show me the output of
gem install --verbose nokogiri -v '1.6.3.1'
without any NOKOGIRI_USE_SYSTEM_LIBRARIES?

@v1k0d3n
v1k0d3n commented Jul 24, 2014

why does this keep getting closed? i'm running into this issue even with vagrant 1.6.3. has there been an approved and proven work around? people are buying the provider package thinking things will work out of the box.

@Tyrael
Tyrael commented Jul 24, 2014

"why does this keep getting closed?"
it is most likely not a vagrant issue per see:
some vagrant plugins depend on nokogiri which can be problematic to install (it is a C extension, depending on multiple libraries with specific versions).
while vagrant itself doesn't depend on nokogiri, it by default ships a pre-built version of it,
unfortunately it doesn't help much, because different vagrant versions can come with different nokogiri versions, and the vagrant plugins have to support multiple versions, so adding an explicit nokogiri version dependency to the vagrant plugin is not a realistic approach.
which means that users using vagrant plugins having immediate or transitive dependency to nokogiri has to be able to install nokogiri to use those plugins.
as mentioned above, nokogiri can be a tricky beast to install, and it has a couple of build issues recently, mostly on the Mac OS platform.
We can help you figure out your build problems here, or if you want a quick workaround, then check out my previous comment(#3769 (comment)) about adding an explicit dependency to the vagrant.gemspec file which forces bundler to accept the pre-build nokogiri version as the most satisfying, so it won't try to install (and fail) the latest one.

@sciurus
Contributor
sciurus commented Jul 31, 2014

@Tyrael thanks for digging into this. Could you open a new issue based on your investigation?

@Tyrael
Tyrael commented Aug 1, 2014

@sciurus sure!

@jokeyrhyme

OSX 10.10 public beta, Xcode 6 beta, Vagrant 1.6.3

I didn't install the Xcode command line tools at all, I simply ran:

xcode-select -s /Applications/Xcode6-Beta3.app/

I got the same error as reported by others here when trying to install a Vagrant plugin (the VMware Fusion plugin, actually). I ran the following to investigate further as suggested by the vagrant error output:

 sudo /Applications/Vagrant/embedded/bin/gem install nokogiri -v '1.6.3.1'

libiconv is missing. please visit http://nokogiri.org/tutorials/installing_nokogiri.html for help with installing dependencies.

I installed Homebrew to /opt/homebrew, installed libiconv with brew install homebrew/dupes/libiconv, then confirmed that I could install nokogiri with Vagrant's embedded Ruby (which completed without errors):

sudo ./gem install nokogiri -v '1.6.3.1' -- --with-xml2-include=/opt/homebrew/opt/libxml2/include/libxml2 --with-xml2-lib=/opt/homebrew/opt/libxml2/lib --with-xslt-dir=/opt/homebrew/opt/libxslt --with-iconv-include=/opt/homebrew/opt/libiconv/include --with-iconv-lib=/opt/homebrew/opt/libiconv/lib

I was still prevented from installed the vagrant plugin at this stage. I feel like I'm so close. It seems like all I need to do is copy the new embedded build of nokogiri somewhere.

@jokeyrhyme

Just tried the following, which causes no errors but does not fix plugin installation:

bundle config build.nokogiri -- --with-xml2-include=/opt/homebrew/opt/libxml2/include/libxml2 --with-xml2-lib=/opt/homebrew/opt/libxml2/lib --with-xslt-dir=/opt/homebrew/opt/libxslt --with-iconv-include=/opt/homebrew/opt/libiconv/include --with-iconv-lib=/opt/homebrew/opt/libiconv/lib

I tried to repeat this with the embedded bundle, but I get the following error:

zsh: ./bundle: bad interpreter: /Users/administrator/bamboo-builds/BUILD-VG-CORE-OSX-209/tmp.y: no such file or directory

@jokeyrhyme

Hooray, found what I needed to copy:

cp /Applications/Vagrant/embedded/lib/ruby/gems/2.0.0/specifications/nokogiri-1.6.3.1.gemspec ~/.vagrant.d/gems/specifications
cp -a /Applications/Vagrant/embedded/lib/ruby/gems/2.0.0/gems/nokogiri-1.6.3.1 ~/.vagrant.d/gems/gems
cp /Applications/Vagrant/embedded/lib/ruby/gems/2.0.0/build_info/nokogiri-1.6.3.1.info ~/.vagrant.d/gems/build_info
cp /Applications/Vagrant/embedded/lib/ruby/gems/2.0.0/cache/nokogiri-1.6.3.1.gem ~/.vagrant.d/gems/cache
cp -a /Applications/Vagrant/embedded/lib/ruby/gems/2.0.0/doc/nokogiri-1.6.3.1 ~/.vagrant.d/gems/doc

Vagrant plugins install just fine now. :D <3

@Tyrael
Tyrael commented Aug 1, 2014

I think we should start telling people to not listen to bundler when a gem installation fails and bundler suggest to gem install the gem manually.
vagrant ships it's own embedded ruby/rubygems and it calls the bundler library through some heavy bootstrapping so if something fails, manually gem installing won't fix the problem not provide much help for figuring out why did the vagrant gem installation not succeed originally.

@jokeyrhyme yeah, manually copying the missing gems into the ~/.vagrant.d gems directory will allow you to proceed with the plugin installation, but you could also just fix your environment to be able to install the gem when installed via vagrant plugin install or simply force vagrant to use the embedded nokogiri version (1.6.2.1).

@jayunit100

Just to reiterate, because i think alot of folks will need this workaround.... I think its alot easier to just do what was suggested earlier, modifying the line: s.add_dependency(%q<nokogiri>, ["= 1.6.2.1"]) . That works fine on macs. And i think other OS's dont have this issue.

@sindarina

So, having run into this problem with the Vagrant 1.6.3 package on OS X 10.9.4, while trying to install the $79 VMware plugin that Hashicorp sells ... can someone explain why the dependency workaround isn't included in that install by default? Because right now, installing the current Vagrant package and then following the instructions one receives after ordering the VMware plugin just ends in failure.

Something I'd expect Hashicorp to pay a bit more attention to, really.

@bjouhier
bjouhier commented Aug 4, 2014

@jayunit100
Many thanks. This solved my problem.

I'm making my first steps with Vagrant today. Looks really cool but this kind of problem makes the experience really miserable for a newcomer. I don't care whether the problem is Vagrant, nokogiri, ruby or command line tools, it should just work, and it does not.

@sindarina

@bjouhier I ended up ripping it out and reinstalling it with VirtualBox instead, because after solving this problem I ran into various issues with the VMware plugin. As the VirtualBox provider is part of the default install it doesn't run into this problem, or any of the VMware issues, and it actually does just work.

@bjouhier
bjouhier commented Aug 4, 2014

@sindarina I did not have the problem. I'm using VirtualBox on OSX Maverick

@patrickaroo

I'm also having an issue with this, installing Wirbelsturm. I just want to register the issue is alive and well.

An error occurred while installing nokogiri (1.6.3.1), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.3.1'` succeeds before bundling.

I did attempt to install it separately as suggested, with the same output.

@tapsboy
tapsboy commented Aug 5, 2014

@jonbuffington Thanks your solution worked for me.

I am on OS X 10.9.4 and Vagrant 1.6.3

@mhahn
Contributor
mhahn commented Aug 6, 2014

this was a huge obstacle for a lot of members of my team. this really discourages using vagrant plugins, which is unfortunate given how much they can do.

is anything being done to avoid something like this in the future?

@ajpolt
ajpolt commented Aug 7, 2014

The steps listed by @jallits solved my problem.

@rabe69
rabe69 commented Aug 7, 2014

In my case the nokogiri folders and there files located in ~/vagrant.d/gems/gems had wrong rights (user/group).

After changing them to username:staff (username = your console user name) it works like a charm.

@sethrosenblum

On mavericks, I was able to get this working by uninstalling nokogiri everywhere, switching rvm to use system ruby, and then reinstalling with:

sudo /Applications/Vagrant/embedded/bin/gem install nokogiri -- --use-system-libraries=true --with-xml2-include=/usr/include/libxml2

Then, when installing the vagrant-omnibus plugin there was no problem.

@daveferrara1

I couldn't get a Vagrant plugin to install relating to a Nokogiri error.

I used xcode-select --install and sudo gem install nokogiri

Probably don't need more than xcode-select --install and sudo gem install nokogiri on

Mavericks. Awesome!

@mitar
mitar commented Aug 13, 2014

I don't want to use sudo.

@daveferrara1

You may not have to. From the terminal I couldn¹t get it to go without sudo.

@gerhard gerhard referenced this issue in cppforlife/vagrant-bosh Aug 13, 2014
Closed

vagrant v1.6.3 & nokogiri v1.6.3.1 incompatibility #2

@gerhard
gerhard commented Aug 13, 2014

Even though I have nokogiri 1.6.3.1 installed in vagrant 1.6.3, vagrant plugin is still not happy:

/Applications/Vagrant/bin/vagrant -v
Vagrant 1.6.3

/Applications/Vagrant/embedded/bin/gem list | grep nokogiri
nokogiri (1.6.3.1)

/Applications/Vagrant/bin/vagrant plugin install vagrant-bosh
Installing the 'vagrant-bosh' plugin. This can take a few minutes...
Building nokogiri using packaged libraries.
Building libxml2-2.8.0 for nokogiri with the following patches applied:
    - 0001-Fix-parser-local-buffers-size-problems.patch
    - 0002-Fix-entities-local-buffers-size-problems.patch
    - 0003-Fix-an-error-in-previous-commit.patch
    - 0004-Fix-potential-out-of-bound-access.patch
    - 0005-Detect-excessive-entities-expansion-upon-replacement.patch
    - 0006-Do-not-fetch-external-parsed-entities.patch
    - 0007-Enforce-XML_PARSER_EOF-state-handling-through-the-pa.patch
    - 0008-Improve-handling-of-xmlStopParser.patch
    - 0009-Fix-a-couple-of-return-without-value.patch
    - 0010-Keep-non-significant-blanks-node-in-HTML-parser.patch
    - 0011-Do-not-fetch-external-parameter-entities.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxml2.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install

However, note that nokogiri does not necessarily support all versions
of libxml2.

For example, libxml2-2.9.0 and higher are currently known to be broken
and thus unsupported by nokogiri, due to compatibility problems and
XPath optimization bugs.
************************************************************************
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing nokogiri (1.6.3.1), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.3.1'` succeeds before bundling.
@patrickheeney

Same issue. @mitchellh Why is this closed when so many people are having issues?

@ctracey
ctracey commented Aug 15, 2014

My solution was to edit /Applications/Vagrant/embedded/gems/specifications/vagrant-1.6.2.gemspec setting the nokogiri gem version to 1.6.1

@davidfitch

I'm running Vagrant 1.6.3, and had Nokogiri 1.6.3.1 installed, on Mavericks 10.9.4 which was failing to install the vagrant-omnibus plugin in the same way described by others.

The solution is to force Vagrant to use Nokogiri 1.6.2.1: edit /Applications/Vagrant/embedded/gems/specifications/vagrant-1.6.3.gemspec and add .add_dependency(%q, ["= 1.6.2.1"])

@joshuakarjala

Please solve this. This still fails on Mavericks. There might be manual workarounds but that doesn't help when some non-devs on the team need this plugin.

@mitchellh
Owner

The solution that has been working for folks has been to completely uninstall/reinstall Vagrant. Unfortunate. I'm still interested in a way to fix this that doesn't involve that but I'm not sure I see the solution yet above...

@Tyrael
Tyrael commented Aug 20, 2014

@mitchellh adding an explicit nokogiri dependency to the bundled version to vagrant gemspec solves the issue for most people.
but I think it would be better to fix the issue that vagrant plugin install tries to install/upgrade gems when there are no dependency for them by the plugin installed(reported as #4278 and #4194).

@metaColin

Why is this issue closed?

This thread shows that it is a irksome and ongoing issue. Personally I haven't updated since 1.5.4 for fear it might break again, and I'll loose valuable time troubleshooting a fix.

Vagrant is amazing because it creates a platform agnostic dev environment, but it kinda defeats that purpose if my whole team can't even reliably get it up and running.

I love Vagrant and want to keep using it, but some members on of my team have abandoned it entirely because of this...

@mitchellh
Owner

@colinsteinmann This was caused by an upstream issue, and there is a reliable workaround. As @Tyrael said a better fix would be to fix the other issues that are still open.

@metaColin

@mitchellh cool. I will eagerly watch #4194 for a resolution.

Our team initially moved to Vagrant for environment consistency, but that only makes sense as long as it's faster to setup Vagrant than it is to manually reproduce an environment using something like Ampps.

@patrickheeney

@mitchellh That is not true. This thread is filled with people trying different things to try and get this to work. Just looking at the last 8 messages I see 5 things to try. Every person seems to have different success with different methods, or they give up like me. I can't get it to work with any of the solutions posted here (perhaps I haven't tried enough of them, only tried 5).

This is a common problem, perhaps you should write a blog post with the official work around that is supposed to work?

@mitchellh
Owner

What we've been prescribing to paying users that has been working (most of the time, not sure what the edge case is that causes it to fail on a few computers): Uninstall Vagrant, remove ~/.vagrant.d (backup boxes if needed), restart computer, reinstall Vagrant, re-install plugins.

@patrickheeney

@mitchellh That is new advice I did not see in this long email thread. That is why it would be nice to have something official instead of trying to decipher hundreds of messages across multiple tickets. I landed on all these through google anyway, after first checking the site for any troubleshooting steps and common problems.

@v1k0d3n
v1k0d3n commented Aug 20, 2014

@mitchellh i agree with some of the others. i paid for the plugin, and ran into these issues...only to have to attempt a few solutions (that didn't work) before finding one that sort-of worked. i still get some random errors...
WARNING: Could not load IOV methods. Check your GSSAPI C library for an update
WARNING: Could not load AEAD methods. Check your GSSAPI C library for an update

...but i think these are unrelated. the perception is if someone's paying for it, it should work or have a clearly documented work around. thankfully i've been working on product development for a while, but there are some others who are trying to get into development, and this problem could be a stumbling block and force them to look at [what i would feel to be inferior] other solutions. i would hate to see that happen to folks AFTER they paid for the vmware plugin. thoughts?

@trinitronx

@mitchellh I can verify that the workaround fix above avoids the problem for me. I am using Vagrant version 1.6.3.
Here is a patch for the change I made.

Workaround

To apply this to your OSX machine:

curl -Ls https://gist.githubusercontent.com/trinitronx/c5ebdda71523b5b1d4e0/raw/c661f16e2dd26f57a9437e9e1eaa6d1d5ecf0f7d/- | patch -p0

To summarize this after reading through the thread:

It seems that there were 2 problems uncovered:

  1. vagrant plugin install tries to install gems that the plugins do not depend on (Bug: #4278 duplicate of #4194)
  2. A Dependency Hell problem: Vagrant's embedded and bundled environment of gems sometimes may conflict with gems plugins (that may try to support multiple vagrant versions, which complicates things) depend on.
    For example: Vagrant bundles nokogiri 1.6.2.1 in /Applications/Vagrant/embedded/gems/gems/nokogiri-1.6.2.1/.
    Plugins that may depend on nokogiri are allowed to install new gems in ~/.vagrant.d/gems/.

    This problem is summarized nicely by @Tyrael earlier in this thread

Fixing the first issue would resolve most user problems relating to this bug. Fixing the second one (although potentially harder due to split gemset dependency runtime environments with multiple components each with multiple possible versions of gems within them) would also fix it, but without fixing the first issue we'd still be installing unnecessary gems to ~/.vagrant.d/gems/, which makes our problem harder to solve anyway.

Dependency Hell Rant/Theorizey-ness

The dependency hell problem seems much harder to me and has always been a hard problem to solve. I can't say that I'm an expert or that I have a general solution for most cases, but I do have a theory or way that I have tried to visualize it in order to understand the problem better.

Dependency Hell crops up in any software development ecosystem where there are multiple components each depending on functionality of other components which are constantly changing over time. Think of a multidimensional Rubik's cube where working solutions are the equivalent of solving the cube (a solution is defined as a working set of gems that gives us a happy working bundle of software). The number of rows (height) of squares represents the number of gems (or software libraries, or modules) we have in Gemset1. The number of columns (width) represents the gems in Gemset2. The depth dimension and differing colors or (degree of rotational freedom on the Rubik's cube) represents a version that we can change of each gem. The connection of each row or column as a solid piece which rotates together represents the dependency constraints that we have either specified (because we know about them), or that already exist due to interconnected behaviors that each gem either provides to or depends on from other gems.

11x11x11 Rubiks Cube
Eek! This is Hard! 😨

This model is still a bit rough, because there seems to be a larger picture going on due to complex dependency graphs (or sets of gems) that each gem may include in its bundle or gemspec dependency list, and these version constraints may also change over time. We also have larger gems or software bundles/packages (like Vagrant) which depend on a lot of gems, and come pre-packaged with many gems inside a runtime environment. Although these are larger macro-level bundles or packages, we can still treat these as normal gems (at the micro-level) due to the fractal-esque pattern of the bundle or set of gem dependencies (they're just a larger set of dependencies & gems... which increases the difficulty of solving the cube). We can further complicate things by adding other configuration data & environment (ENV['foo']) variables to the mix... making our cube a multidimensional problem (4-D, 5-D, or N-D problem). To complicate things further, generally not all of our specified dependency version constraints are inflexible; That is to say: we as humans can make mistakes in specifying some constraints by being pessimistic while not all our pessimistic assumptions are valid. This is even hard for us to visualize, and harder to solve.

So with this rough visual model, we can see that moving a row or column of the cube around gives us a different permutation of gems in our gemsets, and a solution is where we have a working set of software. Because all of these components change over time, and these dependencies or interconnected sets of behaviors that are provided/depended on can change, it gives us added difficulty in solving the cube. As we can see, this problem is hard to solve in the general & purist theoretical case since it's hard to model every possible permutation & configuration.

Ok... enough appreciating the problem and enough making it seem harder than it needs to be...

We have some important tools to simplify this:

  1. Unit testing - tests individual component's behavior
  2. Integration testing - working behavior of combinations of components
  3. Bundler-like tools - declare & specify our dependencies & runtime environment to test
  4. Problem-Space Simplification - We can cut out or simplify the dimensions of difficulty in order to simplify solving the cube.
    Think of this like taking a hard multidimensional differential equation (like Navier-Stokes), and using boundary conditions to simplify and cancel out parts of the problem that don't need to be solved. The general high-level concept I'm getting at here is Cutting the Gordian Knot.
  5. Constraint Solvers - Gecode is a great example of a generalized constraint solver for CSPs.
  6. Dependency Solvers - These Dependency solvers are built specifically for use with Chef & Cookbooks, but the same idea applies in other problem domains like Ruby Gems.

The main point I'm trying to make here is to take each piece of the problem separately & see where we can cut out difficulty by simplifying our problem space.

2x2x2 Rubiks Cube
Aah! Much Easier! 😸

@jokeyrhyme

@trinitronx is there a way to solve the Dependency Hell problem without switching away from Ruby? Would this still be an issue if plugins were (cross-)compiled into a single file including all of their dependencies? Is it possible to write Vagrant plugins in Go or something else with this feature?

@trinitronx

@jokeyrhyme : Not sure I understand the question fully, but I'm presuming that you're talking about cross-compiling nokogiri's native extensions using Go in order to avoid the compilation problems and library dependencies that it has. Indeed, part of our problem here is that nokogiri is notoriously in the top of the list for gems that are problematic to install due to the native extensions and dependencies on external libraries which are different on multiple platforms. So if nokogiri were to be refactored in some way to avoid this problem it would simplify things. (EDIT: That is to say... chopping off a part of the dependency tree simplifies the problem.)

If you mean the more generic case about solving Dependency Hell in general, I'm not sure there's a way to avoid it entirely so long as there are external code and behaviours being depended on that change over time.

@jokeyrhyme

@trinitronx I was actually wondering if there was a way to distribute Vagrant plugins in a way that requires no external dependencies. So an OS X client would download the OS X version of the Vagrant plugin and have no need to compile anything. Can Ruby gems (the current implementation of Vagrant plugins) be distributed like this?

@lynnaloo

I'm running into this issue with Vagrant 1.6.3 and the workaround (uninstall/reinstall) isn't working for me.

@drpebcak

@lynnaloo modifying the vagrant gemspec seems to work for most, if not all. You'll see that suggestion farther up if you read through the comments.

@patrickheeney

@drpebcak , @mitchellh said this works for most, if not all:

What we've been prescribing to paying users that has been working (most of the time, not sure what the edge case is that causes it to fail on a few computers): Uninstall Vagrant, remove ~/.vagrant.d (backup boxes if needed), restart computer, reinstall Vagrant, re-install plugins.

It seems there is no suggestion that seems to work for most. I was hoping we could get an official fix or at least recommendation on what to do without trying the dozens of "fixes" in dozens of comments.

@lynnaloo

Modifying the vagrant gemspec worked for me. That's probably a little bit too technical for people who are using my Vagrant setup to eliminate the complication of setting up our development environment :) I'll pass on the uninstall, remove ~/.vagrant..d, re-install method. Thanks guys!

@ndemoor
ndemoor commented Aug 22, 2014

Had the exact same issue on Mac OS X Mountain Lion and Vagrant 1.6.3.
Nothing of the above workarounds fixed it, except this one:

22 s.add_dependency(%q<nokogiri>, ["= 1.6.2.1"])

Worked like a charm.

@metaColin

What worked for me was entirely removing Vagrant, and going back to using version 1.5.4.

I realize it's not exactly the solution people are asking for, but it works reliably.

@trinitronx

@jokeyrhyme : Well presumably that would be possible, but what you describe probably hasn't been done yet. A quick google search yields this page on a proof of concept for writing native extensions in Go

@johnleetran

SOLVED

was solved by running the following

gem install --install-dir ~/.vagrant.d/gems nokogiri -v '1.6.3.1'

if you get another error like before run the same command on that package, for example

gem install --install-dir ~/.vagrant.d/gems dep_selector -v '1.0.3'
@jokeyrhyme

@johnleetran the danger with that is the gem will be configured / compiled against the global Ruby install, and not the Ruby that comes with Vagrant. It's possible this will be fine in some cases (i.e. where they are the same version number), but this not ideal long-term. Nice thinking outside the box though. :)

@sgerrand

@johnleetran the danger with that is the gem will be configured /
compiled against the global Ruby install, and not the Ruby that comes with
Vagrant. It's possible this will be fine in some cases (i.e. where they are
the same version number), but this not ideal long-term.

Ruby is an interpreted language, so this assumption is incorrect. Libraries
in Ruby are not compiled against a version of the MRI Ruby interpreter,
although they can mark themselves as requiring a specific Ruby version.

This entire thread is related to the Nokogiri gem, which has C extension
files that /are/ compiled during installation, specifically against XML and
XSLT libraries.

@eswenson1

I just updated from vagrant 1.6.3 (where I had this nokogiri issue on macosx) to vagrant 1.6.4 (latest). I'm now able to install the vagrant-berkshelf plugin (2.0.1) with no issues.

@mitchellh
Owner

Yep, I fixed it there, I added the hard nokogiri dependency. Should be good.

@wolf31o2 wolf31o2 added a commit to caskdata/coopr that referenced this issue Dec 18, 2014
@wolf31o2 wolf31o2 Add build-essential, from mitchellh/vagrant#3769 f8f6ff0
@mattjanssen mattjanssen referenced this issue in devopsgroup-io/vagrant-hostmanager Dec 30, 2014
Closed

Unable to install - "Error while installing nokogiri" #126

@7stud
7stud commented Jan 22, 2015

josegonzalez said,
as an aside, people should maybe not use xml and start using json ;)

~/vagrant_getting_started$ vagrant plugin install vagrant-vbguest
Installing the 'vagrant-vbguest' plugin. This can take a few minutes...
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing json (1.8.2), and Bundler cannot continue.

Here are the steps leading up to that error:

~/vagrant_getting_started$ vagrant --version
Vagrant 1.7.2

~/vagrant_getting_started$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'hashicorp/precise32'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'hashicorp/precise32' is up to date...
==> default: Setting the name of the VM: vagrant_getting_started_default_1421891952937_26048
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 => 2222 (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Connection timeout. Retrying...
    default: 
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default: 
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if its present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
    default: The guest additions on this VM do not match the installed version of
    default: VirtualBox! In most cases this is fine, but in rare cases it can
    default: prevent things such as shared folders from working properly. If you see
    default: shared folder errors, please make sure the guest additions within the
    default: virtual machine match the version of VirtualBox you have installed on
    default: your host and reload your VM.
    default: 
    default: Guest Additions Version: 4.2.0
    default: VirtualBox Version: 4.3
==> default: Mounting shared folders...
    default: /vagrant => /Users/7stud/vagrant_getting_started

In an effort to solve the Guest Additions problem:

~/vagrant_getting_started$ vagrant plugin install vagrant-vbguest
Installing the 'vagrant-vbguest' plugin. This can take a few minutes...
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing json (1.8.2), and Bundler cannot continue.
Make sure that `gem install json -v '1.8.2'` succeeds before bundling.

Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.

    /opt/vagrant/embedded/bin/ruby extconf.rb 
/opt/vagrant/embedded/lib/ruby/2.0.0/mkmf.rb:292:in `initialize_copy': Bad file descriptor (Errno::EBADF)
    from /opt/vagrant/embedded/lib/ruby/2.0.0/mkmf.rb:292:in `initialize_dup'
    from /opt/vagrant/embedded/lib/ruby/2.0.0/mkmf.rb:292:in `dup'
    from /opt/vagrant/embedded/lib/ruby/2.0.0/mkmf.rb:292:in `<module:Logging>'
    from /opt/vagrant/embedded/lib/ruby/2.0.0/mkmf.rb:289:in `<module:MakeMakefile>'
    from /opt/vagrant/embedded/lib/ruby/2.0.0/mkmf.rb:47:in `<top (required)>'
    from /opt/vagrant/embedded/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /opt/vagrant/embedded/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from extconf.rb:1:in `<main>'


Gem files will remain installed in /Users/7stud/.vagrant.d/gems/gems/json-1.8.2 for inspection.
Results logged to /Users/7stud/.vagrant.d/gems/gems/json-1.8.2/ext/json/ext/generator/gem_make.out
~/vagrant_getting_started$ gem install json -v '1.8.2'
Fetching: json-1.8.2.gem (100%)
Building native extensions.  This could take a while...
Successfully installed json-1.8.2
1 gem installed
~/vagrant_getting_started$ vagrant plugin install vagrant-vbguest
Installing the 'vagrant-vbguest' plugin. This can take a few minutes...
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:

An error occurred while installing json (1.8.2), and Bundler cannot continue.
Make sure that `gem install json -v '1.8.2'` succeeds before bundling.

Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.

    /opt/vagrant/embedded/bin/ruby extconf.rb 
/opt/vagrant/embedded/lib/ruby/2.0.0/mkmf.rb:292:in `initialize_copy': Bad file descriptor (Errno::EBADF)
    from /opt/vagrant/embedded/lib/ruby/2.0.0/mkmf.rb:292:in `initialize_dup'
    from /opt/vagrant/embedded/lib/ruby/2.0.0/mkmf.rb:292:in `dup'
    from /opt/vagrant/embedded/lib/ruby/2.0.0/mkmf.rb:292:in `<module:Logging>'
    from /opt/vagrant/embedded/lib/ruby/2.0.0/mkmf.rb:289:in `<module:MakeMakefile>'
    from /opt/vagrant/embedded/lib/ruby/2.0.0/mkmf.rb:47:in `<top (required)>'
    from /opt/vagrant/embedded/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /opt/vagrant/embedded/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from extconf.rb:1:in `<main>'


Gem files will remain installed in /Users/7stud/.vagrant.d/gems/gems/json-1.8.2 for inspection.
Results logged to /Users/7stud/.vagrant.d/gems/gems/json-1.8.2/ext/json/ext/generator/gem_make.out
@benjohnson77

I came into this issue yesterday, a bit late to the party. For me updating vagrant to 1.7.2 solved the issue.

@obfuscurity

Same. Weird how boxes suddenly stopped working correctly under 1.6.3 for no apparent reason, but upgrading to 1.7.2 fixed it for me.

@jfryman jfryman added a commit to StackStorm/st2workroom that referenced this issue Feb 2, 2015
@jfryman jfryman Add Vagrant version requirement
Avoids problems with `nokogiri` install as experienced by @enykeev - mitchellh/vagrant#3769
7a21669
@tomav
tomav commented Mar 3, 2015

Same here, upgrading to 1.7.2 fixed the problem. Thank you @benjohnson77 & @obfuscurity

@legal90 legal90 referenced this issue in Parallels/vagrant-parallels Mar 9, 2015
Closed

Installation fails #180

@arnlaugsson

Updating to Vagrant 1.7.2 fixed it for me too.

~  ᐅ vagrant version
Installed Version: 1.7.2
Latest Version: 1.7.2

You're running an up-to-date version of Vagrant!
~  ᐅ vagrant plugin install vagrant-cachier
Installing the 'vagrant-cachier' plugin. This can take a few minutes...
Installed the plugin 'vagrant-cachier (1.2.0)'!
@hadifarnoud

i still have this issue with vagrant 1.6.3 OSX 10.10.3 gem 2.4.6

I have tried ALL the above solutions. At last I edited nokogiri 1.6.2.1 gemspec in /Vagrant folder and modified the version to s.version = "1.6.6.2"

not sure this is going to mess up some things or not.

@iros iros referenced this issue in bocoup/deployment-workflow Apr 27, 2015
Closed

Document potential issue with installing vagrant-hostupdater #12

@fireproofsocks

Having this problem with Vagrant 1.6.3 on Mac OS X 10.9.5, updating to Vagrant 1.7.2 fixed it.

@johnleetran

upgrade to a newer version of vagrant OR

gem install --install-dir ~/.vagrant.d/gems nokogiri -v '1.6.3.1'

if you get another error like before run the same command on that package, for example

gem install --install-dir ~/.vagrant.d/gems dep_selector -v '1.0.3'
@jokeyrhyme

I've had a great run with Vagrant, but I'm transitioning my projects to Docker now. Docker gets easier and easier to use each day, and it doesn't have any of the Ruby-to-C++ interface problems that Vagrant has. If you are having problems with Vagrant, give Docker a try.

@afilp
afilp commented May 14, 2015

@jokeyrhyme Is there a "VVV" equivalent in Docker?

@jokeyrhyme

@af7 if you mean http://varyingvagrantvagrants.org/ , then I'm not sure. I did find this (but it seems incomplete): https://github.com/Varying-Vagrant-Vagrants/vvv-docker/tree/lkwdwrd

@johnleetran

i still prefer vagrant because vagrant boxes are easier to destroy than docker containers/images. i am tired of doing a

docker rmi -f $(docker images -q)
docker rm -f $(docker ps -aq)

Boot2docker, pushing to a privately hosted docker repository is a pain in the butt.

Another pain point, docker caching for building new docker images is a subtle trap. We've been bitten by this several times in development.

@theninthnode

Solution for me with Vagrant 1.6.3 on Mac was adding this to vagrant-1.6.3.gemspec like suggested above:

s.add_dependency(%q, ["< 1.6.3"])

@mikemcguire mikemcguire referenced this issue in cogitatio/vagrant-hostsupdater Jul 1, 2015
Closed

Bundler Says error installing nokogiri #70

@ryancastle

I just had this error on vagrant 1.6.3 for nokogiri 1.6.6.2 when installing vagrant-cachier. But installing that version of nokogiri into ~/.vagrant.d/gems as per @johnleetran's suggestion solved it.

@damienalexandre

On Ubuntu 15.10, I just had to install zlib1g-dev:

$ sudo apt install zlib1g-dev
$ vagrant plugin install vagrant-cachier
    Installing the 'vagrant-cachier' plugin. This can take a few minutes...
    Installed the plugin 'vagrant-cachier (1.2.1)'!

This is cleaner than installing an old version of nokogiri manually 😉

@Tyrael
Tyrael commented Jan 12, 2016

not everybody was having problem of building nokogiri because of the missing zlib1g-dev package ;)

@Donal-Flanagan

I had the exact same issue on ubuntu 15.10, with vagrant 1.7.4. Installing the zlib1g-dev package worked perfectly. Thanks for the fix @damienalexandre

@tartley
tartley commented Mar 21, 2016

I also had the exact same issue on Ubuntu 15.10, with vagrant 1.7.4+dfsg-1_all (Ubuntu apt-get default version), but unlike @Donal-Flanagan, installing zlib1g-dev did not fix it for me. I then did a gem install nokogiri -v '1.6.7.2' (following the instructions in the error message Make sure that "gem install nokogiri -v '1.6.7.2'" succeeds...) and it did succeed, and after that, my vagrant plugin install... started working.

@Mathyn
Mathyn commented Apr 13, 2016

I had this issue today when trying to install the vagrant-triggers plugin with Ubuntu 14.04. When I tried to manually run gem install nokogiri -v '1.6.7.2' as suggested by the error output this also failed to install but did provide the helpful error message zlib is missing; necessary for building libxml2.

After installing zlib (sudo apt-get install zlib1g-dev) I could install nokogiri and with that install vagrant-triggers.

@avishnyakov

Getting this while using vagrant-azure (https://github.com/Azure/vagrant-azure) on win10. Not sure what the resolution might be yet.

@avishnyakov avishnyakov referenced this issue in Azure/vagrant-azure May 30, 2016
Closed

Vagrant up fails on nokogiri module #124

@akeif
akeif commented Jun 14, 2016

I had the same problem on Ubuntu 16.04. Installing the latest version of Vagrant from their website fixed it for me.

@bhouston

@Mathyn thanks for the suggestion:

After installing zlib (sudo apt-get install zlib1g-dev) I could install nokogiri and with that install vagrant-triggers.

On a fresh install of Ubuntu 16.04, running sudo apt-get install zlib1g-dev got lxc running for me. Sweet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment