Skip to content
This repository

"invalid gemspec" with old version gem on bundler 1.1 #1715

Closed
tomykaira opened this Issue March 10, 2012 · 22 comments

5 participants

tomykaira André Arko Rohit Arondekar Jani Patokallio Bradley Schaefer
tomykaira

Gemfile:

source :rubygems

gem 'sinatra'
gem 'milkode', '0.4.0'

In bundler 1.1, bundle fails with the following error.

Unfortunately, the gem milkode (0.2.3) has an invalid gemspec. As a result, Bundler cannot install this Gemfile. Please ask the gem author to yank the bad version to fix this issue. For more information, see http://bit.ly/syck-defaultkey.

But no error with bundler 1.0.

I'm not sure what is the cause of this, but it seems to be a bug for me.

Rohit Arondekar
Collaborator
rohit commented March 11, 2012

This is not a bug in Bundler but a problem with Rubygems.org ― please read the link provided in the error message => http://blog.rubygems.org/2011/08/31/shaving-the-yaml-yak.html

tomykaira

@rohit, Thank you for response.

Sure. I agree that the main cause is in Rubygems.org.

I am sorry that my question was obscure.

I mean, "Why bundler 1.1 references an old gemspec?"
The latest version of milkode was 0.4.0 (when I tested, it's now 0.5.0), but bundler reports the error with milkode 0.2.3.

I know little about bundler, but I heard that the mechanism of dependency resolution was changed in 1.1.
I suspect this change made such buggy behavior.

Rohit Arondekar
Collaborator
rohit commented March 11, 2012

Can you test again with both Bundler 1.0.x and Bundler 1.1 ― by my guesstimate this has to do with your version of Rubygems. I think if you update rubygems then the error should go away.

tomykaira

There are full logs of bundling with 1.0.22 and 1.1.0.

I am willing to try this with other set of versions, if it is needed.

Gemfile:

source :rubygems

gem 'sinatra'
gem 'milkode', '0.5.0'

With ruby 1.9.2-p180, rubygems 1.8.17, bundler 1.0.22

~/programs/ruby/gitomb (bundler-test)> ruby --version
ruby 1.9.2p180 (2011-02-18 revision 30909) [i686-linux]
~/programs/ruby/gitomb (bundler-test)> gem --version
1.8.17
~/programs/ruby/gitomb (bundler-test)> rvm list

rvm rubies

   ruby-1.8.7-p334 [ i386 ]
   ruby-1.9.3-preview1 [ i386 ]
   ruby-head [ i386 ]
=> ruby-1.9.2-p180 [ i386 ]
   ruby-1.9.2-p290 [ i386 ]
   ruby-1.8.7-p352 [ i386 ]

~/programs/ruby/gitomb (bundler-test)> bundle --version
Bundler version 1.0.22
~/programs/ruby/gitomb (bundler-test)> rm .bundle/ vendor/ Gemfile.lock
~/programs/ruby/gitomb (bundler-test)> bundle install --path vendor/gems
Fetching source index for http://rubygems.org/
Installing addressable (2.2.7) 
Installing io-like (0.3.0) 
Installing archive-zip (0.5.0) 
Installing coderay (0.9.8) 
Installing daemons (1.1.8) 
Installing eventmachine (0.12.10) with native extensions 
Installing haml (3.1.4) 
Installing highline (1.6.11) 
Installing hpricot (0.8.6) with native extensions 
Installing json (1.6.5) with native extensions 
Installing launchy (2.0.5) 
Installing nokogiri (1.5.2) with native extensions 
Installing rack (1.4.1) 
Installing pkg-config (1.1.3) 
Installing rroonga (1.3.1) with native extensions 
Installing sass (3.1.15) 
Installing rack-protection (1.2.0) 
Installing tilt (1.3.3) 
Installing sinatra (1.3.2) 
Installing termcolor (1.2.1) 
Installing thin (1.3.1) with native extensions 
Installing milkode (0.5.0) 
Using bundler (1.0.22) 
Your bundle is complete! It was installed into ./vendor/gems

With ruby 1.9.2-p180, rubygems 1.8.17, bundler 1.1.0

~/programs/ruby/gitomb (bundler-test)> gem install bundler 
Fetching: bundler-1.1.0.gem (100%)
Successfully installed bundler-1.1.0
1 gem installed
~/programs/ruby/gitomb (bundler-test)> ruby --version
ruby 1.9.2p180 (2011-02-18 revision 30909) [i686-linux]
~/programs/ruby/gitomb (bundler-test)> gem --version
1.8.17
~/programs/ruby/gitomb (bundler-test)> bundle --version
Bundler version 1.1.0
~/programs/ruby/gitomb (bundler-test)> rm .bundle/ vendor/ Gemfile.lock
~/programs/ruby/gitomb (bundler-test)> bundle install --path vendor/gems
Fetching gem metadata from http://rubygems.org/.
Unfortunately, the gem milkode (0.2.3) has an invalid gemspec. As a result, Bundler cannot install this Gemfile. Please ask the gem author to yank the bad version to fix this issue. For more information, see http://bit.ly/syck-defaultkey.
André Arko
Owner

Hi guys. This bug is slightly complex, but there is unfortunately no way to solve it inside Bundler.

Here's why things go wrong:

  1. The milkode 0.2.3 gemspec on rubygems.org is invalid.
  2. When you have a gemfile that includes milkode, Bundler needs to know about every version of the gem in order to find a version that it can install. That means downloading the milkode 0.2.3 gemspec.
  3. The milkode 0.2.3 gemspec is invalid, and so Bundler explodes.

In Bundler 1.0, the gemspec is never downloaded. Instead, Bundler downloads the entire list of every single gem version that has ever existed, and uses that list. Since that list is marshalled Ruby instead of YAML, it is not corrupt, and does not cause an exception.

You can force Bundler 1.1 to use the old (and much slower) approach, you can force it by running bundle install --full-index. I suggest that you try to get the bad gem yanked instead, though. :)

André Arko indirect closed this March 12, 2012
tomykaira

Thanks @indirect!!

I did not know the --full-index option.
At least, I no longer have to downgrade bundler every time :)

Jani Patokallio

Maybe I'm missing something obvious, but couldn't Bundler simply choose not to "explode" when it hits an invalid gemspec? The current code catches the error and raises another, bringing everything to a screeching halt, but it could just dump the error message to STDERR and carry on.

Also, I found http://blog.rubygems.org/2011/08/31/shaving-the-yaml-yak.html quite unhelpful, it explains why it's blowing up but doesn't actually give users any way to solve the problem. Linking to this discussion, or simply suggesting the use of --full-index, would be much more useful.

Jani Patokallio

One more thing: while --full-index works nicely for bundle install, it's not available for bundle update, which also spits the dummy on encountering an old invalid gemspec. =/

André Arko
Owner

What exactly do you mean by "carry on"? Pretend that the gem doesn't exist? That would produce completely different resolves between 1.0 and 1.1, and isn't acceptable.

The solution for gem end-users is to get the gem owner to yank bad gems. Or possibly get the rubygems.org maintainers to yank the bad gems, if the gem owners aren't available.

If you're interested in patching update to support --full-index, that seems reasonable to me.

Jani Patokallio

No, pretend that version of the gem doesn't exist, just return a nil instead of an array and call compact. Here's a quick shot at it that seems to work: #1763

André Arko
Owner

That's what I mean. It's a complete non-starter to implement anything that results in Bundler 1.0 creating a lock file that 1.1 can't install. Since there's no way for 1.0 to tell which versions have bad gemspecs, 1.1 has to abort when it runs into any.

Jani Patokallio

Not quite following you here. 1.1 is already breaking on lots of Gemfiles that used to work just fine under 1.0, so you have already broken compatibility. My suggested fix would stop it from breaking in the vast majority cases where the broken gemspec is for a version the user does not actually want to install.

André Arko
Owner

We have different definitions of "broken". You just want to install your gem. I have to make sure that Bundler cannot ever, under any circumstances, install a different set of gems between 1.0 and 1.1. That means making --full-index available if you need it, and aborting instead of silently causing huge potential problems later.

Jani Patokallio

OK, I think I finally see where you're coming from: so if the Gemfile is written loosely, and the latest matching version of the gem has an invalid gemspec, then you don't want Bundler 1.0 to resolve to v.invalid and Bundler 1.1 to skip v.invalid and resolve to v.invalid-minus-1. This is the case you're concerned about, right? So would it be possible to make it abort only if the matched version has an invalid gemspec, and just raise a warning otherwise? Because if the gemfile is requiring 0.4.0, and it's 0.2.3 that's broken, then neither version will ever resolve to 0.2.3 and there is no potential for later problems.

André Arko
Owner
Jani Patokallio

Isn't the version of the gem already available in the list of gems? It's already included in the error message output on encountering a broken gemspec: gem #{s[:name]} (#{s[:number]}) has an invalid gemspec

André Arko
Owner
Bradley Schaefer

I ran into this today...

How many bad gemspecs are out there, though? It's got to be a huge number. Getting each one pulled is a bit onorous. I would be satisfied with a better error message AND patching update to be able to specify --full-index . I see this issue is a bit old and stale... should I open another issue for that? Or should this be reopened?

André Arko
Owner

It is practically zero, because the rubygems.org team has proactively gone through and wiped or corrected almost all of them. Plus, the latest versions of Rubygems and Rubygems.org automatically prevent bad gemspecs from being created or uploaded.

Bradley Schaefer

Huh, strange how I still ran up against this. Maybe this is the wrong place to ask, but who do I contact about a problem like this? rake-hooks was the project that seems to have tripped bundler up for me FWIW.. it's not obvious to me which of the 7 versions on rubygems is creating the issue.

Jani Patokallio

No, it's not zero, because this bug report alone has four people who have hit the issue. And because Bundler chokes if any version of the gem has a bad spec, it makes no difference if the later versions are correct.

@soulcutter: The error message should say which version is causing the problem, eg. "Unfortunately, the gem exactish_target (0.2.1) has an invalid gemspec" means it's 0.2.1 that's causing the problem. There may be other broken versions as well though, since it aborts on the first...

Bradley Schaefer

Thanks @jpatokal - I didn't realize that was what the (1) in the message was :P

Bradley Schaefer soulcutter referenced this issue from a commit in soulcutter/bundler April 10, 2012
Added --full-index option to bundle update re #1715 fc44880
Bradley Schaefer soulcutter referenced this issue from a commit in soulcutter/bundler April 10, 2012
Documented a workaround to invalid gemspecs in the error message re #… 6762862
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.