Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Pre-release gems being chosen when they shouldn't #853

Closed
sethvargo opened this Issue Mar 13, 2014 · 28 comments

Comments

Projects
None yet
7 participants

I've created and outlined a complete reproduction case in this gist.

In v2.2.2 of Rubygems, pre-release gems are being installed (although the problem is not affecting bundler) "automatically". Originally I thought this was a Ruby 2.1.1 specific issue, but it turns out that Ruby 2.1.1 just bundles a newer version of Rubygems.

Repro:

  1. Create a new gem

    $ bundle gem foo
    
  2. Add a dependency on something pessimistically (I chose RSpec)

    spec.add_dependency 'rspec', '~> 2.14'
  3. Build the gem:

    $ rake build
    
  4. Install with gem:

    $ gem install pkg/foo-0.0.1.gem
    

Rubygems will install the pre-release versions:

Successfully installed rspec-mocks-2.99.0.beta2
Successfully installed diff-lcs-1.2.5
Successfully installed rspec-expectations-2.99.0.beta2
Successfully installed rspec-core-2.99.0.beta2
Successfully installed rspec-2.99.0.beta2
Successfully installed foo-0.0.1
6 gems installed

I have not been able to reproducible this on older versions of Rubygems.

Owner

indirect commented Mar 13, 2014

May be related to #813 and #828.

@indirect I spent some time digging into this. It seems (according to bisect) that this commit broke things: 9389824

Specifically, I think

found.delete_if { |s| s.version.prerelease? } unless dependency.prerelease?

does not account for transitive pre-release versions (only top-level). Am I incorrect in that thinking? i'm not super familiar with this code base.

Owner

indirect commented Mar 20, 2014

That seems plausible to me, but I am also not familiar with the Rubygems resolver. Bundler has its own resolver, as well as a separate set of tests that are not currently run by Rubygems.

Okay cool. I'll take a look.

Happens to me as well

I ran gem install skylight which depends on activesupport >= 3.0.0 and it pulled in 4.1.0.rc2

@drbrain drbrain added this to the 2.3 milestone Apr 3, 2014

Contributor

TimMoore commented Apr 8, 2014

Another one: https://gist.github.com/TimMoore/10080042

The nokogiri 1.6.2.rc1 release seems to have a few issues, and this bug means that installing any gem that depends on nokogiri can unexpectedly install that release candidate instead of 1.6.1 final.

@drbrain drbrain closed this in 831b320 Apr 22, 2014

Owner

drbrain commented Apr 22, 2014

Looks like I fixed this, but I don't know what commit did it.

@thatothermitch thatothermitch referenced this issue in berkshelf/berkshelf Apr 28, 2014

Closed

Berkshelf depends on pre-release gems #1172

zzak added a commit to zzak/rubygems that referenced this issue May 9, 2014

Mark #853 fixed as I cannot reproduce it anymore
It seems that while working on the related issues I have fixed this bug.
I cannot reproduce it using the examples in the issue, but I may be
missing something still.

Fixes #853
Member

zzak commented May 9, 2014

I was able to repro this using bundler to create foo gem like in the description

@zzak zzak reopened this May 9, 2014

Member

zzak commented May 9, 2014

Using current master I get the following error:

$ ruby -Ilib bin/gem install foo-0.0.1.gem
ERROR:  While executing gem ... (Gem::DependencyError)
    Unresolved dependency found during sorting - rspec (~> 2.14) (requested by foo-0.0.1)
Owner

drbrain commented May 14, 2014

I still can't reproduce:

$ cat a.gemspec
Gem::Specification.new do |s|
  s.name = 'a'
  s.version = 1
  s.summary = 'a'
  s.author = 'a'
  s.add_dependency 'rspec', '~> 2.14'
end
$ ruby -Ilib bin/gem build a.gemspec 
WARNING:  licenses is empty, but is recommended.  Use a license abbreviation from:
http://opensource.org/licenses/alphabetical
WARNING:  no description specified
WARNING:  no email specified
WARNING:  no homepage specified
WARNING:  See http://guides.rubygems.org/specification-reference/ for help
  Successfully built RubyGem
  Name: a
  Version: 1
  File: a-1.gem
$ rm -r ~/tmp/gems; ruby -Ilib bin/gem i a-1.gem -i ~/tmp/gems --no-doc
rm: /Users/drbrain/tmp/gems: No such file or directory
Fetching: rspec-core-2.14.8.gem (100%)
Successfully installed rspec-core-2.14.8
Fetching: diff-lcs-1.2.5.gem (100%)
Successfully installed diff-lcs-1.2.5
Fetching: rspec-expectations-2.14.5.gem (100%)
Successfully installed rspec-expectations-2.14.5
Fetching: rspec-mocks-2.14.6.gem (100%)
Successfully installed rspec-mocks-2.14.6
Fetching: rspec-2.14.1.gem (100%)
Successfully installed rspec-2.14.1
Successfully installed a-1
6 gems installed
$ git hist -n 1
* 8740183 2014-05-13 | Support /quick with prereleases in Gem::Server (HEAD, origin/master, origin/HEAD, master) [Eric Hodel]

Can you show a backtrace?

benmoss commented May 17, 2014

I'm also having this issue. The gem I am installing has a dependency of thin >= 1.3.1, and under MRI 2.1.1 and RubyGems 2.2.2 I am having thin 2.0.0.pre installed. The regression seems to have been introduced in 2.2.1 as far as I could tell by trying to reinstall with varying versions of RubyGems.

Confirmed - this is still an issue for me. The steps to reproduce I listed in the description are also still valid.

benmoss commented May 17, 2014

I did a bisect and the bug I'm experiencing was introduced by 204d1b9 and has been fixed by d9c25c5

benmoss commented May 17, 2014

I just checked the repro steps @sethvargo provided and d9c25c5 also fixes that issue.

Owner

drbrain commented May 17, 2014

Thanks, closing

@drbrain drbrain closed this May 17, 2014

@benmoss I'm 99% certain that's not the case. I'm also certain that @zzak was experiencing these same issues too. There's obviously something different in our envs.

Member

zzak commented May 17, 2014

Repro:

# foo.gemspec

# coding: utf-8
Gem::Specification.new do |spec|
  spec.name          = "foo"
  spec.summary       = "foo"
  spec.version       = "0.0.2"
  spec.authors       = ["Zachary Scott"]
  spec.email         = ["e@zzak.io"]

  spec.add_dependency "rspec", "~> 2.14"
end
ruby -Ilib bin/gem build foo.gemspec
WARNING:  licenses is empty, but is recommended.  Use a license abbreviation from:
http://opensource.org/licenses/alphabetical
WARNING:  no description specified
WARNING:  no homepage specified
WARNING:  See http://guides.rubygems.org/specification-reference/ for help
  Successfully built RubyGem
  Name: foo
  Version: 0.0.2
  File: foo-0.0.2.gem
ruby -Ilib bin/gem install foo-0.0.2.gem --verbose
HEAD https://api.rubygems.org/api/v1/dependencies
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=foo
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=rspec
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=rspec-core,rspec-expectations,rspec-mocks
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=diff-lcs
200 OK
ERROR:  While executing gem ... (Gem::DependencyError)
    Unresolved dependency found during sorting - rspec (~> 2.14) (requested by foo-0.0.2)
ruby 2.2.0dev (2014-05-18 trunk 45977) [x86_64-darwin13]

@zzak zzak reopened this May 17, 2014

benmoss commented May 18, 2014

Ok, sounds like there might be multiple issues here. I followed the same steps you just listed and it succeeded: https://gist.github.com/benmoss/1518a9e82955a17e35a9

I also was under the impression the issue here was that prereleases were being installed, it looks like @zzak 's error here was some unresolved dependency error preventing anything from being installed. I admit I might not know what I'm talking about here.

The only thing I see different (aside from the obvious paths) is Ruby version.

Owner

drbrain commented May 19, 2014

I still can't reproduce on @bbbb459 with either 2.1.2p95 or trunk from today:

$ cat foo.gemspec 
# coding: utf-8
Gem::Specification.new do |spec|
  spec.name          = "foo"
  spec.summary       = "foo"
  spec.version       = "0.0.2"
  spec.authors       = ["Zachary Scott"]
  spec.email         = ["e@zzak.io"]

  spec.add_dependency "rspec", "~> 2.14"
end
$ ruby -Ilib bin/gem build foo.gemspec 
WARNING:  licenses is empty, but is recommended.  Use a license abbreviation from:
http://opensource.org/licenses/alphabetical
WARNING:  no description specified
WARNING:  no homepage specified
WARNING:  See http://guides.rubygems.org/specification-reference/ for help
  Successfully built RubyGem
  Name: foo
  Version: 0.0.2
  File: foo-0.0.2.gem
$ rm -fr ~/tmp/gems; ruby -vIlib bin/gem install foo-0.0.2.gem -i ~/tmp/gems --backtrace --no-doc
ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-darwin13.0]
Fetching: rspec-core-2.14.8.gem (100%)
Successfully installed rspec-core-2.14.8
Fetching: diff-lcs-1.2.5.gem (100%)
Successfully installed diff-lcs-1.2.5
Fetching: rspec-expectations-2.14.5.gem (100%)
Successfully installed rspec-expectations-2.14.5
Fetching: rspec-mocks-2.14.6.gem (100%)
Successfully installed rspec-mocks-2.14.6
Fetching: rspec-2.14.1.gem (100%)
Successfully installed rspec-2.14.1
Successfully installed foo-0.0.2
6 gems installed
$ rm -rf ~/tmp/gems; ~/.rubies/trunk/bin/ruby -vIlib bin/gem i foo-0.0.2.gem -i ~/tmp/gems --backtrace --no-doc
ruby 2.2.0dev (2014-05-20 trunk 46025) [x86_64-darwin13]
Fetching: rspec-core-2.14.8.gem (100%)
Successfully installed rspec-core-2.14.8
Fetching: diff-lcs-1.2.5.gem (100%)
Successfully installed diff-lcs-1.2.5
Fetching: rspec-expectations-2.14.5.gem (100%)
Successfully installed rspec-expectations-2.14.5
Fetching: rspec-mocks-2.14.6.gem (100%)
Successfully installed rspec-mocks-2.14.6
Fetching: rspec-2.14.1.gem (100%)
Successfully installed rspec-2.14.1
Successfully installed foo-0.0.2
6 gems installed

Nor when using already installed gems:

$ ruby -vIlib bin/gem i foo-0.0.2.gem -i ~/tmp/gems --backtrace --no-doc
ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-darwin13.0]
Successfully installed foo-0.0.2
1 gem installed
$ ~/.rubies/trunk/bin/ruby -vIlib bin/gem i foo-0.0.2.gem -i ~/tmp/gems --backtrace --no-doc
ruby 2.2.0dev (2014-05-20 trunk 46025) [x86_64-darwin13]
Successfully installed foo-0.0.2
1 gem installed
Member

zzak commented May 19, 2014

Here's my env:

 ruby -Ilib bin/gem env
RubyGems Environment:
  - RUBYGEMS VERSION: 2.2.2
  - RUBY VERSION: 2.2.0 (2014-05-18 patchlevel -1) [x86_64-darwin13]
  - INSTALLATION DIRECTORY: /Users/zscott/.gem/ruby/2.2.0
  - RUBYGEMS PREFIX: /Users/zscott/projects/rubygems
  - RUBY EXECUTABLE: /Users/zscott/.rubies/ruby-trunk/bin/ruby
  - EXECUTABLE DIRECTORY: /Users/zscott/.gem/ruby/2.2.0/bin
  - SPEC CACHE DIRECTORY: /Users/zscott/.gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: /Users/zscott/.rubies/ruby-trunk/etc
  - RUBYGEMS PLATFORMS:
    - ruby
    - x86_64-darwin-13
  - GEM PATHS:
     - /Users/zscott/.gem/ruby/2.2.0
     - /Users/zscott/.rubies/ruby-trunk/lib/ruby/gems/2.2.0
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - /Users/zscott/.gem/ruby/2.2.0/bin
     - /Users/zscott/.rubies/ruby-trunk/lib/ruby/gems/2.2.0/bin
     - /Users/zscott/.rubies/ruby-trunk/bin
     - /Users/zscott/opendylan-2013.2/bin
     - /Users/zscott/.bin
     - /usr/local/bin
     - /usr/local/sbin
     - /usr/bin
     - /bin
     - /usr/sbin
     - /sbin
     - /usr/local/bin
     - /usr/local/MacGPG2/bin
Member

zzak commented May 19, 2014

with --backtrace:

ERROR:  While executing gem ... (Gem::DependencyError)
    Unresolved dependency found during sorting - rspec (~> 2.14) (requested by foo-0.0.2)
    /Users/zscott/projects/rubygems/lib/rubygems/request_set.rb:334:in `block in tsort_each_child'
    /Users/zscott/projects/rubygems/lib/rubygems/request_set.rb:326:in `each'
    /Users/zscott/projects/rubygems/lib/rubygems/request_set.rb:326:in `tsort_each_child'
    /Users/zscott/.rubies/ruby-trunk/lib/ruby/2.2.0/tsort.rb:411:in `call'
    /Users/zscott/.rubies/ruby-trunk/lib/ruby/2.2.0/tsort.rb:411:in `each_strongly_connected_component_from'
    /Users/zscott/.rubies/ruby-trunk/lib/ruby/2.2.0/tsort.rb:347:in `block in each_strongly_connected_component'
    /Users/zscott/projects/rubygems/lib/rubygems/request_set.rb:322:in `each'
    /Users/zscott/projects/rubygems/lib/rubygems/request_set.rb:322:in `tsort_each_node'
    /Users/zscott/.rubies/ruby-trunk/lib/ruby/2.2.0/tsort.rb:345:in `call'
    /Users/zscott/.rubies/ruby-trunk/lib/ruby/2.2.0/tsort.rb:345:in `each_strongly_connected_component'
    /Users/zscott/.rubies/ruby-trunk/lib/ruby/2.2.0/tsort.rb:280:in `strongly_connected_components'
    /Users/zscott/.rubies/ruby-trunk/lib/ruby/2.2.0/tsort.rb:255:in `strongly_connected_components'
    /Users/zscott/projects/rubygems/lib/rubygems/request_set.rb:308:in `sorted_requests'
    /Users/zscott/projects/rubygems/lib/rubygems/request_set.rb:140:in `install'
    /Users/zscott/projects/rubygems/lib/rubygems/commands/install_command.rb:230:in `install_gem'
    /Users/zscott/projects/rubygems/lib/rubygems/commands/install_command.rb:272:in `block in install_gems'
    /Users/zscott/projects/rubygems/lib/rubygems/commands/install_command.rb:268:in `each'
    /Users/zscott/projects/rubygems/lib/rubygems/commands/install_command.rb:268:in `install_gems'
    /Users/zscott/projects/rubygems/lib/rubygems/commands/install_command.rb:183:in `execute'
    /Users/zscott/projects/rubygems/lib/rubygems/command.rb:307:in `invoke_with_build_args'
    /Users/zscott/projects/rubygems/lib/rubygems/command_manager.rb:167:in `process_args'
    /Users/zscott/projects/rubygems/lib/rubygems/command_manager.rb:137:in `run'
    /Users/zscott/projects/rubygems/lib/rubygems/gem_runner.rb:54:in `run'
Owner

drbrain commented May 19, 2014

If you can reproduce it can you run with this patch and --backtrace:

Member

zzak commented May 20, 2014

I got the following backtrace

Owner

drbrain commented May 20, 2014

What revision are you using?

Member

zzak commented May 20, 2014

@drbrain just to be sure i rebased and tried again

@zzak zzak closed this May 21, 2014

@drbrain drbrain reopened this May 21, 2014

Owner

drbrain commented May 21, 2014

I can finally reproduce this.

Reproduction requires having a prerelease rspec installed and not using gem install -i and using GEM_HOME=… gem install or installing into your default GEM_HOME instead.

drbrain added a commit that referenced this issue May 21, 2014

Add Resolver::DependencyRequest#match?
This new method only matches prerelease versions when this is a
prerelease dependency.

Part of #853

@drbrain drbrain closed this in 0f96754 May 21, 2014

drbrain added a commit that referenced this issue May 21, 2014

aspiers added a commit to aspiers/barclamp-pacemaker that referenced this issue Jun 19, 2014

fix Travis breakage
Travis started failing with:

  uninitialized constant RSpec::Matchers::BuiltIn::RaiseError::MatchAliases (NameError)

due to bundler installing rspec-expectations-2.99 which is a
pre-release:

  rubygems/rubygems#853

aspiers added a commit to aspiers/cookbook-pacemaker that referenced this issue Nov 7, 2014

fix Travis breakage
Travis started failing with:

  uninitialized constant RSpec::Matchers::BuiltIn::RaiseError::MatchAliases (NameError)

due to bundler installing rspec-expectations-2.99 which is a
pre-release:

  rubygems/rubygems#853

nikolai-b added a commit to cyclestreets/cyclescape-chef that referenced this issue Jan 9, 2016

Bump rubygems version
rubygems/rubygems#853 was causing
rack 2.0.0-alpha to be installed incorrectly
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment