RUBYGEMS_GEMDEPS and git source bug. #822

Closed
ghost opened this Issue Feb 12, 2014 · 18 comments

Comments

Projects
None yet
2 participants
@ghost

ghost commented Feb 12, 2014

this one issue might be less helpful because i can't share the Gemfile that causes it. two things to note is that it has two gem sources (rubygems and a private gem server) and that it also uses git sources for gems.

gem -v
/Users/robert/.rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/request_set/lockfile.rb:352:in `parse_GIT': undefined method `add_dependency' for #<Gem::Resolver::GitSpecification:0x007fe98a1983d8> (NoMethodError)
        from /Users/robert/.rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/request_set/lockfile.rb:222:in `parse'
        from /Users/robert/.rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/request_set.rb:228:in `load_gemdeps'
        from /Users/robert/.rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems.rb:1031:in `use_gemdeps'
        from /Users/robert/.rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems.rb:1216:in `<top (required)>'
        from <internal:gem_prelude>:1:in `require'
        from <internal:gem_prelude>:1:in `<compiled>'

suan commented Mar 14, 2014

I have this same exact issue and have a similar Gemfile as well

Owner

drbrain commented Apr 3, 2014

Can either of you send an example Gemfile and Gemfile.lock that was anonymized?

Replacing the private gem names with "a", "b", "c", etc. and the private sources with http://a.example, etc. should allow me to fix this bug faster than trying to reproduce it properly myself.

@drbrain drbrain added this to the 2.3 milestone Apr 3, 2014

@ghost

ghost commented Apr 4, 2014

i can do that but it'll be by the end of today before i can. it could be related to #822. the error looks almost identical.

@ghost

ghost commented Apr 4, 2014

never-mind my last comment, i linked back to the same issue.

@ghost

ghost commented Apr 5, 2014

i think there might be bugs with this feature in general, i tried again with the private Gemfile and got a totally different error. in an effort to reproduce, i created a minimal Gemfile that didn't reproduce the error but also had strange behavior:

➜  h0h0  bundle
Using coderay (1.1.0)
Using method_source (0.8.2)
Using slop (3.5.0)
Using pry (0.10.0.pre) from git@github.com:pry/pry.git (at master)
Using yard (0.8.7.4)
Using pry-doc (0.6.0)
Using bundler (1.5.3)
Your bundle is complete!
It was installed into ./vendor
➜  h0h0  export RUBYGEMS_GEMDEPS=-
➜  h0h0  gem -v
2.2.2
➜  h0h0  pry
/Users/giorgio/.rubies/ruby-1.9.3-p484/lib/ruby/site_ruby/1.9.1/rubygems/specification.rb:2052:in `raise_if_conflicts': can't activate pry-0.9.12.6, already activated pry-0.10.0.pre (Gem::LoadError)
        from /Users/giorgio/.rubies/ruby-1.9.3-p484/lib/ruby/site_ruby/1.9.1/rubygems/specification.rb:1262:in `activate'
        from /Users/giorgio/.rubies/ruby-1.9.3-p484/lib/ruby/site_ruby/1.9.1/rubygems/core_ext/kernel_gem.rb:54:in `gem'
        from /Users/giorgio/.gem/ruby/1.9.3/bin/pry:22:in `<main>'

pry 0.10 is in my Gemfile and i have 0.9.12 of the gem (outside bundler). this seems to confuse rubygems/RUBYGEMS_GEMDEPS. my Gemfile:

source "https://rubygems.org"
gem "pry-doc"
gem "pry", git: "git@github.com:pry/pry.git"

the original error that led me to create that Gemfile is here, i think it has been seen before and seems to relate to git, but in a different way, there's no NoMethodError this time:

/Users/giorgio/.rubies/ruby-1.9.3-p484/lib/ruby/site_ruby/1.9.1/rubygems/request_set/lockfile.rb:202:in `get': unexpected token [:entry, "ref"], expected [:entry, "specs"] (at line 3 column 2) (Gem::RequestSet::Lockfile::ParseError)
        from /Users/giorgio/.rubies/ruby-1.9.3-p484/lib/ruby/site_ruby/1.9.1/rubygems/request_set/lockfile.rb:340:in `parse_GIT'
@ghost

ghost commented Apr 5, 2014

i see how that happens. pry-doc depends on pry ~> 0.9, but this can also happen:

➜  h0h0  chruby 1.9
fatal: Not a git repository (or any of the parent directories): .git
fatal: Not a git repository (or any of the parent directories): .git
fatal: Not a git repository (or any of the parent directories): .git
fatal: repository '/Users/giorgio/.rubies/ruby-1.9.3-p484/lib/ruby/gems/1.9.1/cache/bundler/git/pry-bb70c2dcae0a0dbc5af20aaa5ecbc19903347a6f' does not exist
fatal: Not a git repository (or any of the parent directories): .git
/Users/giorgio/.rubies/ruby-1.9.3-p484/lib/ruby/site_ruby/1.9.1/rubygems/source/git.rb:101:in `chdir': No such file or directory - /Users/giorgio/.rubies/ruby-1.9.3-p484/lib/ruby/gems/1.9.1/bundler/gems/pry- (Errno::ENOENT)
➜  h0h0  unset RUBYGEMS_GEMDEPS
➜  h0h0  chruby 1.9

pretty much there seems to be a million different ways to break this.

@ghost

ghost commented Apr 5, 2014

even with rubygems HEAD, error on private Gemfile is the same:

RUBYGEMS_GEMDEPS=- ruby -I rubygems/lib rubygems/bin/gem -v
/Users/giorgio/code/xx/rubygems/lib/rubygems/request_set/lockfile.rb:202:in `get': unexpected token [:entry, "ref"], expected [:entry, "specs"] (at line 3 column 2) (Gem::RequestSet::Lockfile::ParseError)
@ghost

ghost commented Apr 5, 2014

sorry i can't provide a better reproduction, i am trying but the gemfile is quite big and nothing minimal has worked so far.

@ghost

ghost commented Apr 5, 2014

minimal reproduction that works for me on HEAD.

Gemfile

source 'https://rubygems.org'
gem 'pry', git: 'git://github.com/pry/pry.git', ref: 'fce269f3769bb00b83b70260be0dc7d2e68d28df'
$ bundle
Fetching git://github.com/pry/pry.git
Fetching gem metadata from https://rubygems.org/..........
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies...
Using coderay 1.1.0
Using method_source 0.8.2
Using slop 3.5.0
Using pry 0.10.0.pre from git://github.com/pry/pry.git (at fce269f)
Using bundler 1.6.0
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.
$ RUBYGEMS_GEMDEPS=- ruby -I ../lib ../bin/gem -v
/Users/giorgio/code/xx/rubygems/lib/rubygems/request_set/lockfile.rb:202:in `get': unexpected token [:entry, "ref"], expected [:entry, "specs"] (at line 3 column 2) (Gem::RequestSet::Lockfile::ParseError)
        from /Users/giorgio/code/xx/rubygems/lib/rubygems/request_set/lockfile.rb:344:in `parse_GIT'
        from /Users/giorgio/code/xx/rubygems/lib/rubygems/request_set/lockfile.rb:222:in `parse'

@ghost

ghost commented Apr 5, 2014

this is true for branche's as well, there's also http://bundler.io/v1.5/git.html for a list of possibles.

RUBYGEMS_GEMDEPS=- ruby -I ../lib ../bin/gem -v
/Users/giorgio/code/xx/rubygems/lib/rubygems/request_set/lockfile.rb:202:in `get': unexpected token [:entry, "branch"], expected [:entry, "specs"] (at line 3 column 2) (Gem::RequestSet::Lockfile::ParseError)

drbrain added a commit that referenced this issue Apr 7, 2014

Add support for ref: in gem.deps.rb.lock for git
This support was omitted.  The ref: doesn't seem to do anything useful
in bundler (its only fetched for an equality comparison) so it is
ignored here.

Bug #822

drbrain added a commit that referenced this issue Apr 7, 2014

Add support for git branch: in gem.deps.rb.lock
This support was omitted.  Like ref: this appears to be ignored (see
@56aa22c).

Bug #822

drbrain added a commit that referenced this issue Apr 7, 2014

Add support for git tag: in gem.deps.rb.lock
This support was omitted.  Like ref: this appears to be ignored (see
@56aa22c).

Bug #822
Owner

drbrain commented Apr 7, 2014

Please try master (@aa1b3aa) I've added support for the (seemingly nearly ignored) ref, branch, and tag entries.

While the syntax of the Gemfile is documented, the Gemfile.lock is not. I haven't returned to my work on getting the Bundler tests running atop RubyGems to find the rest of these bugs.

Thanks a bunch for your patience and support!

@ghost

ghost commented Apr 8, 2014

we're getting there, there's a new parse error this time, on remote.

RUBYGEMS_GEMDEPS=- ruby -I rubygems/lib rubygems/bin/gem -v
/Users/giorgio/code/xx/rubygems/lib/rubygems/request_set/lockfile.rb:202:in `get': unexpected token [:entry, "remote"], expected [:entry, "specs"] (at line 9 column 2) (Gem::RequestSet::Lockfile::ParseError)

this doesn't seem to be an option but something bundler can add itself(not sure). if you need a further reproduction let me know.

drbrain added a commit that referenced this issue Apr 15, 2014

@ghost

ghost commented Apr 18, 2014

@drbrain i'll work on getting a further reproduction then.

Owner

drbrain commented Apr 18, 2014

I think #851 has a reproduction, I added @aeaf89a to address it but have concerns

@ghost

ghost commented Apr 18, 2014

alright, i'll give it a shot.

@ghost

ghost commented Apr 18, 2014

i can't speak much about the concerns you have but #851 is no longer an issue for me, gem runs fine. i bumped the version to be sure i have the right one:

$ ruby -I rubygems/lib/ rubygems/bin/gem -v
2.2.3

there's some issues around version requirements though, i'm not sure how this happens but it does:

$ pry
../lib/rubygems/specification.rb:2075:in `check_version_conflict': can't activate pry-0.9.12.6, already activated pry-0.9.12.3 (Gem::LoadError)

last time it happened it appeared to be because one dependency had a requirement that matched an older version of pry(which i had installed), and somewhere along the way that seems to cause an issue. otherwise, works good!

Owner

drbrain commented Apr 18, 2014

If you can create a Gemfile and that reproduces the pry problem can you open a new issue? Please include gem list so I can install the related gems to track it down.

I'm going to close this issue since the Gemfile.lock problems appear to be resolved.

@ghost

ghost commented Apr 18, 2014

sure, i'll do that.

@drbrain drbrain closed this Apr 18, 2014

zzak added a commit to zzak/rubygems that referenced this issue May 9, 2014

Add support for ref: in gem.deps.rb.lock for git
This support was omitted.  The ref: doesn't seem to do anything useful
in bundler (its only fetched for an equality comparison) so it is
ignored here.

Bug #822

zzak added a commit to zzak/rubygems that referenced this issue May 9, 2014

Add support for git branch: in gem.deps.rb.lock
This support was omitted.  Like ref: this appears to be ignored (see
@56aa22c).

Bug #822

zzak added a commit to zzak/rubygems that referenced this issue May 9, 2014

Add support for git tag: in gem.deps.rb.lock
This support was omitted.  Like ref: this appears to be ignored (see
@56aa22c).

Bug #822

zzak added a commit to zzak/rubygems that referenced this issue May 9, 2014

drbrain added a commit that referenced this issue May 28, 2014

Use commit when resetting git sources
The git reset command accepts only commits, not references, so parse the
reference before checking out the branch.  (Thanks bundler sources for
making this easier to figure out!)

Fixes #931

Fixes #822
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment