Unresolved specs during Gem::Specification.reset

[jordansissel]

WARN: Unresolved specs during Gem::Specification.reset:
      ffi (>= 0)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.

I don't know what causes this, and while it appears to cause no behavioral problems, the 4-line warning will cause users of my project to file bugs about it.

[tenderlove]

How are you getting this warning? I need more information than this to fix it.

[jordansissel]

Almost anything that invokes rubygems, it seems, though I'm not sure how to be more descriptive than that. However, what I can do is show a few examples! :)

% gem install whatever
WARN: Unresolved specs during Gem::Specification.reset:
      ffi (>= 0)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.

% bundle install
WARN: Unresolved specs during Gem::Specification.reset:
      ffi (>= 0)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
...

% gem --version
WARN: Unresolved specs during Gem::Specification.reset:
      ffi (>= 0)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
2.4.4

I see this most commonly under JRuby, if that helps:

• JRuby 1.7.16
• Rubygems 2.4.4

% gem list ffi
...
*** LOCAL GEMS ***

ffi (1.9.6 java, 1.9.3 java)
ffi-rzmq (2.0.1, 1.0.0, 0.9.3)
ffi-rzmq-core (1.0.3)

I can try reproducing this other ways or provide further data if you need it! Happy to help get this resolved.

[jordansissel]

Relevant backtrace after patching lib/ruby/shared/rubygems/specification.rb to emit things like caller for debugging:

/home/jls/.rvm/rubies/jruby-1.7.16/lib/ruby/shared/rubygems.rb:276:in `clear_paths'
/home/jls/.rvm/rubies/jruby-1.7.16/lib/ruby/shared/rubygems.rb:334:in `paths='
/home/jls/.rvm/rubies/jruby-1.7.16/lib/ruby/shared/rubygems.rb:926:in `use_paths'
/home/jls/projects/logstash/rakelib/gems.rake:6:in `(root)'

gems.rake line 5 & 6:

ENV["GEM_HOME"] = ENV["GEM_PATH"] = "build/bootstrap/"
Gem.use_paths(ENV["GEM_HOME"])

[jordansissel]

Bundler arrives at this warning from this path:

/home/jls/.rvm/rubies/jruby-1.7.16/lib/ruby/shared/rubygems.rb:276:in `clear_paths'
/home/jls/projects/logstash/build/bootstrap/gems/bundler-1.7.6/lib/bundler/rubygems_integration.rb:121:in `clear_paths'
/home/jls/projects/logstash/build/bootstrap/gems/bundler-1.7.6/lib/bundler.rb:416:in `configure_gem_home'
/home/jls/projects/logstash/build/bootstrap/gems/bundler-1.7.6/lib/bundler.rb:407:in `configure_gem_home_and_path'
/home/jls/projects/logstash/build/bootstrap/gems/bundler-1.7.6/lib/bundler.rb:91:in `configure'
/home/jls/projects/logstash/build/bootstrap/gems/bundler-1.7.6/lib/bundler.rb:152:in `definition'
/home/jls/projects/logstash/build/bootstrap/gems/bundler-1.7.6/lib/bundler/cli/install.rb:77:in `run'
/home/jls/projects/logstash/build/bootstrap/gems/bundler-1.7.6/lib/bundler/cli.rb:145:in `install'

[tenderlove]

Interesting. Can you reproduce the error when running against rubygems master? Inside a checkout of rubygems, do:

$ ruby -I lib -S gem whatever

The jruby specific RG path seems suspicious.

[jordansissel]

% git remote -v
origin  git://github.com/rubygems/rubygems.git (fetch)
% git rev-parse master
3fca844ebb49196e8e14a105515e508653d8b1b0

% ruby -I lib -S gem whatever
WARN: Unresolved specs during Gem::Specification.reset:
      ffi (>= 0)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
ERROR:  While executing gem ... (Gem::CommandLineError)
    Unknown command whatever

[drbrain]

You Gem.use_paths in gems.rake which ultimately calls Gem::Specification.reset I suspect that you have already activated some gems before you hit Gem.use_paths that depend on ffi but have not needed to activate it (which is why they are unresolved).

You should be able to address this by adjusting your require order. gem dep -R ffi will show you the gems that have FFI dependencies in the 'Used by' section.

[jordansissel]

@drbrain I'll buy that we are doing weird things (possibly in weird orders) in gems.rake and elsewhere. Thanks for the advice on tracking that down!

On the other hand simply invoking gem whatever directly from the command line seems to cause this as well. It's totally funky. It may be because JRuby ships with a kind of fake ffi gemspec with the core release and it's always being loaded somehow?

[jordansissel]

Myself, @colinsurprenant, and @ph are hitting this pretty regularly, so I expect we'll be directing some more energy into debugging this. I hope we are able to fix it, and if there's any code needed, I think we can help provide that.

More details as we have them :)

[mkristian]

@jordansissel with the next release 1.7.20 and already with 9k there is no default gem ffi anymore in jruby. before this a require 'openssl' did activated ffi due to jruby-openssl needed krypt which again depended on ffi (but never used it in the jruby case).

maybe worth looking a snapshot build of the jruby-1_7 first ;)

[skull-squadron]

We're also seeing this issue all over the place too in deployment-bundled rails apps and elsewhere. The logstash donkey-punch appears to silence it, but are not completely sold on it either as a long-term or trustworthy solution.

Furthermore, we've found requiring gems before bundler/setup also causes an issue similar to this (which makes sense).