Gems defined with extensions dirs conflict with diff versions

[jules2689]

Problem:

• Using ruby version X, two projects define some gem at differing versions.
• Stub Specification does not pass extension dir, which causes issues.

We saw this as

** Notice: CityHash was not loaded. **

For optimal performance, use of the cityhash gem is recommended.

Run the following command, or add it to your Gemfile:

gem install cityhash

Environment

Bundler   1.15.0.pre.1
Rubygems  2.6.11
Ruby      2.3.3p222 (2016-11-21 revision 56859) [x86_64-darwin16]
GEM_HOME  /Users/ryanmcilmoyl/.gem/ruby/2.3.3
GEM_PATH  /Users/ryanmcilmoyl/.gem/ruby/2.3.3:/opt/rubies/2.3.3/lib/ruby/gems/2.3.0
Git       2.11.0 (Apple Git-81)
Platform  x86_64-darwin-16

Bundler settings

jobs
  Set for your local app (/Users/ryanmcilmoyl/src/github.com/Shopify/shopify/.bundle/config): "4"

Relevant Gemfile

Shopify Project:

gem 'cityhash', git: 'csfrancis/cityhash', sha: 'XYZ'

Another Project

gem 'cityhash'

[segiddins]

see lib/bundler/rubygems_ext.rb, we need that impl of extension_dir for the stub spec, just replacing rg_extension_dir with stub.extension_dir

[jules2689]

To replicate:

Gemfile

source 'https://rubygems.org'
gem 'cityhash', git: 'https://github.com/csfrancis/cityhash.git', ref: '3cfc7d01f333c01811d5e834f1495eaa29f87c36'

test.rb

require 'bundler/setup'
puts $:.grep(/cityhash/)

Which outputs

/Users/juliannadeau/.gem/ruby/2.3.3/bundler/gems/cityhash-3cfc7d01f333/lib
/Users/juliannadeau/.gem/ruby/2.3.3/bundler/gems/extensions/x86_64-darwin-16/2.3.0-static/cityhash-0.6.0

That cityhash-0.6.0 should be a hash

[jules2689]

Ok, so looking into this more, stub_specification doesn't know what full_name should be because we don't have a record of the source, so it resolves to cityhash-0.6.0 rather than cityhash-3cfc7d01f333

[jules2689]

This seems to work:

diff --git a/lib/bundler/remote_specification.rb b/lib/bundler/remote_specification.rb
index d89abd9..0f2ec9c 100644
--- a/lib/bundler/remote_specification.rb
+++ b/lib/bundler/remote_specification.rb
@@ -12,7 +12,7 @@ module Bundler

     attr_reader :name, :version, :platform
     attr_writer :dependencies
-    attr_accessor :source, :remote
+    attr_accessor :remote

     def initialize(name, version, platform, spec_fetcher)
       @name         = name
@@ -76,11 +76,6 @@ module Bundler
       @dependencies || method_missing(:dependencies)
     end

-    def git_version
-      return unless loaded_from && source.is_a?(Bundler::Source::Git)
-      " #{source.revision[0..6]}"
-    end
-
   private

     def to_ary
diff --git a/lib/bundler/rubygems_ext.rb b/lib/bundler/rubygems_ext.rb
index a0f8fa8..bc32d2e 100644
--- a/lib/bundler/rubygems_ext.rb
+++ b/lib/bundler/rubygems_ext.rb
@@ -23,6 +23,27 @@ require "bundler/match_platform"
 module Gem
   @loaded_stacks = Hash.new {|h, k| h[k] = [] }

+  class StubSpecification
+    attr_accessor :source
+
+    def git_version
+      return unless loaded_from && source.is_a?(Bundler::Source::Git)
+      " #{source.revision[0..6]}"
+    end
+
+    if method_defined?(:extension_dir)
+      alias_method :rg_extension_dir, :extension_dir
+      def extension_dir
+        @bundler_extension_dir ||= if git_version
+          ext_name = [name, git_version.strip].join('-')
+          File.expand_path(File.join(extensions_dir, ext_name))
+        else
+          rg_extension_dir
+        end
+      end
+    end
+  end
+
   class Specification
     attr_accessor :remote, :location, :relative_loaded_from

diff --git a/lib/bundler/stub_specification.rb b/lib/bundler/stub_specification.rb
index bc1b18f..c0b6232 100644
--- a/lib/bundler/stub_specification.rb
+++ b/lib/bundler/stub_specification.rb
@@ -9,7 +9,12 @@ module Bundler
       spec
     end

-    attr_accessor :stub, :ignored
+    attr_accessor :stub, :ignored, :source
+
+    def source=(source)
+      stub.source = source
+      @source = source
+    end

[segiddins]

We have a spec for extensions being usable, and I can't reproduce this issue...

[jules2689]

I think the specs don't cover all cases. Using #5594 (comment) between 1.14.6 and 1.15.0.pre.1, I get this:

~/src/github.com/Shopify/bund ➜ bundle exec ruby test.rb
1.14.6
/Users/juliannadeau/.gem/ruby/2.3.3/bundler/gems/cityhash-3cfc7d01f333/lib
/Users/juliannadeau/.gem/ruby/2.3.3/bundler/gems/extensions/x86_64-darwin-16/2.3.0-static/cityhash-3cfc7d01f333

~/src/github.com/Shopify/bund ➜ bundle exec ruby test.rb
1.15.0.pre.1
/Users/juliannadeau/.gem/ruby/2.3.3/bundler/gems/cityhash-3cfc7d01f333/lib
/Users/juliannadeau/.gem/ruby/2.3.3/bundler/gems/extensions/x86_64-darwin-16/2.3.0-static/cityhash-0.6.0