Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unsychronized update of shared mutable state #3367

Closed
brixen opened this issue Jan 25, 2018 · 6 comments
Closed

Unsychronized update of shared mutable state #3367

brixen opened this issue Jan 25, 2018 · 6 comments

Comments

@brixen
Copy link

brixen commented Jan 25, 2018

Error Report

Questions

Please fill out answers to these questions, it'll help us figure out
why things are going wrong.

  • What did you do?

    I ran the command /source/rubinius/rubinius/build/rubinius/gems/bin/bundle install --jobs=40 in a new Rails 5 app.

  • What did you expect to happen?

    I expected Bundler to complete the install.

  • What happened instead?

    Instead, what happened was the install failed with the exception listed below.

  • Have you tried any solutions posted on similar issues in our issue tracker, stack overflow, or google?

    I examined the backtrace and noted that the following code @all_specs = Hash.new {|h, k| h[k] = EMPTY_SEARCH } located in lib/bundler/index.rb:32 updates the Hash instance with no synchronization.

  • Have you read our issues document, https://github.com/bundler/bundler/blob/master/doc/contributing/ISSUES.md?

    Yes.

Backtrace

Rubinius::ObjectBoundsExceededError: Tuple::copy_from: index 11 out of bounds for size 10
  core/hash.rb:224:in `insert'
  core/hash.rb:421:in `[]='
  /source/rubinius/rubinius/build/rubinius/gems/gems/bundler-1.16.1/lib/bundler/index.rb:32:in `initialize_copy'
  core/proc.rb:20:in `call'
  core/hash.rb:489:in `default'
  core/hash.rb:412:in `[]'
  /source/rubinius/rubinius/build/rubinius/gems/gems/bundler-1.16.1/lib/bundler/index.rb:52:in `search_all'
  /source/rubinius/rubinius/build/rubinius/gems/gems/bundler-1.16.1/lib/bundler/source/rubygems.rb:297:in `remotes_for_spec'
  /source/rubinius/rubinius/build/rubinius/gems/gems/bundler-1.16.1/lib/bundler/source/rubygems.rb:119:in `install'
  /source/rubinius/rubinius/build/rubinius/gems/gems/bundler-1.16.1/lib/bundler/installer/gem_installer.rb:56:in `install'
  /source/rubinius/rubinius/build/rubinius/gems/gems/bundler-1.16.1/lib/bundler/installer/gem_installer.rb:16:in `install_from_spec'
  /source/rubinius/rubinius/build/rubinius/gems/gems/bundler-1.16.1/lib/bundler/installer/parallel_installer.rb:162:in `do_install'
  /source/rubinius/rubinius/build/rubinius/gems/gems/bundler-1.16.1/lib/bundler/installer/parallel_installer.rb:153:in `worker_pool'
  core/proc.rb:20:in `call'
  /source/rubinius/rubinius/build/rubinius/gems/gems/bundler-1.16.1/lib/bundler/worker.rb:64:in `apply_func'
  /source/rubinius/rubinius/build/rubinius/gems/gems/bundler-1.16.1/lib/bundler/worker.rb:59:in `process_queue'
  core/kernel.rb:599:in `loop'
  /source/rubinius/rubinius/build/rubinius/gems/gems/bundler-1.16.1/lib/bundler/worker.rb:56:in `process_queue'
  /source/rubinius/rubinius/build/rubinius/gems/gems/bundler-1.16.1/lib/bundler/worker.rb:90:in `create_threads'
  core/proc.rb:20:in `call'

Environment

Bundler       1.16.1
  Platforms   ruby, x86_64-darwin-17, x86_64-rubinius-3.95
Ruby          2.3.1p0 (2018-01-25 revision 0) [x86_64-darwin17.4.0]
  Full Path   /source/rubinius/rubinius/bin/rbx
  Config Dir  /source/rubinius/rubinius/etc
RubyGems      2.7.4
  Gem Home    /Users/brianshirai/.gem/rbx/3.95.c1
  Gem Path    /Users/brianshirai/.gem/rbx/3.95.c1:/source/rubinius/rubinius/build/rubinius/gems
  User Path   /Users/brianshirai/.gem/rbx/2.3
  Bin Dir     /Users/brianshirai/.gem/rbx/3.95.c1/bin
OpenSSL
  Compiled    OpenSSL 1.0.2m  2 Nov 2017
  Loaded      OpenSSL 1.0.2m  2 Nov 2017
  Cert File   /usr/local/etc/openssl/cert.pem
  Cert Dir    /usr/local/etc/openssl/certs
Tools
  Git         2.14.1
  RVM         not installed
  rbenv       not installed
  chruby      0.3.9.1
Gem.ruby      /source/rubinius/rubinius/bin/rbx
bundle #!     /usr/bin/env rbx

Bundler Build Metadata

Built At          2017-12-21
Git SHA           0034ef341
Released Version  true

Bundler settings

gem.test
  Set for the current user (/Users/brianshirai/.bundle/config): false
gem.mit
  Set for the current user (/Users/brianshirai/.bundle/config): false
gem.coc
  Set for the current user (/Users/brianshirai/.bundle/config): true
build.eventmachine
  Set for the current user (/Users/brianshirai/.bundle/config): "--with-cppflags=-I/usr/local/opt/openssl/include"
jobs
  Set for your local app (/source/repros/test_rails_5.1.4/.bundle/config): "40"

Gemfile

Gemfile

source 'https://rubygems.org'

git_source(:github) do |repo_name|
  repo_name = "#{repo_name}/#{repo_name}" unless repo_name.include?("/")
  "https://github.com/#{repo_name}.git"
end


# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '~> 5.1.4'
# Use sqlite3 as the database for Active Record
gem 'sqlite3'
# Use Puma as the app server
gem 'puma', '~> 3.7'
# Use SCSS for stylesheets
gem 'sass-rails', '~> 5.0'
# Use Uglifier as compressor for JavaScript assets
gem 'uglifier', '>= 1.3.0'
# See https://github.com/rails/execjs#readme for more supported runtimes
# gem 'therubyracer', platforms: :ruby

# Use CoffeeScript for .coffee assets and views
gem 'coffee-rails', '~> 4.2'
# Turbolinks makes navigating your web application faster. Read more: https://github.com/turbolinks/turbolinks
gem 'turbolinks', '~> 5'
# Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder
gem 'jbuilder', '~> 2.5'
# Use Psych as the YAML engine, instead of Syck, so serialized data can be read safely from different rubies (see http://git.io/uuLVag)
gem 'psych', '~> 2.0', platforms: :rbx

# Use Redis adapter to run Action Cable in production
# gem 'redis', '~> 3.0'
# Use ActiveModel has_secure_password
# gem 'bcrypt', '~> 3.1.7'

# Use Capistrano for deployment
# gem 'capistrano-rails', group: :development


# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]

Gemfile.lock

GEM
  remote: https://rubygems.org/
  specs:
    actioncable (5.1.4)
      actionpack (= 5.1.4)
      nio4r (~> 2.0)
      websocket-driver (~> 0.6.1)
    actionmailer (5.1.4)
      actionpack (= 5.1.4)
      actionview (= 5.1.4)
      activejob (= 5.1.4)
      mail (~> 2.5, >= 2.5.4)
      rails-dom-testing (~> 2.0)
    actionpack (5.1.4)
      actionview (= 5.1.4)
      activesupport (= 5.1.4)
      rack (~> 2.0)
      rack-test (>= 0.6.3)
      rails-dom-testing (~> 2.0)
      rails-html-sanitizer (~> 1.0, >= 1.0.2)
    actionview (5.1.4)
      activesupport (= 5.1.4)
      builder (~> 3.1)
      erubi (~> 1.4)
      rails-dom-testing (~> 2.0)
      rails-html-sanitizer (~> 1.0, >= 1.0.3)
    activejob (5.1.4)
      activesupport (= 5.1.4)
      globalid (>= 0.3.6)
    activemodel (5.1.4)
      activesupport (= 5.1.4)
    activerecord (5.1.4)
      activemodel (= 5.1.4)
      activesupport (= 5.1.4)
      arel (~> 8.0)
    activesupport (5.1.4)
      concurrent-ruby (~> 1.0, >= 1.0.2)
      i18n (~> 0.7)
      minitest (~> 5.1)
      tzinfo (~> 1.1)
    arel (8.0.0)
    builder (3.2.3)
    coffee-rails (4.2.2)
      coffee-script (>= 2.2.0)
      railties (>= 4.0.0)
    coffee-script (2.4.1)
      coffee-script-source
      execjs
    coffee-script-source (1.12.2)
    concurrent-ruby (1.0.5)
    crass (1.0.3)
    erubi (1.7.0)
    execjs (2.7.0)
    ffi (1.9.18)
    globalid (0.4.1)
      activesupport (>= 4.2.0)
    i18n (0.9.1)
      concurrent-ruby (~> 1.0)
    jbuilder (2.7.0)
      activesupport (>= 4.2.0)
      multi_json (>= 1.2)
    loofah (2.1.1)
      crass (~> 1.0.2)
      nokogiri (>= 1.5.9)
    mail (2.7.0)
      mini_mime (>= 0.1.1)
    method_source (0.9.0)
    mini_mime (1.0.0)
    mini_portile2 (2.3.0)
    minitest (5.11.1)
    multi_json (1.13.1)
    nio4r (2.2.0)
    nokogiri (1.8.1)
      mini_portile2 (~> 2.3.0)
    psych (2.2.4)
    puma (3.11.2)
    rack (2.0.3)
    rack-test (0.8.2)
      rack (>= 1.0, < 3)
    rails (5.1.4)
      actioncable (= 5.1.4)
      actionmailer (= 5.1.4)
      actionpack (= 5.1.4)
      actionview (= 5.1.4)
      activejob (= 5.1.4)
      activemodel (= 5.1.4)
      activerecord (= 5.1.4)
      activesupport (= 5.1.4)
      bundler (>= 1.3.0)
      railties (= 5.1.4)
      sprockets-rails (>= 2.0.0)
    rails-dom-testing (2.0.3)
      activesupport (>= 4.2.0)
      nokogiri (>= 1.6)
    rails-html-sanitizer (1.0.3)
      loofah (~> 2.0)
    railties (5.1.4)
      actionpack (= 5.1.4)
      activesupport (= 5.1.4)
      method_source
      rake (>= 0.8.7)
      thor (>= 0.18.1, < 2.0)
    rake (12.3.0)
    rb-fsevent (0.10.2)
    rb-inotify (0.9.10)
      ffi (>= 0.5.0, < 2)
    sass (3.5.5)
      sass-listen (~> 4.0.0)
    sass-listen (4.0.0)
      rb-fsevent (~> 0.9, >= 0.9.4)
      rb-inotify (~> 0.9, >= 0.9.7)
    sass-rails (5.0.7)
      railties (>= 4.0.0, < 6)
      sass (~> 3.1)
      sprockets (>= 2.8, < 4.0)
      sprockets-rails (>= 2.0, < 4.0)
      tilt (>= 1.1, < 3)
    sprockets (3.7.1)
      concurrent-ruby (~> 1.0)
      rack (> 1, < 3)
    sprockets-rails (3.2.1)
      actionpack (>= 4.0)
      activesupport (>= 4.0)
      sprockets (>= 3.0.0)
    sqlite3 (1.3.13)
    thor (0.20.0)
    thread_safe (0.3.6)
    tilt (2.0.8)
    turbolinks (5.1.0)
      turbolinks-source (~> 5.1)
    turbolinks-source (5.1.0)
    tzinfo (1.2.4)
      thread_safe (~> 0.1)
    uglifier (4.1.3)
      execjs (>= 0.3.0, < 3)
    websocket-driver (0.6.5)
      websocket-extensions (>= 0.1.0)
    websocket-extensions (0.1.3)

PLATFORMS
  ruby

DEPENDENCIES
  coffee-rails (~> 4.2)
  jbuilder (~> 2.5)
  psych (~> 2.0)
  puma (~> 3.7)
  rails (~> 5.1.4)
  sass-rails (~> 5.0)
  sqlite3
  turbolinks (~> 5)
  tzinfo-data
  uglifier (>= 1.3.0)

BUNDLED WITH
   1.16.1
@segiddins
Copy link
Member

I think something like the following should fix that:

diff --git a/lib/bundler/definition.rb b/lib/bundler/definition.rb
index 0349af39f..5272becd8 100644
--- a/lib/bundler/definition.rb
+++ b/lib/bundler/definition.rb
@@ -270,7 +270,7 @@ module Bundler
         end
 
         double_check_for_index(idx, dependency_names)
-      end
+      end.freeze
     end
 
     # Suppose the gem Foo depends on the gem Bar.  Foo exists in Source A.  Bar has some versions that exist in both
diff --git a/lib/bundler/index.rb b/lib/bundler/index.rb
index 9166a9273..33a0a47fb 100644
--- a/lib/bundler/index.rb
+++ b/lib/bundler/index.rb
@@ -21,15 +21,15 @@ module Bundler
     def initialize
       @sources = []
       @cache = {}
-      @specs = Hash.new {|h, k| h[k] = {} }
-      @all_specs = Hash.new {|h, k| h[k] = EMPTY_SEARCH }
+      @specs = {}
+      @all_specs = {}
     end
 
     def initialize_copy(o)
       @sources = o.sources.dup
       @cache = {}
-      @specs = Hash.new {|h, k| h[k] = {} }
-      @all_specs = Hash.new {|h, k| h[k] = EMPTY_SEARCH }
+      @specs = {}
+      @all_specs = {}
 
       o.specs.each do |name, hash|
         @specs[name] = hash.dup
@@ -39,6 +39,13 @@ module Bundler
       end
     end
 
+    def freeze
+      @sources.each(&:freeze).freeze
+      @specs.freeze
+      @all_specs.freeze
+      super
+    end
+
     def inspect
       "#<#{self.class}:0x#{object_id} sources=#{sources.map(&:inspect)} specs.size=#{specs.size}>"
     end
@@ -49,7 +56,7 @@ module Bundler
     end
 
     def search_all(name)
-      all_matches = local_search(name) + @all_specs[name]
+      all_matches = local_search(name) + (@all_specs[name] || EMPTY_SEARCH)
       @sources.each do |source|
         all_matches.concat(source.search_all(name))
       end
@@ -102,7 +109,7 @@ module Bundler
     alias_method :[], :search
 
     def <<(spec)
-      @specs[spec.name][spec.full_name] = spec
+      (@specs[spec.name] ||= {})[spec.full_name] = spec
       spec
     end
 
@@ -182,7 +189,11 @@ module Bundler
   private
 
     def specs_by_name(name)
-      @specs[name].values
+      if s = @specs[name]
+        s.values
+      else
+        [] # must return a mutable array for compatibility with the other branch
+      end
     end
 
     def search_by_dependency(dependency, base = nil)
@@ -206,7 +217,8 @@ module Bundler
     EMPTY_SEARCH = [].freeze
 
     def search_by_spec(spec)
-      spec = @specs[spec.name][spec.full_name]
+      versions = @specs[spec.name]
+      spec = versions && versions[spec.full_name]
       spec ? [spec] : EMPTY_SEARCH
     end
   end

@colby-swandale
Copy link
Member

I consider Rubinius no longer active, so closing.

@brixen
Copy link
Author

brixen commented Apr 29, 2019

Rubinius is working just fine and this is a credible report of a concurrency bug in Bundler.

Fairly recently, I significantly updated the object allocation mechanisms to make it possible to track when threads are modifying objects the thread did not create (or has not been given ownership of). I’ll be adding more capabilities and tooling around this because nothing really exists for debugging concurrency issues in the Ruby space.

For reference, what is your guidance on reporting concurrency bugs in Bundler? Are you simply unconcerned about that class of bug?

@segiddins segiddins reopened this May 12, 2019
@segiddins
Copy link
Member

Definitely a bug we should fix.

@hsbt hsbt transferred this issue from rubygems/bundler Mar 14, 2020
@deivid-rodriguez
Copy link
Member

Is this reproducible these days?

@deivid-rodriguez
Copy link
Member

Closing due to lack of feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants