Skip to content
This repository has been archived by the owner on Apr 14, 2021. It is now read-only.

--jobs > 1 raises Bundler::GemspecError on Rubinius #3269

Closed
jc00ke opened this issue Nov 22, 2014 · 13 comments
Closed

--jobs > 1 raises Bundler::GemspecError on Rubinius #3269

jc00ke opened this issue Nov 22, 2014 · 13 comments

Comments

@jc00ke
Copy link
Contributor

jc00ke commented Nov 22, 2014

~/projects/travis-rbx-bundler-debug master | 4e19626*  33s
$> rbx -v
rubinius 2.3.0.n313 (2.1.0 21267107 2014-11-09 3.4.2 JI) [x86_64-linux-gnu]

~/projects/travis-rbx-bundler-debug master | 4e19626*  
$> bundler --version
Bundler version 1.7.6

~/projects/travis-rbx-bundler-debug master | 4e19626*  27s
$> bundle
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Using rake 10.3.2
Using multi_json 1.10.1
Using i18n 0.6.11
Using builder 3.0.4
Using erubis 2.7.0
Using journey 1.0.4
Using rack 1.4.5
Using hike 1.2.3
Using mime-types 1.25.1
Using polyglot 0.3.5
Using tilt 1.4.1
Using arel 3.0.3
Using bundler 1.7.6
Using htmlentities 4.3.1
Using json 1.8.1
Using nokogiri 1.5.11
Using thor 0.19.1
Using activesupport 3.2.19
Using rack-cache 1.2
Using rack-test 0.6.2
Using rack-ssl 1.3.4
Using puma 2.6.0
Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/sprockets-2.2.3.gem. It may be corrupted.
Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/pg-0.17.1.gem. It may be corrupted.
Using treetop 1.4.15
Using rdoc 3.12.2
Using activemodel 3.2.19

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/tzinfo-0.3.42.gem. It may be corrupted.
An error occurred while installing sprockets (2.2.3), and Bundler cannot continue.
Make sure that `gem install sprockets -v '2.2.3'` succeeds before bundling.

~/projects/travis-rbx-bundler-debug master | 4e19626*  17s
$> bundle --jobs=1
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Using rake 10.3.2
Using i18n 0.6.11
Using multi_json 1.10.1
Using activesupport 3.2.19
Using builder 3.0.4
Using activemodel 3.2.19
Using erubis 2.7.0
Using journey 1.0.4
Using rack 1.4.5
Using rack-cache 1.2
Using rack-test 0.6.2
Using hike 1.2.3
Using tilt 1.4.1
Installing sprockets 2.2.3
Using actionpack 3.2.19
Using mime-types 1.25.1
Using polyglot 0.3.5
Using treetop 1.4.15
Using mail 2.5.4
Using actionmailer 3.2.19
Using arel 3.0.3
Installing tzinfo 0.3.42
Using activerecord 3.2.19
Using activeresource 3.2.19
Using bundler 1.7.6
Using htmlentities 4.3.1
Using json 1.8.1
Using nokogiri 1.5.11
Using rack-ssl 1.3.4
Using rdoc 3.12.2
Using thor 0.19.1
Using railties 3.2.19
Using paper_trail 2.7.2
Installing pg 0.17.1
Using puma 2.6.0
Using rails 3.2.19
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

And, after I updated calagator to use Bundler 1.7.7 I'm still seeing the same errors.

Some gems start to install, then some fail.

I decided to delete my Gemfile.lock and regen it with rbx. Check out the failures below:

~/projects/calagator rbx-bundler-travis | 1cecde9  
$> bundle
Fetching gem metadata from https://rubygems.org/........
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies...................
Using rake 10.3.2
Using i18n 0.6.11
Using multi_json 1.10.1
Using builder 3.0.4
Using erubis 2.7.0
Using journey 1.0.4
Using hike 1.2.3
Using rack 1.4.5
Using tilt 1.4.1
Using polyglot 0.3.5
Using mime-types 1.25.1
Using tzinfo 0.3.42
Using arel 3.0.3
Using bundler 1.7.7
Using json 1.8.1                                                                                                                      [60/2265]
Using thor 0.19.1

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/coderay-1.1.0.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/debug_inspector-0.0.2.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/bluecloth-2.2.0.gem. It may be corrupted.
Using nokogiri 1.5.11

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/colorize-0.7.3.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/net-ssh-2.9.1.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/cliver-0.3.2.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/columnize-0.3.6.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/netrc-0.8.0.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/simplecov-html-0.8.0.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/tins-1.3.3.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/diff-lcs-1.2.5.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/execjs-2.2.2.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/ffi-1.9.6.gem. It may be corrupted.
Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/highline-1.6.21.gem. It may be corrupted.               [30/2265]
Using htmlentities 4.3.1

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/multi_xml-0.5.5.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/docile-1.1.5.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/lucene_query-0.1.gem. It may be corrupted.
Using mofo 0.2.8 from source at vendor/gems/mofo-0.2.8

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/database_cleaner-1.3.0.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/options-2.3.2.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/libv8-3.16.14.7-x86_64-linux.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/websocket-driver-0.4.0.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/pr_geohash-1.0.0.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/ref-1.0.5.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/rack-robustness-1.1.0.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/rspec-support-3.1.2.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/ri_cal-0.8.8.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/rubyzip-1.1.6.gem. It may be corrupted.
undler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/sass-3.2.19.gem. It may be corrupted.                    [0/2265]

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/will_paginate-3.0.5.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/spring-1.1.3.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/sqlite3-1.3.10.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/timecop-0.7.1.gem. It may be corrupted.
Using activesupport 3.2.19

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/utf8-cleaner-0.0.9.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/geokit-1.6.5.gem. It may be corrupted.
Using rack-cache 1.2
Using rack-test 0.6.2
Using sprockets 2.2.3
Using rack-ssl 1.3.4
Using puma 2.6.0
Using treetop 1.4.15
Using rdoc 3.12.2

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/rsolr-1.0.10.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/sunspot_solr-2.1.1.gem. It may be corrupted.
An error occurred while installing coderay (1.1.0), and Bundler cannot continue.
Make sure that `gem install coderay -v '1.1.0'` succeeds before bundling.

--jobs=1 worked locally though

~/projects/calagator rbx-bundler-travis | 1cecde9  1m 13s
$> bundle --jobs=1
Fetching gem metadata from https://rubygems.org/........
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies.........
Using rake 10.3.2
Using i18n 0.6.11
Using multi_json 1.10.1
Using activesupport 3.2.19
Using builder 3.0.4
Using activemodel 3.2.19
Using erubis 2.7.0
Using journey 1.0.4
Using rack 1.4.5
Using rack-cache 1.2
Using rack-test 0.6.2
Using hike 1.2.3
Using tilt 1.4.1
Using sprockets 2.2.3
Using actionpack 3.2.19
Using mime-types 1.25.1
Using polyglot 0.3.5
Using treetop 1.4.15
Using mail 2.5.4
Using actionmailer 3.2.19
Using arel 3.0.3
Using tzinfo 0.3.42
Using activerecord 3.2.19                                                                                                              [4/2324]
Using activeresource 3.2.19
Using bundler 1.7.7
Using rack-ssl 1.3.4
Using json 1.8.1
Using rdoc 3.12.2
Using thor 0.19.1
Using railties 3.2.19
Using rails 3.2.19
Installing acts-as-taggable-on 2.4.1
Installing coderay 1.1.0
Installing better_errors 1.1.0
Installing debug_inspector 0.0.2
Installing binding_of_caller 0.7.2
Installing bluecloth 2.2.0
Installing colorize 0.7.3
Installing net-ssh 2.9.1
Installing net-scp 1.2.1
Installing sshkit 1.5.1
Installing capistrano 3.0.1
Installing capistrano-bundler 1.0.0
Installing capistrano-rails 1.0.0
Using nokogiri 1.5.11
Installing xpath 2.0.0
Installing capybara 2.4.3
Installing cliver 0.3.2
Installing columnize 0.3.6
Installing netrc 0.8.0
Installing rest-client 1.7.2
Installing docile 1.1.5
Installing simplecov-html 0.8.0
Installing simplecov 0.9.1
Installing tins 1.3.3
Installing term-ansicolor 1.3.0
Installing coveralls 0.7.0
Installing database_cleaner 1.3.0
Installing diff-lcs 1.2.5
Installing exception_notification 2.6.1
Installing execjs 2.2.2
Installing factory_girl 4.5.0
Installing factory_girl_rails 4.5.0
Installing ffi 1.9.6
Installing font-awesome-rails 3.2.1.3
Installing formtastic 2.2.1
Installing geokit 1.6.5
Installing highline 1.6.21
Using htmlentities 4.3.1
Installing multi_xml 0.5.5
Installing httparty 0.11.0
Installing jquery-rails 1.0.19
Installing libnotify 0.8.4
Installing libv8 3.16.14.7
Installing loofah 1.2.1
Installing loofah-activerecord 1.1.0
Installing lucene_query 0.1
Using mofo 0.2.8 from source at vendor/gems/mofo-0.2.8
Installing options 2.3.2
Using paper_trail 2.7.2
Installing will_paginate 3.0.5
Installing paper_trail_manager 0.2.0
Installing websocket-driver 0.4.0
Installing poltergeist 1.5.1
Installing pr_geohash 1.0.0
Installing progress_bar 1.0.0
Using puma 2.6.0
Installing rack-robustness 1.1.0
Installing rails_autolink 1.1.3
Installing ref 1.0.5
Installing ri_cal 0.8.8
Installing rsolr 1.0.10
Installing rspec-support 3.1.2
Installing rspec-mocks 3.1.3
Installing rspec-activemodel-mocks 1.0.1
Installing rspec-core 3.1.7
Installing rspec-expectations 3.1.2
Installing rspec-its 1.1.0
Installing rspec-rails 3.1.0
Installing rubyzip 1.1.6
Installing sass 3.2.19
Installing sass-rails 3.2.6
Installing spring 1.1.3
Installing spring-commands-rspec 1.0.2
Installing sqlite3 1.3.10
Installing sunspot 2.1.1
Installing sunspot_rails 2.1.1
Installing sunspot_solr 2.1.1
Installing therubyracer 0.12.1
Installing timecop 0.7.1
Installing uglifier 2.5.3
Installing utf8-cleaner 0.0.9
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.
Post-install message from capistrano:
If you're updating Capistrano from 2.x.x, we recommend you to read the upgrade guide: http://www.capistranorb.com/documentation/upgrading/
Post-install message from httparty:
When you HTTParty, you must party hard!

Hopefully some of this helps. I'll try to find some time to remove all of my bundler gem caches per the ISSUES doc and verify the errors above. Will also verify on 1.7.7.

@Who828
Copy link
Contributor

Who828 commented Nov 22, 2014

Can you update to the latest rubygems version and try again?

@jc00ke
Copy link
Contributor Author

jc00ke commented Nov 24, 2014

 ~/p/travis-rbx-bundler-debug   master ±  bundle --version
Bundler version 1.7.7
 ~/p/travis-rbx-bundler-debug   master ±  bundle
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Using rake 10.3.2

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/builder-3.0.4.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/multi_json-1.10.1.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/i18n-0.6.11.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/erubis-2.7.0.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/journey-1.0.4.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/hike-1.2.3.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/tilt-1.4.1.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/mime-types-1.25.1.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/polyglot-0.3.5.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/rack-1.4.5.gem. It may be corrupted.
Using bundler 1.7.7

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/arel-3.0.3.gem. It may be corrupted.
Using json 1.8.1
Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/htmlentities-4.3.1.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/tzinfo-0.3.42.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/nokogiri-1.5.11.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/thor-0.19.1.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/pg-0.17.1.gem. It may be corrupted.
An error occurred while installing builder (3.0.4), and Bundler cannot continue.
Make sure that `gem install builder -v '3.0.4'` succeeds before bundling.

😦

@Who828
Copy link
Contributor

Who828 commented Nov 24, 2014

@jc00ke I am not asking for the bundler version but for the rubygems version, can you upgrade it to the latest one? Also please give output of

bundle env

@jc00ke
Copy link
Contributor Author

jc00ke commented Nov 24, 2014

Ah, sorry, missed that. Same thing though

 ~/p/travis-rbx-bundler-debug   master ±  bundle env
Bundler 1.7.4
Ruby 2.1.0 (2014-11-09 patchlevel 0) [x86_64-linux-gnu]
Rubygems 2.4.4
GEM_HOME /home/jesse/.gem/rbx/2.1.0
GEM_PATH /home/jesse/.gem/rbx/2.1.0:/home/jesse/projects/rubinius/rubinius/gems

Bundler settings
  jobs
    Set for your local app (/home/jesse/projects/travis-rbx-bundler-debug/.bundle/config): "4"
    Set for the current user (/home/jesse/.bundle/config): "4"


Gemfile
source 'https://rubygems.org'

gem 'pg'
gem 'puma'

# Run-time dependencies
gem 'rails', '3.2.19'
gem 'nokogiri', '1.5.11'
gem 'rdoc', '3.12.2', :require => false
gem 'htmlentities', '4.3.1'
gem 'paper_trail', '2.7.2'


Gemfile.lock
<No Gemfile.lock found>


 ~/p/travis-rbx-bundler-debug   master ±  bundle
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies....
Using rake 10.3.2

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/builder-3.0.4.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/multi_json-1.10.1.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/erubis-2.7.0.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/i18n-0.6.11.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/journey-1.0.4.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/hike-1.2.3.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/mime-types-1.25.1.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/tilt-1.4.1.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/polyglot-0.3.5.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/arel-3.0.3.gem. It may be corrupted.
Using bundler 1.7.4

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/rack-1.4.5.gem. It may be corrupted.
Using json 1.8.1
Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/htmlentities-4.3.1.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/tzinfo-0.3.42.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/nokogiri-1.5.11.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/thor-0.19.1.gem. It may be corrupted.

Bundler::GemspecError: Could not read gem at /home/jesse/.gem/rbx/2.1.0/cache/pg-0.17.1.gem. It may be corrupted.
An error occurred while installing builder (3.0.4), and Bundler cannot continue.
Make sure that `gem install builder -v '3.0.4'` succeeds before bundling.

@Who828
Copy link
Contributor

Who828 commented Nov 25, 2014

This is strange, I ran bundle install on same Gemfile with JRuby and I didn't see the errors (while they do occur with Rubinius). Both JRuby and Rubinius use the same code in Bundler for parallel jobs, so it seems like the problem is somewhere else.

@jc00ke
Copy link
Contributor Author

jc00ke commented Nov 25, 2014

@brixen FYI

@accgetter
Copy link

I have same problem.
rm gem file that "It may be corrupted" and then bundle install.
Problem was solved

@indirect
Copy link
Member

Since I can't reproduce this on JRuby, which also lacks a GIL, I'm not sure what to do. Are we missing a race condition? Does JRuby have some lock that isn't global but protects against this problem? Totally open to input here, but since I'm personally unfamiliar with Rubinius, I have no idea how to even try to fix this.

@jc00ke
Copy link
Contributor Author

jc00ke commented Dec 1, 2014

I've seen other issues that suggest it's something to do with zlib but I can't find them at the moment. I'll try to find those issues.

@brixen
Copy link
Contributor

brixen commented Dec 2, 2014

I've identified and fixed one issue involved here: fix: rubinius/rubinius@7772d05 and specs: rubinius/rubinius@5be264f

Troubleshooting this was made significantly harder by code like this in Bundler: https://github.com/bundler/bundler/blob/master/lib/bundler/rubygems_integration.rb#L176-L177

There's no good reason to write code like this, so please consider not doing so. At the very least, included the original exception kind and message somewhere.

There is very likely still a race condition somewhere here or we would not be seeing different behavior with --jobs=4. I'm continuing to debug this.

A couple other points:

  1. Whether or not JRuby has a GIL is not dispositive of whether a race condition exists in RubyGems or Bundler. "It works on JRuby" is exactly the same as "It works on MRI" when it comes to threading issues. Race conditions do not magically manifest in the absence of a GIL. Anytime there is any concurrency, there is the possibility of a race condition. Whatever locking strategy is used and whatever the particular configuration of those lock does not per se determine whether the absence of a race condition is a fact. To be even more simple: "Absence of evidence is not evidence of absence."
  2. Debugging a threading issue, like a race condition, is 99% the same as debugging any other issue. In other words, one proceeds by asking the same questions: "Where am I in the code?", "How did I get here?", "What values are here?", "Where do these values come from?", "What exactly is the error message or condition?". The only thing that changes is that at any point in the code, more than one thread of execution could be running.
  3. As for the question of where to start, the answer is, with the error message. It took 2 minutes to grep the Bundler code and identify the location of the error message. That action dictated most of my next actions, which basically amounted to trying to get information out of the program that Bundler purposely obscured. So, Q. Where do I start? A. With the error message.

I'm not trying to lecture anyone, but it's frustrating to constantly hear things like, "It works on MRI." or "I'm unfamiliar with Rubinius." We're trying to make Ruby better. If people want to have a better Ruby, then we need people to do the bare minimum to help look at issues. Threading bugs are not terribly scary, but they may look different superficially. Underneath, it's the same debugging we all do every time a program doesn't behave as expected.

@indirect
Copy link
Member

indirect commented Dec 2, 2014

Thanks for letting us know! Because the bundle command is user-facing, and many of its users are not developers, rescuing errors and attempting to explain what is going on is by far the best option that we have available. We try to also print the exception and backtrace in verbose/debug mode, but it looks like that particular exception case is missing the line of code to do that. I'll get that fixed soon.

In the bigger scheme of things, I am of course happy to dig in to debugging things myself when I have time to do so. Since it does work on MRI and JRuby, and I am unfamiliar with Rubinius, it takes more time to work on this sort of bug than most other open issues. I'm especially short on time for that at the moment, which is why I asked if anyone else had ideas. Thanks again for your work on this.

@brixen
Copy link
Contributor

brixen commented Dec 5, 2014

@indirect I've released 2.4.1, which appears to run with --jobs > 1 correctly.

I still don't know what the race was that would result in the constant being an unresolved autoload instance in the concurrent case. Perhaps the race is benign and only resulted in different orderings of loading a particular constant. As you can see from this gist, the autoload issue was hit while somewhere inside Psych https://gist.github.com/brixen/a5971cc930abd2ace163.

I'm going to ask Travis to re-enable --jobs for Rubinius and see if we observe any other issue. I think the behavior we have observed is that a race exists. I don't know where it is. I don't know that it will result in pathological execution. That said, I'll leave it up to you whether to close this ticket or not.

@TimMoore
Copy link
Contributor

I'll assume that no news is good news and that there haven't been further issues after re-enabling --jobs on Travis. Please comment if you know otherwise and we can reopen.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants