dead lock on multi-threaded code? #987

Closed
ko1 opened this Issue Jun 8, 2011 · 6 comments

Comments

Projects
None yet
4 participants

ko1 commented Jun 8, 2011

I made the following micro-benchmarking script. However rbx (rubinius 2.0.0dev (1.8.7 fa5346d yyyy-mm-dd JI) [i686-pc-linux-gnu]) seems to stop dead-lock. Is it intentional?

require 'benchmark'
require 'thread'

tmax = 100
max = 10_000_000 / tmax

Benchmark.bm{|x|
  x.report("T/Sh"){
    a = []
    m = Mutex.new
    (1..tmax).map{
      Thread.new{
        max.times{
          m.synchronize{
            a.push true
            a.pop
          }
        }
      }
    }.each{|t|
      t.join
    }
  }
  x.report("T/Sh2"){
    a = []
    (1..tmax).map{
      Thread.new{
        max.times{
          a.push true
          a.pop
        }
      }
    }.each{|t|
      t.join
    }
  }  if RUBY_ENGINE != 'rbx'
  x.report("T/Ex"){
    (1..tmax).map{
      Thread.new{
        a = []
        max.times{
          a.push true
          a.pop
        }
      }
    }.each{|t|
      t.join
    }
  }
  x.report("Single"){
    a = []
    (tmax * max).times{
      a.push true
      a.pop
    }
  }
}
Contributor

headius commented Jun 8, 2011

When I run it on JRuby I get concurrency errors. The pushing and popping on the array are not threadsafe operations for the first two benchmarks.

If I modify those arrays to extend JRuby::Synchronized (which make all methods synchronize against the object) it runs to completion:

(JRuby master on Java 7 with invokedynamic)
user system total real
T/Sh 12.072000 0.000000 12.072000 ( 12.072000)
T/Sh2 2.667000 0.000000 2.667000 ( 2.667000)
T/Ex 0.680000 0.000000 0.680000 ( 0.680000)
Single 1.035000 0.000000 1.035000 ( 1.035000)

Ruby 1.9.2:
T/Sh 36.740000 50.590000 87.330000 ( 69.283367)
T/Sh2 2.280000 0.010000 2.290000 ( 2.301094)
T/Ex 2.210000 0.020000 2.230000 ( 2.224261)
Single 2.180000 0.000000 2.180000 ( 2.189540)

I don't know what's simplest to make a given object thread-safe in Rubinius, so I didn't try to rig up some mutexing to make it safe.

Perhaps there's a simply a bug/deadlock in Rubinius when concurrency errors are triggered by thread-unsafe code?

ko1 commented Jun 9, 2011

Thanks, headius. T/Sh2 is a benchmark for MRI, assuming thread safe Array#push, pop. This is why skipping with this condition "if RUBY_ENGINE != 'rbx'". I need to change this condition to "if RUBY_ENGINE == 'ruby'". Maybe your change is for T/Sh2, isn't it?

Please teach me one point. You mention "the first two benchmarks". The first benchmark is guarded by a Mutex. So you don't need to add any other synchronization, do you?

Contributor

headius commented Jun 9, 2011

Re: not running on Rubinius... Oh, I understand that guard now. You wanted to prevent the unsafe bench from running. Yes, == 'ruby' is a better guard.

Re: "first two benchmarks"... Yes, my mistake. Only T/Sh2 is thread-unsafe.

Since I posted before...updated run on JRuby with those fixes in place and no use of JRuby::Synchronized

                 user     system      total        real
    T/Sh    10.822000   0.000000  10.822000 ( 10.822000)
    T/Ex     0.735000   0.000000   0.735000 (  0.735000)
    Single   0.910000   0.000000   0.910000 (  0.909000)

You can see the perf impact that synchronization has on the first benchmark; I still believe it's unacceptable to penalize all code like this, and so therefore I support Rubinius not making these core classes thread-safe.

As far as Rubinius goes...I guess I was wrong about it being a bad benchmark, so there may indeed be some deadlocking case somewhere.

Owner

evanphx commented Jun 23, 2011

I'm unable to get a deadlock with this code. I have fixed a few deadlocks since this was entered. @ko1 could you please rerun and see if you still issues? Thanks!

@ghost ghost assigned evanphx Jun 23, 2011

ko1 commented Jun 23, 2011

I still got a deadlock on the T/Sh. There are no response, but CPU usage is 0% with this process. 2 core, VirtualBox guest Linux (debian/squeeze).

$ ~/tmp/rbx-2.0.0pre/bin/rbx -v
rubinius 2.0.0dev (1.8.7 45e0601 yyyy-mm-dd JI) [i686-pc-linux-gnu]

Owner

dbussink commented Oct 6, 2011

There have been a bunch of fixes which also include some potential deadlocks. I can't reproduce this anymore on OS X Lion and also not on Debian Squeeze with the latest master. Therefore I'm closing this one, if it's still happening please reopen this issue.

@dbussink dbussink closed this Oct 6, 2011

brixen added a commit that referenced this issue Oct 21, 2014

Updated RubyGems to 2.4.2.
Installing RubyGems 2.4.2
RubyGems 2.4.2 installed

=== 2.4.2 / 2014-10-01

This release was sponsored by Ruby Central.

Bug fixes:

* RubyGems now correctly matches wildcard no_proxy hosts.  Issue #997 by
  voelzemo.
* Added support for missing git_source method in the gem dependencies API.
* Fixed handling of git gems with an alternate install directory.
* Lockfiles will no longer be truncated upon resolution errors.
* Fixed messaging for `gem owner -a`.  Issue #1004 by Aaron Patterson, Ryan
  Davis.
* Removed meaningless ensure.  Pull request #1003 by gogotanaka.
* Improved wording of --source option help.  Pull request #989 by Jason Clark.
* Empty build_info files are now ignored.  Issue #903 by Adan Alvarado.
* Gem::Installer ignores dependency checks when installing development
  dependencies.  Issue #994 by Jens Willie.
* `gem update` now continues after dependency errors.  Issue #993 by aaronchi.
* RubyGems no longer warns about semantic version dependencies for the 0.x
  range.  Issue #987 by Jeff Felchner, pull request #1006 by Hsing-Hui Hsu.
* Added minimal lock to allow multithread installation of gems.  Issue #982
  and pull request #1005 by Yorick Peterse
* RubyGems now considers prerelease dependencies as it did in earlier versions
  when --prerelease is given.  Issue #990 by Jeremy Tryba.
* Updated capitalization in README.  Issue #1010 by Ben Bodenmiller.
* Fixed activating gems from a Gemfile for default gems.  Issue #991 by khoan.
* Fixed windows stub script generation for Cygwin.  Issue #1000 by Brett
  DiFrischia.
* Allow gem bindir and ruby.exe to live in separate diretories.  Pull request
  #942 by Ian Flynn.
* Fixed handling of gemspec in gem dependencies files to match Bundler
  behavior.  Issue #1020 by Michal Papis.
* Fixed `gem update` when updating to prereleases.  Issue #1028 by Santiago
  Pastorino.
* RubyGems now fails immediately when a git reference cannot be found instead
  of spewing git errors.  Issue #1031 by Michal Papis

=== 2.4.1 / 2014-07-17

Bug fixes:

* RubyGems can now be updated on Ruby implementations that do not support
  vendordir in RbConfig::CONFIG.  Issue #974 by net1957.

=== 2.4.0 / 2014-07-16

Minor enhancements:

* The contents command now supports a --show-install-dir option that shows
  only the directory the gem is installed in.  Feature request #966 by Akinori
  MUSHA.
* Added a --build-root option to the install command for packagers.  Pull
  request #965 by Marcus Rückert.
* Added vendor gem support to RubyGems.  Package managers may now install gems
  in Gem.vendor_dir with the --vendor option to gem install.  Issue #943 by
  Marcus Rückert.

Bug fixes:

* Kernel#gem now respects the prerelease flag when activating gems.
  Previously this behavior was undefined which could lead to bugs when a
  prerelease version was unintentionally activated.  Bug #938 by Joe Ferris.
* RubyGems now prefers gems from git over installed gems.  This allows gems
  from git to override an installed gem with the same name and version.  Bug
  #944 by Thomas Kriechbaumer.
* Fixed handling of git gems in a lockfile with unversioned dependencies.  Bug
  #940 by Michael Kaiser-Nyman.
* The ruby directive in a gem dependencies file is ignored when installing.
  Bug #941 by Michael Kaiser-Nyman.
* Added open to list of builtin commands (`gem open` now works).  Reported by
  Espen Antonsen.
* `gem open` now works with command-line editors.  Pull request #962 by Tim
  Pope.
* `gem install -g` now respects `--conservative`.  Pull request #950 by Jeremy
  Evans.
* RubyGems releases announcements now now include checksums.  Bug #939 by
  Alexander E. Fischer.
* RubyGems now expands ~ in $PATH when checking if installed executables will
  be runnable.  Pull request #945 by Alex Talker.
* Fixed `gem install -g --explain`.  Issue #947 by Luis Lavena.  Patch by
  Hsing-Hui Hsu.
* RubyGems locks less during gem activation.  Pull request #951 by Aaron
  Patterson and Justin Searls, #969 by Jeremy Tryba.
* Kernel#gem is now thread-safe.  Pull request #967 by Aaron Patterson.
* RubyGems now handles spaces in directory names for some parts of extension
  building.  Pull request #949 by Tristan Hill.
* RubyGems no longer defines an empty Date class.  Pull Request #948 by Benoit
  Daloze.
* RubyGems respects --document options for `gem update` again.  Bug 946 by
  jonforums.  Patch by Hsing-Hui Hsu.
* RubyGems generates documentation again with --ignore-dependencies.  Bug #961
  by Pulfer.
* RubyGems can install extensions across partitions now.  Pull request #970 by
  Michael Scherer.
* `-s` is now short for `--source` which resolves an ambiguity with
  --no-suggestions.  Pull request #955 by Alexander Kahn.
* Added extra test for ~> for 0.0.X versions.  Pull request #958 by Mark
  Lorenz.
* Fixed typo in gem updated help.  Pull request #952 by Per Modin.
* Clarified that the gem description should not be excessively long.  Part of
  bug #956 by Renier Morales.
* Hid documentation of outdated test_files related methods in Specification.
  Guides issue #90 by Emil Soman.
* RubyGems now falls back to the old index if the rubygems.org API fails
  during gem resolution.

=== 2.3.0 / 2014-06-10

Minor enhancements:

* Added the `open` command which allows you to inspect the source of a gem
  using your editor.
  Issue #789 by Mike Perham. Pull request #804 by Vitali F.
* The `update` command shows a summary of which gems were and were not
  updated.  Issue #544 by Mark D. Blackwell.
  Pull request #777 by Tejas Bubane.
* Improved "could not find 'gem'" error reporting.  Pull request #913 by
  Richard Schneeman.
* Gem.use_gemdeps now accepts an argument specifying the path of the gem
  dependencies file.  When the file is not found an ArgumentError is raised.
* Writing a .lock file for a gem dependencies file is now controlled by the
  --[no-]lock option.  Pull reuqest #774 by Jeremy Evans.
* Suggestion of alternate names and spelling corrections during install can be
  suppressed with the --no-suggestions option.  Issue #867 by Jimmy Cuadra.
* Added mswin64 support.  Pull request #881 by U. Nakamura.
* A gem is installable from an IO again (as in RubyGems 1.8.x and older).
  Pull request #716 by Xavier Shay.
* RubyGems no longer attempts to build extensions during activation.  Instead
  a warning is issued instructing you to run `gem pristine` which will build
  the extensions for the current platform.  Issue #796 by dunric.
* Added Gem::UserInteraction#verbose which prints when the --verbose option is
  given.  Pull request #811 by Aaron Patterson.
* RubyGems can now fetch gems from private repositories using S3.  Pull
  request #856 by Brian Palmer.
* Added Gem::ConflictError subclass of Gem::LoadError so you can distinguish
  conflicts from other problems.  Pull request #841 by Aaron Patterson.
* Cleaned up unneeded load_yaml bootstrapping in Rakefile.  Pull request #815
  by Zachary Scott.
* Improved performance of conflict resolution.  Pull request #842 by Aaron
  Patterson.
* Add documentation of "~> 0" to Gem::Version.  Issue #896 by Aaron Suggs.
* Added CONTRIBUTING file.  Pull request #849 by Mark Turner.
* Allow use of bindir in windows_stub_script in .bat
  Pull request #818 by @unak and @nobu
* Use native File::PATH_SEPARATOR and remove $ before gem env on
  Gem::Dependency#to_specs. Pull request #915 by @parkr
* RubyGems recommends SPDX IDs for licenses now.  Pull request #917 by
  Benjamin Fleischer.

Bug fixes:

* RubyGems now only fetches the latest specs to find misspellings which speeds
  up gem suggestions.  Pull request #808 by Aaron Patterson.
* The given .gem is installed again when multiple versions of the same gem
  exist in the current directory.  Bug #875 by Prem Sichanugrist.
* Local gems are preferred by name over remote gems again.  Bug #834 by
  jonforums.
* RubyGems can install local prerelease gems again.  Pull request #866 by
  Aaron Patterson.  Issue #813 by André Arko.
* RubyGems installs development dependencies correctly again.  Issue #893 by
  Jens Wille.
* RubyGems only installs prerelease versions when they are requested again.
  Issue #853 by Seth Vargo, special thanks to Zachary Scott and Ben Moss.
  Issue #884 by Nathaniel Bibler.
* Fixed RubyGems list and search command help.  Pull request #905 and #928 by
  Gabriel Gilder.
* The list of gems to uninstall is always sorted now.  Bug #918 by postmodern.
* The update command only updates exactly matching gem names now.  Bug #919 by
  postmodern.
* Gem::Server now supports prerelease versions.  Bug #857 by Marcelo Alvim.
* RubyGems no longer raises an exception immediately when gems are missing
  with RUBYGEMS_GEMDEPS.  A warning is printed instead.  Issue #886 by Michael
  Kaiser-Nyman.
* Commands using the rubygems.org API no longer try to sign-in when a
  non-rubygems API key has been chosen.  Bug #826 by Ben Sedat.
* Updated documentation of Gem::Specification#executables to indicate that
  only ruby scripts are allowed.  Bug #830 by Geoff Nixon.
* Gem dependency API supports multiple platforms for #platform and #platforms
  now.  Bug #821 by johnny5-.
* Gem dependency API supports lockfiles without explicit sources.  Bug #820 by
  johnny5-.
* Gem dependency API supports lockfiles with multiple sources.  Bug #822 by
  johnny5-, bug #851 by sumit shah.
* Gem dependency API supports lockfiles with git sources using branch, tag and
  ref.  Bug #822 by johnny5-, #931 by Christoph Blank.
* Gem dependency API no longer raises an exception when a gem does not exist
  in one of the configured sources.  Bug #897 by Michael Kaiser-Nyman.
* Gem dependency API no longer lists development dependencies in the lockfile.
  Bug #768 by Diego Viola, #916 by Santiago Pastorino.
* SSL configuration entries in ~/.gemrc are properly round-tripped.  Bug #837
  by Noah Luck Easterly.
* The environment command now shows the system configuration directory where
  the all-users gemrc lives.  Bug #827 by Ben Langfeld.
* Improved speed of conflict checking when activating gems.  Pull request #843
  by Aaron Patterson.
* Improved speed of levenshtein distance for gem suggestion misspellings.
  Pull requests #809, #812 by Aaron Patterson.
* Restored persistent connections.  Pull request #869 by Aaron Patterson.
* Reduced requests when fetching gems with the bundler API.  Pull request #773
  by Charlie Somerville.
* Reduced dependency prefetching to improve install speed.  Pull requests
  #871, #872 by Matthew Draper.
* RubyGems now avoids net/http auto-proxy detection.  Issue #824 by HINOHARA
  Hiroshi.
* Removed conversion of Gem::List (used for debugging installs) to unless
  necessary.  Pull request #870 by Aaron Patterson.
* RubyGems now prints release notes from the current release.  Bug #814 by
  André Arko.
* RubyGems allows installation of unsigned gems again with -P MediumSecurity
  and lower.  Bug #859 by Justin S. Collins.
* Fixed typo in Jim Weirich's name.  Ruby pull request #577 by Mo Khan.
* Fixed typo in Gem.datadir documentation.  Pull request #868 by Patrick
  Jones.
* Fixed File.exists? warnings.  Pull request #829 by SHIBATA Hiroshi.
* Fixed show_release_notes test for LANG=C.  Issue #862 by Luis Lavena.
* Fixed Gem::Package from IO tests on windows.  Patch from issue #861 by Luis
  Lavena.
* Check for nil extensions as BasicSpecification does not initialize them.
  Pull request #882 by André Arko.
* Fixed Gem::BasicSpecification#require_paths receives a String for
  @require_paths. Pull requrest #904 by @danielpclark
* Fixed circular require warnings.  Bug #908 by Zachary Scott.
* Gem::Specification#require_paths can no longer accidentally be an Array.
  Pull requests #904, #909 by Daniel P. Clark.
* Don't build extensions if `build_dir/extensions` isn't writable.
  Pull request #912 by @dunric
* Gem::BasicSpecification#require_paths respects default_ext_dir_for now.  Bug
  #852 by Vít Ondruch.

------------------------------------------------------------------------------

brixen added a commit that referenced this issue Nov 28, 2014

Updated rubygems to 2.4.4.
RubyGems 2.4.4 installed

=== 2.4.4 / 2014-11-12

Bug Fixes:

* Add alternate Root CA for upcoming certificate change. Fixes #1050 by
  Protosac

=== 2.4.3 / 2014-11-10

Bug fixes:

* Fix redefine MirrorCommand issue. Pull request #1044 by @akr.
* Fix typo in platform= docs.  Pull request #1048 by @jasonrclark
* Add root SSL certificates for upcoming certificate change.  Fixes #1050 by
  Protosac

=== 2.4.2 / 2014-10-01

This release was sponsored by Ruby Central.

Bug fixes:

* RubyGems now correctly matches wildcard no_proxy hosts.  Issue #997 by
  voelzemo.
* Added support for missing git_source method in the gem dependencies API.
* Fixed handling of git gems with an alternate install directory.
* Lockfiles will no longer be truncated upon resolution errors.
* Fixed messaging for `gem owner -a`.  Issue #1004 by Aaron Patterson, Ryan
  Davis.
* Removed meaningless ensure.  Pull request #1003 by gogotanaka.
* Improved wording of --source option help.  Pull request #989 by Jason Clark.
* Empty build_info files are now ignored.  Issue #903 by Adan Alvarado.
* Gem::Installer ignores dependency checks when installing development
  dependencies.  Issue #994 by Jens Willie.
* `gem update` now continues after dependency errors.  Issue #993 by aaronchi.
* RubyGems no longer warns about semantic version dependencies for the 0.x
  range.  Issue #987 by Jeff Felchner, pull request #1006 by Hsing-Hui Hsu.
* Added minimal lock to allow multithread installation of gems.  Issue #982
  and pull request #1005 by Yorick Peterse
* RubyGems now considers prerelease dependencies as it did in earlier versions
  when --prerelease is given.  Issue #990 by Jeremy Tryba.
* Updated capitalization in README.  Issue #1010 by Ben Bodenmiller.
* Fixed activating gems from a Gemfile for default gems.  Issue #991 by khoan.
* Fixed windows stub script generation for Cygwin.  Issue #1000 by Brett
  DiFrischia.
* Allow gem bindir and ruby.exe to live in separate diretories.  Pull request
  #942 by Ian Flynn.
* Fixed handling of gemspec in gem dependencies files to match Bundler
  behavior.  Issue #1020 by Michal Papis.
* Fixed `gem update` when updating to prereleases.  Issue #1028 by Santiago
  Pastorino.
* RubyGems now fails immediately when a git reference cannot be found instead
  of spewing git errors.  Issue #1031 by Michal Papis

------------------------------------------------------------------------------
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment