Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

No live threads left. Deadlock? with specific Gemfile using 1.5.1 #2813

Closed
schneems opened this Issue · 12 comments

3 participants

@schneems

Gemfile:
https://gist.github.com/schneems/8342838/raw/a8be3926cd9f8e311dfd92af92bd29acdf37bc2a/gistfile1.txt

Error:

Unfortunately, a fatal error has occurred. Please see the Bundler troubleshooting documentation at http://bit.ly/bundler-issues. Thanks!
/Users/schneems/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.5.1/lib/bundler/parallel_workers/worker.rb:33:in `pop': No live threads left. Deadlock? (fatal)
    from /Users/schneems/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.5.1/lib/bundler/parallel_workers/worker.rb:33:in `deq'
    from /Users/schneems/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.5.1/lib/bundler/installer.rb:325:in `install_in_parallel'
    from /Users/schneems/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.5.1/lib/bundler/installer.rb:95:in `run'
    from /Users/schneems/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.5.1/lib/bundler/installer.rb:15:in `install'
    from /Users/schneems/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.5.1/lib/bundler/cli.rb:257:in `install'
    from /Users/schneems/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.5.1/lib/bundler/vendor/thor/command.rb:27:in `run'
    from /Users/schneems/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.5.1/lib/bundler/vendor/thor/invocation.rb:120:in `invoke_command'
    from /Users/schneems/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.5.1/lib/bundler/vendor/thor.rb:363:in `dispatch'
    from /Users/schneems/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.5.1/lib/bundler/vendor/thor/base.rb:438:in `start'
    from /Users/schneems/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.5.1/lib/bundler/cli.rb:10:in `start'
    from /Users/schneems/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.5.1/bin/bundle:22:in `block in <top (required)>'
    from /Users/schneems/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.5.1/lib/bundler/friendly_errors.rb:5:in `with_friendly_errors'
    from /Users/schneems/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.5.1/bin/bundle:22:in `<top (required)>'
    from /Users/schneems/.rbenv/versions/2.1.0/bin/bundle:23:in `load'
    from /Users/schneems/.rbenv/versions/2.1.0/bin/bundle:23:in `<main>'

Original Report: heroku/heroku-buildpack-ruby#197 (comment)

It works if you remove mongoid_auto_increment or increment it's version number.

@indirect
Owner

Creepy. Somehow one gem is killing the threads?

@indirect
Owner

@gnufied, any theories about what could be causing this?

@schneems

Heres the gemspec diff http://www.diffchecker.com/di61b3ov maybe there's an issue with 0.1.1 referencing itself? mongoid_auto_increment

@indirect
Owner
@schneems

If that was the case wouldn't it fail if we installed without parallel?

@schneems

Another Gemfile with a problem gem: https://gist.github.com/schneems/8344363

Not sure if they're related, here's the original SO post: http://stackoverflow.com/questions/21032735/bundle-without-j4-on-heroku

Problem gem was: gem 'formtastic-bootstrap', " ~> 2.0.0"

@schneems

Apparently formtastic-bootstrap has the same circular dependency: http://rubygems.org/gems/formtastic-bootstrap/versions/2.0.0

@gnufied
Collaborator

At least the problem is definitely caused by cyclic dependecy, but TSort algorithm does not raise error on self cycles because technically they are resolvable. For example:

class Hash
  include TSort
  alias tsort_each_node each_key
  def tsort_each_child(node, &block)
    fetch(node).each(&block)
  end
end

{ 1 => [2, 3], 2 => [3], 3 => [], 4 => [], 5 => [5] }.tsort

Will not raise error.

As for why it causes problems during parallel installation is - that is because we are using hash keys as a way of keeping track of installed stuff. But when we hit a self dependency using gem name as key screws up the algorithm. I will try and come up with a fix.

@schneems schneems referenced this issue from a commit in schneems/bundler
@schneems schneems [close #2813] Fix circular parallel gem resolution
Right now if you install via parallel a gem that has a circular dependency it will know that it needs to be installed based on the `remains` hash, but does not actually enqueue the circular dependency to be installed. This commit fixes that by checking for duplicate name as well as whether or not dependencies have been previously enqueued.
aa7a2b0
@schneems

@gnufied i've got a proposed fix in #2814 it solves all the test cases I have for parallel resolution, though I'm new to this code, perhaps there are improvements to this solution.

@schneems schneems referenced this issue from a commit in schneems/bundler
@schneems schneems [close #2813] Fix circular parallel gem resolution
Right now if you install via parallel a gem that has a circular dependency it will know that it needs to be installed based on the `remains` hash, but does not actually enqueue the circular dependency to be installed. This commit fixes that by checking for duplicate name as well as whether or not dependencies have been previously enqueued.
49933a7
@eagletmt eagletmt referenced this issue from a commit in eagletmt/bundler
@eagletmt eagletmt Fix #2813: Ignore self dependency 1c5c7e6
@eagletmt eagletmt referenced this issue from a commit in eagletmt/bundler
@eagletmt eagletmt Fix #2813: Ignore self dependency 338357d
@indirect indirect closed this in b7c4534
@indirect indirect referenced this issue from a commit
@eagletmt eagletmt Fix #2813: Ignore self dependency
Conflicts:
	spec/realworld/parallel_install_spec.rb
749e062
@schneems
@indirect
Owner

lol I already pushed 1.5.2 :)

@schneems

That was fast, i'm going to try to push 1.5.2 out to production on Heroku this monday. Had to roll back 1.5.1 due to this bug.

Thanks :heart:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.