Bundler 2.4 loops forever under JRuby

[andsel]

Describe the problem as clearly as you can

Executing Bundler (2.4.11) to install gems present in a Gemfile under JRuby ends that the resolution never stop and create a memory leak.

Did you try upgrading rubygems & bundler?

Bundler is already updated to the latest.

Post steps to reproduce the problem

• Ruby: JRuby 9.4.2.0 installed with rvm
• Bundler was installed explicitly with gem:

gem install bundler -v 2.4.11

Which command did you run?

bundle _2.4.11_ install

Gemfile used

source "https://rubygems.org"
gem "logstash-integration-aws"
gem "logstash-input-cloudwatch"

bundle _2.4.11_ install

What were you expecting to happen?

I would expected that the process terminates with resolution success or failure.

What actually happened?

Loop forever until an OutOfMemory error happens.

If not included with the output of your command, run bundle env and paste the output below

Environment

Bundler             2.4.11
  Platforms         ruby, universal-java-11
Ruby                3.1.0p0 (2023-03-08 revision 90d2913fdae43d5ea049f2a99b5ccffb70bf051e) [universal-java-11]
  Full Path         /usr/share/rvm/rubies/jruby-9.4.2.0/bin/jruby
  Config Dir        /usr/share/rvm/rubies/jruby-9.4.2.0/etc
RubyGems            3.3.25
  Gem Home          /usr/share/rvm/gems/jruby-9.4.2.0
  Gem Path          /usr/share/rvm/gems/jruby-9.4.2.0:/usr/share/rvm/rubies/jruby-9.4.2.0/lib/ruby/gems/shared
  User Home         /home/andrea
  User Path         /home/andrea/.gem/jruby/3.1.0
  Bin Dir           /usr/share/rvm/gems/jruby-9.4.2.0/bin
Tools               
  Git               2.25.1
  RVM               1.29.12 (manual)
  rbenv             not installed
  chruby            not installed
  rubygems-bundler  (1.4.5)

Bundler Build Metadata

Built At          2023-04-10
Git SHA           be1d1b4623
Released Version  true

Bundler settings

gem.test
  Set for the current user (/home/andrea/.bundle/config): "rspec"

Gemfile

Gemfile

source "https://rubygems.org"
gem "logstash-integration-aws"
gem "logstash-input-cloudwatch"

Gemfile.lock

<No /home/andrea/workspace/bundler_oom_reproducer/tmp/Gemfile.lock found>

Further investigations

Running Bundler with debug flag:

DEBUG_RESOLVER=1 bundle _2.4.11_ install --verbose

it turn out that after backtracking a lot, for each version of aws-sdk-resources gems from 2.11.263 down to 2.0.23

conflict: aws-sdk-resources >= 2.11.632, < 3.0.0 depends on aws-sdk-core = 2.11.632
backtracking to 5
derived: not aws-sdk-resources >= 2.11.632, < 3.0.0
.
.
.
conflict: aws-sdk-resources < 2.0.23 depends on aws-sdk-core = 2.0.22
backtracking to 5
derived: not aws-sdk-resources < 2.0.23

resolving into a resolution conflict, the log remains in infinite loop, printing

! not logstash-mixin-aws >= 0.1.7 is partially satisfied by not logstash-mixin-aws < 0.1.7
! which is caused by logstash-mixin-aws < 0.1.7 depends on logstash >= 1.4.0, < 2.0.0
! thus logstash-input-cloudwatch < 2.1.0 requires logstash >= 1.4.0, < 2.0.0 or logstash-mixin-aws >= 0.1.7

These logs comes from

rubygems/bundler/lib/bundler/vendor/pub_grub/lib/pub_grub/version_solver.rb

Lines 227 to 229 in b2fe65b

	logger.info { "! #{most_recent_term} is#{partially} satisfied by #{most_recent_satisfier.term}" }
	logger.info { "! which is caused by #{most_recent_satisfier.cause}" }
	logger.info { "! thus #{incompatibility}" }

and the motivation while it keeps locked into the loop and memory continues increasing, is due to

while !incompatibility.failure?
  ...
  incompatibility = Incompatibility.new(new_terms, cause: Incompatibility::ConflictCause.new(incompatibility, most_recent_satisfier.cause))
  ...
  logger.info { "! #{most_recent_term} is#{partially} satisfied by #{most_recent_satisfier.term}" }
  logger.info { "! which is caused by #{most_recent_satisfier.cause}" }
  logger.info { "! thus #{incompatibility}" }
end  

The incompatibility instance created has failed? that evaluates always to true while the Incompatibility instances are kept alive by new_terms variable, and this made the OutOfMemory error.

Looking into the definition of failed? at

rubygems/bundler/lib/bundler/vendor/pub_grub/lib/pub_grub/incompatibility.rb

Line 36 in b2fe65b

terms.empty? || (terms.length == 1 && Package.root?(terms[0].package) && terms[0].positive?)

def failure?
  terms.empty? || (terms.length == 1 && Package.root?(terms[0].package) && terms[0].positive?)
end

it turns out that:

• terms is not empty and contains 3 Terms
• none of them satisfies Package.root?
• the three terms are:
  • logstash-input-cloudwatch < 2.1.0
  • not logstash >= 1.4.0, < 2.0.0
  • not logstash-mixin-aws >= 0.1.7
    this made the failure? method to return continuously false and so the infinite loop.

[deivid-rodriguez]

Hi, thanks for the report and investigation.

I think this may be the same root cause at #6686. No resolution yet, but jhawthorn/pub_grub#29 should alleviate the issue a bit.

[deivid-rodriguez]

The infinite loop has been fixed, but there's still underlying resolution bug which we're tracking at jhawthorn/pub_grub#30.

Let me close this issue in favor of the upstream one.

[andsel]

Oks great to know, and thanks a lot @deivid-rodriguez for looking into it, PubGrub and in general constraint resolution is not an easy thing.