Fix dependency source bug in bundler

[eileencodes]

I stumbled across a bundler bug that had me scratching my head for awhile, because I hadn't experienced it before.

In some cases when changing the source in a gemfile from a Source::Gemspec to either a Source::Path or Source::Git only the parent gem will have it's gem replaced and updated and the child components will retain the original version. This only happens if the gem version of the Source::Gemspec and Source::Git are the same. It also requires another gem to share a dependency with the one being updated.

For example if I have the following gemfile:

gem "rails", "~> 8.1.1"
gem "propshaft"

Rails has a component called actionpack which propshaft depends on.

If I change rails to point at a git source (or path source), only the path for rails gets updated:

gem "rails", github: "rails/rails", branch: "8-1-stable"
gem "propshaft"

Because actionpack is a dependency of propshaft, it will remain in the rubygems source in the lock file WHILE the other gems are correctly pointing to the git source.

Gemfile.lock:

GIT
  remote: https://github.com/rails/rails.git
  revision: 9439f463e0ef974c447edcc1b17882f072e99d22
  branch: 8-1-stable
  specs:
    actioncable (8.1.1)
      ...
    actionmailbox (8.1.1)
      ...
    actionmailer (8.1.1)
      ...
    actiontext (8.1.1)
      ...
    activejob (8.1.1)
      ...
    activemodel (8.1.1)
      ...
    activerecord (8.1.1)
      ...
    activestorage (8.1.1)
      ...
    rails (8.1.1)
      ...
    railties (8.1.1)
      ...

GEM
  remote: https://rubygems.org/
  specs:
    action_text-trix (2.1.15)
      railties
    actionpack (8.1.1) <===== incorrectly left in Rubygems source
      ...

The gemfile will contain actionpack in the rubygems source, but will be missing in the git source so the path will be incorrect. A bundle show on Rails will point to the correct place:

$ bundle show rails
/Users/eileencodes/.gem/ruby/3.4.4/bundler/gems/rails-9439f463e0ef

but a bundle show on actionpack will be incorrect:

$ bundle show actionpack
/Users/eileencodes/.gem/ruby/3.4.4/gems/actionpack-8.1.1

This bug requires the following to reproduce:

1. A gem like Rails that contains components that are released as their own standalone gem is added to the gemfile pointing to rubygems
2. A second gem is added that depends on one of the gems in the first gem (like propshaft does on actionpack)
3. The Rails gem is updated to use a git source, pointing to the same version that is being used by rubygems (ie 8.1.1)
4. bundle will only update the path for Rails component gems if no other gem depends on it.

This incorrectly leaves Rails (or any gem like it) using two different codepaths / gem source code.

What was the end-user or developer problem that led to this PR?

What is your fix for the problem, implemented in this PR?

Make sure the following tasks are checked

• Describe the problem / feature
• Write tests for features and bug fixes
• Write code to solve the problem
• Make sure you follow the current code style and write meaningful commit messages without tags

[hsbt]

/cc @zzak This PR may resolve your issue too.

[hsbt]

I will look this after releasing Ruby 4.0. Sorry to not merge this for Bundler 4.0.3.

[eileencodes]

Sorry to not merge this for Bundler 4.0.3.

No worries at all! I was out for two weeks anyway so I wouldn't have been around to make changes.

[ngan]

I think this change introduced a bug. Here's a copy-paste of what I'm seeing in terminal with some gem names and URLs redacted:

$ bundle update kt-paperclip

Fetching gem metadata from https://gemstash.my-private-gemstash.com/private/.....
Fetching gem metadata from https://gems.karafka.io/.
Fetching gem metadata from https://enterprise.contribsys.com/..
Fetching gem metadata from https://gems.graphql.pro/..
Fetching gem metadata from https://rubygems.org/........
Resolving dependencies...
Could not find compatible versions

Because my_custom_gem >= 2.3.0 depends on aws-sdk-elasticache ~> 1
  and aws-sdk-elasticache ~> 1 could not be found in rubygems repository https://gemstash.my-private-gemstash.com/private/,
  gusto-my_custom_gem >= 2.3.0 cannot be used.
So, because Gemfile depends on my_custom_gem = 2.4.7,
  version solving has failed.

I'm trying to bump the kt-paperclip gem and it's saying the aws-sdk-elasticache gem couldn't be found in my private gemstash--which makes sense. But, it shouldn't be looking for aws-sdk-elasticache there in the first place. My my_custom_gem is in my private gemstash. My guess is that the source for my gem is bleeding into aws-sdk-elasticache somehow.

I removed my_custom_gem from my gemfile and another, similar, problem surfaced where it was looking for a public gem inside a pathed source.

One piece of information that might be relevant is that my Gemfile does not declare a direct dependency on aws-sdk-elasticache. It is a dependency of my custom gem. I think the bundler rule is look in the same source as my_custom_gem and if it's not there, fallback to the global source: https://bundler.io/v4.0/man/gemfile.5.html#SOURCE-PRIORITY

Let me know if y'all need more information for reducibility, happy to provide.

Downgrading to bundler 4.0.3 fixes the issue.

[eileencodes]

I don't think this was included in 4.0.3 though because it was merged after the release, as noted by @hsbt #9213 (comment)

[eileencodes]

Oh I misread which version you were on and downgraded to. 🤦🏼‍♀️

[ngan]

I've created an issue: #9258

@eileencodes this change went out with Bundler 4.0.4, which is the version of bundler we upgraded to and started seeing the problem. Downgrading to 4.0.3 fixes the issue.