Prepare RubyGems 4.0.9 and Bundler 4.0.9 by hsbt · Pull Request #9426 · ruby/rubygems

hsbt · 2026-03-25T00:35:31Z

Split the download and install process of a gem #9381
Fix gem which command test isolation #9222
Introduce a priority queue #9389
fix: include owner role in gem owner #9403
Retry git fetch without --depth for dumb HTTP transport #9405
Add exponential backoff to bundler retries #9163
fix: Ensure trailing slash is added to source URIs added via gem sources #9055
Prevent tests from modifying user's global git config #9397
Refactor Bundler tests to invoke RubyGems API directly #9195
Normalize the number of workers when performing parallel operations #9400
Check the git version only once per bundle install #9406
[DOC] Fix link #9409

Split the download and install process of a gem (cherry picked from commit bcc4469)

Fix gem which command test isolation (cherry picked from commit 8b405fa)

Introduce a priority queue (cherry picked from commit 115228e)

fix: include owner role in `gem owner` (cherry picked from commit 0bcfe4b)

Retry git fetch without --depth for dumb HTTP transport (cherry picked from commit cb6fbe3)

…koff Add exponential backoff to bundler retries (cherry picked from commit fd31044)

fix: Ensure trailing slash is added to source URIs added via gem sources (cherry picked from commit 19debfb)

Prevent tests from modifying user's global git config (cherry picked from commit 1be513c)

Refactor Bundler tests to invoke RubyGems API directly (cherry picked from commit 9d755be)

Normalize the number of workers when performing parallel operations (cherry picked from commit be5febe)

Check the git version only **once** per `bundle install` (cherry picked from commit e3685c9)

[DOC] Fix link (cherry picked from commit 0e9dd7b)

Copilot

Pull request overview

Prepares the RubyGems and Bundler 4.0.9 release by bumping versions/lockfiles and folding in a set of RubyGems CLI and Bundler installer/runtime changes (plus accompanying tests and documentation updates).

Changes:

Bump RubyGems/Bundler versions to 4.0.9 and update related lockfiles and changelogs.
Update RubyGems commands (sources, owner) behavior and tests (trailing-slash normalization; include owner roles).
Update Bundler internals (split download vs install, priority queue support, retry backoff, git version memoization) and adjust/spec-add tests and helpers.

Reviewed changes

Copilot reviewed 40 out of 49 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
tool/bundler/vendor_gems.rb.lock	Update `BUNDLED WITH` to 4.0.9.
tool/bundler/test_gems.rb.lock	Update `BUNDLED WITH` to 4.0.9.
tool/bundler/standard_gems.rb.lock	Update `BUNDLED WITH` to 4.0.9.
tool/bundler/rubocop_gems.rb.lock	Update `BUNDLED WITH` to 4.0.9.
tool/bundler/release_gems.rb.lock	Update `BUNDLED WITH` to 4.0.9.
tool/bundler/lint_gems.rb.lock	Update `BUNDLED WITH` to 4.0.9.
tool/bundler/dev_gems.rb.lock	Update `BUNDLED WITH` to 4.0.9.
test/rubygems/test_gem_commands_which_command.rb	Remove stale “fails in isolation” TODO after test isolation fixes.
test/rubygems/test_gem_commands_sources_command.rb	Add coverage for trailing-slash normalization + adjust existing expectations.
test/rubygems/test_gem_commands_owner_command.rb	Update fixtures/assertions to include owner roles in output.
lib/rubygems/commands/sources_command.rb	Normalize source URIs (trailing slash) and centralize source construction/validation.
lib/rubygems/commands/owner_command.rb	Print owner role alongside identifier in `gem owner` output.
lib/rubygems.rb	Bump RubyGems version and update RubyGems API documentation link.
bundler/spec/support/windows_tag_group.rb	Assign new/updated installer spec to a Windows runner group.
bundler/spec/support/helpers.rb	Refactor gem install/uninstall/list helpers to use RubyGems API in-process; add git proxy reset.
bundler/spec/support/filters.rb	Add `git:` version filter support for selectively skipping specs by git version.
bundler/spec/support/builders.rb	Build gems in-process via `Gem::Package.build` for skip-validation path.
bundler/spec/spec_helper.rb	Disable retry delays in tests and isolate global git config via env vars when supported.
bundler/spec/realworld/fixtures/warbler/Gemfile.lock	Update fixture `BUNDLED WITH` to 4.0.9.
bundler/spec/realworld/fixtures/tapioca/Gemfile.lock	Update fixture `BUNDLED WITH` to 4.0.9.
bundler/spec/install/gems/compact_index_spec.rb	Switch from shelling out to `gem` to in-process uninstall helper.
bundler/spec/commands/clean_spec.rb	Switch from `gem list` subprocess to in-process listing helper.
bundler/spec/commands/check_spec.rb	Switch from `gem uninstall` subprocess to in-process uninstall helper.
bundler/spec/bundler/worker_spec.rb	Add spec coverage for priority-queue processing in worker pool.
bundler/spec/bundler/source/git/git_proxy_spec.rb	Update tests for git version memoization + fetch/clone depth fallback behavior.
bundler/spec/bundler/retry_spec.rb	Add test coverage for exponential backoff/jitter/max delay behavior.
bundler/spec/bundler/installer/parallel_installer_spec.rb	New spec validating native-extension prioritization in parallel installer.
bundler/spec/bundler/gem_helper_spec.rb	Switch from `gem list` subprocess to in-process listing helper.
bundler/spec/bundler/env_spec.rb	Update git version reporting spec to reflect new capture mechanism.
bundler/lib/bundler/worker.rb	Add a priority queue for worker jobs.
bundler/lib/bundler/version.rb	Bump Bundler version to 4.0.9.
bundler/lib/bundler/source/rubygems.rb	Split download vs install and cache installer objects; add synchronization.
bundler/lib/bundler/source/git/git_proxy.rb	Memoize git version once per run; add fetch retry without `--depth` for dumb HTTP.
bundler/lib/bundler/source.rb	Introduce default `download` hook on sources.
bundler/lib/bundler/settings.rb	Add `installation_parallelization` helper (jobs vs CPU count).
bundler/lib/bundler/self_manager.rb	Call `download` before `install` when installing a specific bundler version.
bundler/lib/bundler/retry.rb	Add exponential backoff + jitter + max delay, configurable via base delay.
bundler/lib/bundler/plugin/installer.rb	Call `download` before `install` when installing plugins from specs.
bundler/lib/bundler/plugin/api/source.rb	Add source-plugin `download` hook to split download and install phases.
bundler/lib/bundler/man/bundle-config.1.ronn	Update `jobs` documentation to reflect download + install parallelism.
bundler/lib/bundler/man/bundle-config.1	Generated manpage update matching the `.ronn` change.
bundler/lib/bundler/installer/parallel_installer.rb	Implement separate download/install states and enqueue priority for native extensions.
bundler/lib/bundler/installer/gem_installer.rb	Add `download` path that calls `spec.source.download`.
bundler/lib/bundler/installer.rb	Use unified `Bundler.settings.installation_parallelization`.
bundler/lib/bundler/fetcher/gem_remote_fetcher.rb	Tie HTTP pool size to `installation_parallelization`.
bundler/lib/bundler/definition.rb	Use `installation_parallelization` for git source preloading worker count.
bundler/lib/bundler/cli/pristine.rb	Use `installation_parallelization` for pristine parallelism.
bundler/CHANGELOG.md	Add 4.0.9 section (currently formatted differently than existing Bundler changelog entries).
CHANGELOG.md	Add RubyGems 4.0.9 release notes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-25T00:44:15Z

bundler/lib/bundler/self_manager.rb

+      spec.source.download(spec)
      spec.source.install(spec)


spec.source.download(spec) is invoked without an options hash here. If a source plugin overrides download(spec, opts) (as documented in the plugin API), this call will raise unless opts is optional. Prefer calling download(spec, {}) (or download(spec, options)), and keep call sites consistent with the plugin API signature.

Suggested change

spec.source.download(spec)

spec.source.install(spec)

spec.source.download(spec, {})

spec.source.install(spec, {})

Copilot · 2026-03-25T00:44:15Z

bundler/spec/support/filters.rb

+git_version = Gem::Version.new(`git --version`[/(\d+\.\d+\.\d+)/, 1])
+


git_version is computed eagerly at file load time and will raise if git is missing or the version output can’t be parsed (regex returns nil). That would prevent the entire spec suite from loading. Consider guarding with a presence check (e.g., Bundler.git_present?) and/or rescuing, defaulting to a very low version so git:-tagged specs are skipped cleanly.

Copilot · 2026-03-25T00:44:15Z

bundler/lib/bundler/installer/parallel_installer.rb

        raise "failed to find a spec to enqueue while installing serially" unless spec_install = @specs.find(&:ready_to_enqueue?)
        spec_install.state = :enqueued
+        do_download(spec_install, 0)
        do_install(spec_install, 0)


install_serially downloads and then immediately installs the next :none spec without checking whether it’s actually ready to install (e.g., native extension gems that need their dependencies installed first). This can reintroduce the original install-order issues when @size is 1. Consider applying the same ready_to_install?/dependency gating logic used in the parallel path, even in serial mode.

Suggested change

raise "failed to find a spec to enqueue while installing serially" unless spec_install = @specs.find(&:ready_to_enqueue?)

spec_install.state = :enqueued

do_download(spec_install, 0)

do_install(spec_install, 0)

if spec_to_install = @specs.find(&:ready_to_install?)

do_install(spec_to_install, 0)

next

end

spec_to_enqueue = @specs.find(&:ready_to_enqueue?)

raise "failed to find a spec to enqueue while installing serially" unless spec_to_enqueue

spec_to_enqueue.state = :enqueued

do_download(spec_to_enqueue, 0)

Copilot · 2026-03-25T00:44:16Z

bundler/lib/bundler/plugin/installer.rb


        specs.each do |spec|
-          spec.source.install spec
+          spec.source.download(spec)


spec.source.download(spec) is invoked without an options hash here. If a source plugin overrides download(spec, opts) (as documented in the plugin API), this call will raise unless opts is optional. Prefer calling download(spec, {}) (or passing the same options hash you pass to install).

Suggested change

spec.source.download(spec)

spec.source.download(spec, {})

Copilot · 2026-03-25T00:44:16Z

bundler/spec/spec_helper.rb

+    # Prevent tests from modifying the user's global git config.
+    # GIT_CONFIG_GLOBAL and GIT_CONFIG_NOSYSTEM are available since Git 2.32.
+    git_version = `git --version`[/(\d+\.\d+\.\d+)/, 1]
+    if Gem::Version.new(git_version) >= Gem::Version.new("2.32")
+      ENV["GIT_CONFIG_GLOBAL"] = File.join(ENV["HOME"], ".gitconfig")
+      ENV["GIT_CONFIG_NOSYSTEM"] = "1"
+    end


This git --version probe will raise if git is not available or the version string can’t be parsed (e.g., git_version becomes nil). Since this runs in before :suite, it can take down the whole spec run. Consider checking for git availability (or rescuing) and only setting GIT_CONFIG_GLOBAL/GIT_CONFIG_NOSYSTEM when git is present and the parsed version is valid.

Copilot · 2026-03-25T00:44:16Z

bundler/CHANGELOG.md

+## 4.0.9 / 2026-03-25
+
+### Enhancements:
+
+* Check the git version only **once** per `bundle install`. Pull request [#9406](https://github.com/ruby/rubygems/pull/9406) by Edouard-chin
+* Normalize the number of workers when performing parallel operations. Pull request [#9400](https://github.com/ruby/rubygems/pull/9400) by Edouard-chin
+* Add exponential backoff to bundler retries. Pull request [#9163](https://github.com/ruby/rubygems/pull/9163) by ChrisBr
+* Introduce a priority queue. Pull request [#9389](https://github.com/ruby/rubygems/pull/9389) by Edouard-chin
+* Split the download and install process of a gem. Pull request [#9381](https://github.com/ruby/rubygems/pull/9381) by Edouard-chin
+
+### Bug fixes:
+
+* Retry git fetch without --depth for dumb HTTP transport. Pull request [#9405](https://github.com/ruby/rubygems/pull/9405) by hsbt


The new 4.0.9 entry doesn’t match the established Bundler changelog format below (e.g. ## 4.0.8 (YYYY-MM-DD) and indented - bullets with PR links, without “Pull request … by …” text). Please reformat this entry to be consistent with the rest of bundler/CHANGELOG.md.

Suggested change

## 4.0.9 / 2026-03-25

### Enhancements:

* Check the git version only **once** per `bundle install`. Pull request [#9406](https://github.com/ruby/rubygems/pull/9406) by Edouard-chin

* Normalize the number of workers when performing parallel operations. Pull request [#9400](https://github.com/ruby/rubygems/pull/9400) by Edouard-chin

* Add exponential backoff to bundler retries. Pull request [#9163](https://github.com/ruby/rubygems/pull/9163) by ChrisBr

* Introduce a priority queue. Pull request [#9389](https://github.com/ruby/rubygems/pull/9389) by Edouard-chin

* Split the download and install process of a gem. Pull request [#9381](https://github.com/ruby/rubygems/pull/9381) by Edouard-chin

### Bug fixes:

* Retry git fetch without --depth for dumb HTTP transport. Pull request [#9405](https://github.com/ruby/rubygems/pull/9405) by hsbt

## 4.0.9 (2026-03-25)

### Enhancements:

- Check the git version only **once** per `bundle install` [#9406](https://github.com/ruby/rubygems/pull/9406)

- Normalize the number of workers when performing parallel operations [#9400](https://github.com/ruby/rubygems/pull/9400)

- Add exponential backoff to bundler retries [#9163](https://github.com/ruby/rubygems/pull/9163)

- Introduce a priority queue [#9389](https://github.com/ruby/rubygems/pull/9389)

- Split the download and install process of a gem [#9381](https://github.com/ruby/rubygems/pull/9381)

### Bug fixes:

- Retry git fetch without --depth for dumb HTTP transport [#9405](https://github.com/ruby/rubygems/pull/9405)

Copilot · 2026-03-25T00:44:17Z

bundler/lib/bundler/installer/parallel_installer.rb

+      elsif spec.ready_to_install?(installed_specs)
+        spec.state = :installable
+      end
+
+      worker_pool.enq(spec, priority: spec.enqueue_with_priority?)


process_specs re-enqueues specs even when they are in :downloaded state but not yet ready_to_install?. Since the worker pool lambda treats any non-:enqueued/non-:installable state as a no-op, this can devolve into a tight deq/enq loop (high CPU + queue contention) until dependencies become installed. Consider not re-enqueuing until the spec becomes installable, and instead enqueue downloaded specs when installed_specs changes (e.g., after an install completes, scan for :downloaded specs that are now ready_to_install?).

Copilot · 2026-03-25T00:44:17Z

bundler/lib/bundler/source/rubygems.rb

+      def rubygems_gem_installer(spec, options)
+        @gem_installers_mutex.synchronize { @gem_installers[spec.name] } || begin
+          path = fetch_gem_if_possible(spec, options[:previous_spec])
+          raise GemNotFound, "Could not find #{spec.file_name} for installation" unless path
+
+          REQUIRE_MUTEX.synchronize { require_relative "../rubygems_gem_installer" }
+
+          installer = Bundler::RubyGemsGemInstaller.at(
+            path,
+            security_policy: Bundler.rubygems.security_policies[Bundler.settings["trust-policy"]],
+            install_dir: rubygems_dir.to_s,
+            bin_dir: Bundler.system_bindir.to_s,
+            ignore_dependencies: true,
+            wrappers: true,
+            env_shebang: true,
+            build_args: options[:build_args],
+            bundler_extension_cache_path: extension_cache_path(spec)
+          )
+          @gem_installers_mutex.synchronize { @gem_installers[spec.name] ||= installer }
+        end


@gem_installers is keyed only by spec.name, but Bundler can legitimately have multiple specs with the same name (different versions and/or platforms) in a single resolution. This cache can return the wrong installer (wrong .gem path / checksum) and install the wrong artifact. Key the cache by something version+platform specific (e.g. spec.full_name or [spec.name, spec.version, spec.platform]).

Copilot · 2026-03-25T00:44:17Z

bundler/lib/bundler/source/rubygems.rb

+      # We are using a mutex to reaed and write from/to the hash.
+      # The reason this double synchronization was added is for performance
+      # and lock the mutex for the shortest possible amount of time. Otherwise,
+      # all threads are fighting over this mutex and when it gets acquired it gets locked
+      # until a threads finishes downloading a gem, leaving the other threads waiting


Typo in comment: “reaed” → “read”, and “until a threads finishes” → “until a thread finishes”.

Suggested change

# We are using a mutex to reaed and write from/to the hash.

# The reason this double synchronization was added is for performance

# and lock the mutex for the shortest possible amount of time. Otherwise,

# all threads are fighting over this mutex and when it gets acquired it gets locked

# until a threads finishes downloading a gem, leaving the other threads waiting

# We are using a mutex to read and write from/to the hash.

# The reason this double synchronization was added is for performance

# and lock the mutex for the shortest possible amount of time. Otherwise,

# all threads are fighting over this mutex and when it gets acquired it gets locked

# until a thread finishes downloading a gem, leaving the other threads waiting

Copilot · 2026-03-25T00:44:17Z

bundler/lib/bundler/plugin/api/source.rb

+        # installation of a gem.
+        #
+        # @return [Boolean] Whether the download of the gem succeeded.
+        def download(spec, opts); end


The new download hook is called in Bundler with a single argument in some places (e.g. spec.source.download(spec)), but this default definition requires two positional args. Make opts optional (e.g. default to {}) to avoid ArgumentError for source plugins, and consider returning a boolean (true) from the default no-op implementation to match the YARD contract.

Suggested change

def download(spec, opts); end

def download(spec, opts = {})

true

end

Edouard-chin and others added 16 commits March 25, 2026 09:34

Merge pull request #9381 from Shopify/ec-split-download-install

267bb16

Split the download and install process of a gem (cherry picked from commit bcc4469)

Merge pull request #9222 from afurm/fix-which-command-test-isolation

224508e

Fix gem which command test isolation (cherry picked from commit 8b405fa)

Merge pull request #9389 from Shopify/ec-queue-with-priority

7ca6583

Introduce a priority queue (cherry picked from commit 115228e)

Merge pull request #9403 from gjtorikian/include-owner-role-in-gem-owner

5bcb26c

fix: include owner role in `gem owner` (cherry picked from commit 0bcfe4b)

Merge pull request #9405 from ruby/fix-gh-9049

998ccb2

Retry git fetch without --depth for dumb HTTP transport (cherry picked from commit cb6fbe3)

Merge pull request #9163 from Shopify/cbruckmayer/add-exponential-bac…

0c8abca

…koff Add exponential backoff to bundler retries (cherry picked from commit fd31044)

Merge pull request #9055 from zirni/fix-gem-sources-trailing-slash

c670ce7

fix: Ensure trailing slash is added to source URIs added via gem sources (cherry picked from commit 19debfb)

Merge pull request #9397 from ruby/prevent-git-config

6a89954

Prevent tests from modifying user's global git config (cherry picked from commit 1be513c)

Merge pull request #9195 from ruby/colby/remove-test-subshelling

d79e916

Refactor Bundler tests to invoke RubyGems API directly (cherry picked from commit 9d755be)

Merge pull request #9400 from Shopify/ec-more-downloads

dd4cc31

Normalize the number of workers when performing parallel operations (cherry picked from commit be5febe)

Merge pull request #9406 from Shopify/ec-git-version-class

5bec2a5

Check the git version only **once** per `bundle install` (cherry picked from commit e3685c9)

Merge pull request #9409 from BurdetteLamar/link_fix

515488c

[DOC] Fix link (cherry picked from commit 0e9dd7b)

Changelog for Bundler version 4.0.9

70dd702

Bump Bundler version to 4.0.9

01361dd

Changelog for Rubygems version 4.0.9

5033c7e

Bump Rubygems version to 4.0.9

3ce4a32

Copilot AI review requested due to automatic review settings March 25, 2026 00:35

Copilot started reviewing on behalf of hsbt March 25, 2026 00:35 View session

Copilot AI reviewed Mar 25, 2026

View reviewed changes

hsbt merged commit badae96 into 4.0 Mar 25, 2026
92 checks passed

hsbt deleted the release/4.0.9 branch March 25, 2026 01:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Prepare RubyGems 4.0.9 and Bundler 4.0.9#9426

Prepare RubyGems 4.0.9 and Bundler 4.0.9#9426
hsbt merged 16 commits into4.0from
release/4.0.9

hsbt commented Mar 25, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		git_version = Gem::Version.new(`git --version`[/(\d+\.\d+\.\d+)/, 1])

-        raise "failed to find a spec to enqueue while installing serially" unless spec_install = @specs.find(&:ready_to_enqueue?)
-        spec_install.state = :enqueued
-        do_download(spec_install, 0)
-        do_install(spec_install, 0)
+        if spec_to_install = @specs.find(&:ready_to_install?)
+          do_install(spec_to_install, 0)
+          next
+        end
+        spec_to_enqueue = @specs.find(&:ready_to_enqueue?)
+        raise "failed to find a spec to enqueue while installing serially" unless spec_to_enqueue
+        spec_to_enqueue.state = :enqueued
+        do_download(spec_to_enqueue, 0)

Uh oh!

Conversation

hsbt commented Mar 25, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants