Skip to content

Support Content Addressable File Naming - Gem Build Flow - #170

Closed
OughtPuts wants to merge 4 commits into
masterfrom
ho/ca-changes-gem-build
Closed

Support Content Addressable File Naming - Gem Build Flow#170
OughtPuts wants to merge 4 commits into
masterfrom
ho/ca-changes-gem-build

Conversation

@OughtPuts

@OughtPuts OughtPuts commented Jul 16, 2026

Copy link
Copy Markdown

TL;DR

As part of the work to speed up bundle install, this PR adjusts gem build to support using content-addressable naming at build-time for a gem which only supports a single Ruby ABI.

Summary

This PR introduces changes to the gem build process to assign a content-addressable file name during the build of a gem that only supports one Ruby ABI (a skinny gem).

The bulk of the changes are in Gem::Package. In self.build, we now check if the ruby_abi flag has been set when the gem build command is run. Only if it has do we first validate the Ruby ABI value (and compare this to the required_ruby_version for compatibility etc.) and then build a content addressable file if the validation passes without raising. If the user has passed a file name value at time of running gem build, we raise, as we would not expect a content addressable gem to have a custom file name (it instead has a SHA value based on the file contents). As part of the CA file name compilation, we build the gem first in memory (using StringIO.new), generate the SHA value from the file contents, and then write the file including the SHA value as the filename.

There is also a minor change to package_task which ensures that we are eventually transferring the correct file output of the build (as now the name could take three different forms - standard, platformed or content addressed), and a helper method in specification.rb which helps compare the Ruby ABI to the required_ruby_version provided.

Testing

Tests written in test_gem_commands_build_command, test_gem_package, test_gem_package_task, and test_gem_specification to cover new behaviour.

Tophat

Within IRB terminal, paste this script so you can see the output of using the Gem::Package.build command in different scenarios:

require "rubygems/package"
  require "tmpdir"

  Dir.mktmpdir do |dir|
    Dir.chdir(dir) do
      Dir.mkdir("lib")
      File.write("lib/code.rb", "# hello\n")

      def spec(name, ruby_version: "~> 3.4.0", platform: "arm64-darwin")
        Gem::Specification.new(name, "1") do |s|
          s.summary = name
          s.authors = ["reviewer"]
          s.files = ["lib/code.rb"]
          s.platform = platform
          s.required_ruby_version = ruby_version if ruby_version
        end
      end

      puts "\n1. Explicit ruby_abi builds content-addressed filename"
      Gem::Package.build(spec("skinny"), false, false, nil, "3.4")
      p Dir["skinny-1-*.gem"]

      puts "\n2. No ruby_abi keeps normal platform filename"
      Gem::Package.build(spec("normal"))
      p Dir["normal-1-*.gem"]

      puts "\n3. Missing required_ruby_version is filled from ruby_abi"
      defaulted = spec("defaulted", ruby_version: nil)
      Gem::Package.build(defaulted, false, false, nil, "3.4")
      p Dir["defaulted-1-*.gem"]
      p defaulted.required_ruby_version

      puts "\n4. Conflicting required_ruby_version raises"
      begin
        Gem::Package.build(spec("bad", ruby_version: "~> 3.5.0"), false, false, nil, "3.4")
      rescue ArgumentError => e
        puts e.message
      end

      puts "\nBuilt files:"
      puts Dir["*.gem"].sort
    end
  end

To try the gem build command locally against this branch, you can run it through the RubyGems checkout rather than your installed RubyGems. From a test gem directory with a platformed gemspec, use: ruby --disable-gems -I/path/to/rubygems/lib /path/to/rubygems/exe/gem build *.gemspec --ruby-abi 3.4. As an example, for my checkout that is: ruby --disable-gems -I/Users/harrietoughton/rubygems/lib /Users/harrietoughton/rubygems/exe/gem build *.gemspec --ruby-abi 3.4.

The single Ruby ABI built file should be named <name>-<version>-<sha10>.gem rather than <name>-<version>-<platform>.gem. It's also worth testing out that you receive the expected behaviour with a wrong format ruby ABI (e.g. --ruby-abi 3), with a mismatched Ruby ABI and required_ruby_version and when a gem is platformed with no Ruby ABI and when it is neither platformed nor intended for a single ABI.

@OughtPuts
OughtPuts marked this pull request as draft July 16, 2026 09:23
@OughtPuts
OughtPuts force-pushed the ho/ca-changes-gem-build branch from f8e754f to 56b9bd9 Compare July 16, 2026 17:51
@OughtPuts
OughtPuts force-pushed the ho/ca-changes-gem-build branch 4 times, most recently from b7ed231 to f69c971 Compare July 20, 2026 12:42
@OughtPuts
OughtPuts marked this pull request as ready for review July 20, 2026 14:20
@OughtPuts
OughtPuts requested a review from Copilot July 20, 2026 14:20
@OughtPuts
OughtPuts marked this pull request as draft July 20, 2026 14:25

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the gem build flow to optionally produce content-addressable gem filenames for platform gems that are scoped to a single Ruby ABI (via a new --ruby-abi flag). It introduces Ruby ABI derivation from required_ruby_version, validates/sets ABI requirements during build, and updates build output/packaging behavior accordingly.

Changes:

  • Add Gem::Specification#content_addressable_ruby_abi to derive an X.Y Ruby ABI from a single pessimistic required_ruby_version.
  • Extend Gem::Package.build with an optional ruby_abi argument to produce a digest-derived filename and validate/apply required_ruby_version.
  • Add/adjust tests and CLI help/options for the new build mode, and ensure PackageTask moves whatever filename Gem::Package.build returns.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
test/rubygems/test_gem_specification.rb Adds coverage for deriving a Ruby ABI from required_ruby_version.
test/rubygems/test_gem_package.rb Adds coverage for content-addressable build naming and ruby ABI validation/error cases.
test/rubygems/test_gem_package_task.rb Ensures packaging moves the filename returned by Gem::Package.build.
test/rubygems/test_gem_commands_build_command.rb Adds CLI coverage for --ruby-abi and default option behavior.
lib/rubygems/specification.rb Implements content_addressable_ruby_abi helper.
lib/rubygems/package.rb Implements content-addressable build path, ruby ABI validation/application, and adjusts build output printing.
lib/rubygems/package_task.rb Uses the actual built gem filename when moving artifacts to pkg/.
lib/rubygems/commands/build_command.rb Adds the --ruby-abi option and help text; passes it through to Gem::Package.build.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/rubygems/specification.rb
Comment thread lib/rubygems/package.rb
Comment thread lib/rubygems/package.rb
Comment on lines +139 to +142
bytes = io.string
gem_file = "#{spec.name}-#{spec.version}-#{Digest::SHA256.hexdigest(bytes)[0, 10]}.gem"
File.binwrite(gem_file, bytes)
package.say " File: #{gem_file}"

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A good point here. The requirement in the issue is "The gem is built in memory" so it would depend how tightly we're tied to that decision.

If open to an alternative, I think the next steps are probably to benchmark the largest platformed gem we have on RG.o and see what the build time would be for a Tempfile vs io.string to see what we would be gaining as a tradeoff for the extra complexity around a Tempfile.

Comment thread lib/rubygems/package.rb
Comment thread lib/rubygems/commands/build_command.rb Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants