Verify content address test coverage - #186
Merged
jenshenny merged 15 commits intoAug 27, 2026
Merged
Conversation
- test_match: add realistic SHA-256 prefix '78be552b', empty string, non-String types (Integer, Symbol), and whitespace-padded hex - test_content_addressed_with_eligible_spec_and_invalid_address: eligible spec with non-hex content_address should not be considered content-addressed Assisted-By: devx/25d3c4be-88ab-425e-a76b-08a00d8e9d71
jenshenny
marked this pull request as ready for review
August 27, 2026 20:07
jenshenny
force-pushed
the
js/ca-test-gaps-content-address
branch
3 times, most recently
from
August 27, 2026 20:22
2859ce4 to
89aa39b
Compare
There was a problem hiding this comment.
Copilot encountered an error: Your billing is not configured or you have Copilot licenses from multiple standalone organizations or enterprises. To use premium requests, select a billing entity via the GitHub site, under Settings > Copilot > Features.
…ptions - test_ruby_abi_rejects_invalid_format: '3', '3.4.1', 'abc', '3.x' raise - test_ruby_abi_rejects_ruby_platform: no platform set raises - test_ruby_abi_rejects_nil_platform: nil platform raises - test_ruby_abi_rejects_mismatched_required_ruby_version: spec with ~> 3.3.0 and --ruby-abi 3.4 raises - test_ruby_abi_defaults_required_ruby_version_when_unset: no RRV set defaults to ~> 3.4.0, verified from built gem spec - test_ruby_abi_with_output_raises: both --output and --ruby-abi raises - test_ruby_abi_hash_is_deterministic: two builds produce same SHA - Rename test_handle_options to test_handle_options_force_strict_platform Assisted-By: devx/25d3c4be-88ab-425e-a76b-08a00d8e9d71
- test_execute_with_both_selectors_raises_when_multiple_gems_match_without_suggestion: both selectors set, multiple matches, verifies no suggestion is appended - test_execute_with_both_selectors_selects_single_matching_gem: single gem matching both --platform and --ruby-abi pushes successfully - test_execute_with_platform_selector_selects_single_non_content_addressable_gem: single non-content-addressable gem with --platform only pushes - test_execute_with_ruby_abi_selector_matches_source_gem: RUBY-platform gem with ~> 3.4.0 matched by --ruby-abi, documenting ruby_matches? doesn't check platform - Rename fat/skinny/broad references in existing tests to non_content_addressable and content_addressable Assisted-By: devx/25d3c4be-88ab-425e-a76b-08a00d8e9d71
- test_ruby_abi_built_gem_preserves_spec_metadata: reads back built CA gem with Gem::Package.new(file).spec, verifies name, version, platform, required_ruby_version, and ruby_abi are preserved - test_required_ruby_version_unchanged_after_successful_matching_build: builds with matching ~> 3.4.0 and --ruby-abi 3.4, asserts RRV unchanged Assisted-By: devx/25d3c4be-88ab-425e-a76b-08a00d8e9d71
- test_install_assigns_content_address_from_filename: assert the gem is cached under the CA filename (cache/a-2-<address>.gem) in addition to gems/ and specifications/ paths Assisted-By: devx/25d3c4be-88ab-425e-a76b-08a00d8e9d71
- test_execute_remote_content_addressable_and_platform_gems_display_together: add a source gem (version 4, RUBY platform) alongside CA and platform gems to verify it displays as bare version without platform info Assisted-By: devx/25d3c4be-88ab-425e-a76b-08a00d8e9d71
- test_execute_content_addressable_gems_displays_ruby_abis_next_to_their_platforms: same version, different platforms/ABIs on separate lines - test_execute_content_addressable_gems_displays_multiple_ruby_abis_on_the_same_line: same version+platform, different ABIs grouped - test_execute_content_addressable_gems_displays_multiple_versions_on_separate_lines: multiple versions each on own line - test_execute_platform_gem_displays_version_once_for_multiple_platforms: platform gem with multiple platforms shows version once - test_execute_content_addressable_platform_and_source_gems_display_together: mixed CA, platform, and source gems display together Assisted-By: devx/25d3c4be-88ab-425e-a76b-08a00d8e9d71
- test_execute_remote_content_addressable_gem_displays_multiple_ruby_abis_on_same_platform: same version+platform, different ABIs grouped on one line - test_execute_remote_content_addressable_platform_and_source_gems_display_together: mixed CA, platform, and source gems; source gem appears in header but not in Platforms section - test_execute_remote_platform_gem_displays_version_once_for_multiple_platforms: platform gem with multiple platforms uses comma-separated display format Assisted-By: devx/25d3c4be-88ab-425e-a76b-08a00d8e9d71
- test_load_specs_compact_index_decodes_mixed_content_addressable_and_platform_entries: versions list with both CA hash and platform suffix; verifies both returned, CA entry has content_address and ruby_abi, platform entry does not - test_load_specs_compact_index_content_addressable_without_ruby_field: /info/ metadata with platform:= but no ruby: field; verifies ruby_abi is nil while platform and content_address are still set Assisted-By: devx/25d3c4be-88ab-425e-a76b-08a00d8e9d71
- test_prefers_compatible_content_addressed_gem_when_multiple_abis_available: two CA gems with different ABIs, one incompatible, verifies compatible one is selected - test_raises_when_only_content_addressed_gem_is_incompatible: only CA gem with incompatible required_ruby_version, no fallback, verifies DependencyResolutionError is raised - Rename test_falls_back_to_fat_* to test_falls_back_to_non_content_addressable_* and fat_spec to non_content_addressable_spec Assisted-By: devx/25d3c4be-88ab-425e-a76b-08a00d8e9d71
- test_ruby_abi_returns_nil_for_default_required_ruby_version: a fresh spec with default required_ruby_version (>= 0) returns nil for ruby_abi, covering the most common case of gems without RRV Assisted-By: devx/25d3c4be-88ab-425e-a76b-08a00d8e9d71
…sable - test_content_addressable_spec_name: verifies spec_name uses CA suffix (a-1-abcdef12.gemspec) - test_content_addressable_tuples_with_different_addresses_are_distinct: verifies == includes content_address but hash does not (based on to_a), documenting hash collision behavior - Rename test_fat_tuple_* to test_non_content_addressable_tuple_* and fat/skinny variables to non_content_addressable/content_addressable Assisted-By: devx/25d3c4be-88ab-425e-a76b-08a00d8e9d71
- Add test for two CA gems with different mismatched ABIs plus non-CA platform fallback, verifies non-CA gem is installed - Rename 'fat' to remove it from test #7 description for consistency Assisted-By: devx/25d3c4be-88ab-425e-a76b-08a00d8e9d71
- test_execute_with_ruby_abi_and_platform_no_matching_gem_displays_error: verifies yank with both --platform and --ruby-abi sends all params to API and displays the 404 error message from rubygems.org Assisted-By: devx/2658d4ad-be37-4929-8dda-b1d772952c7b
- test_install_assigns_content_address_from_filename_with_full_sha: verifies installer accepts full 64-char SHA256 as content address - test_two_content_addressed_gems_with_same_name_version_coexist: verifies two gems with same name/version/platform but different content addresses install to distinct directories and gemspecs without overwriting Assisted-By: devx/2658d4ad-be37-4929-8dda-b1d772952c7b
jenshenny
force-pushed
the
js/ca-test-gaps-content-address
branch
from
August 27, 2026 20:57
6ffb66b to
8f38fbc
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
test/rubygems/test_gem_commands_build_command.rb:240
- This glob can return the first build's existing file after the second build, so the assertion may pass even if the second build produced a different content address. Return the filename from
@cmd.executedirectly so each invocation's actual result is compared.
Dir[File.join(@tempdir, "platformed_gem-2-*.gem")].first
jenshenny
merged commit Aug 27, 2026
aa7f984
into
feature-branch-ca-changes-rubygems
107 of 110 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Audited all 37 test files changed in ruby#9773 (content-addressable gems support) and added tests for gaps in CA-specific behavior — edge cases in validation, command flows, resolution, serialization, and display that the PR's tests didn't cover. Also cross-referenced the manual QA spreadsheet to map each manual test case to automated coverage.
QA Spreadsheet Cross-Reference
The QA spreadsheet defines manual test cases per command. Below, each case is mapped to automated coverage status: Covered (existing) = PR already has an automated test, Covered (added) = this PR adds an automated test, Gap = no automated test (manual QA only or untested).
gem build (8 cases)
test_raise_if_ruby_abi_is_not_in_x_y_formatin package.rb)gem push (10 cases)
gem list (6 cases)
gem search (4 cases)
gem info (4 cases)
gem yank (6 cases)
test_execute)test_execute)test_execute_with_ruby_abi_sends_platform_and_ruby_abi_to_yank_api)test_execute_with_ruby_abi_and_platform_no_matching_gem_displays_error)gem install — local (16 cases)
gem install — remote (22 cases)
remote_specification_spec.rbconfirms content_address always nil on legacy index by design)bundle install — remote (25 cases)
test_gem_installer.rbtest_gem_source.rbbundle install — local (14 cases)
test_gem_installer.rb)lockfile_parser_spec.rbfalse positive test)lockfile_parser_spec.rb)content_addressable_spec.rb)Files with test additions (16 files, 41 tests)
test/rubygems/test_gem_content_address.rbAlready covered: match with various prefix lengths, uppercase rejection, eligible spec with/without address, ineligible spec with address, applicability checks (RRV, platform, RUBY platform, nil platform)
Added — 5 tests:
test/rubygems/test_gem_commands_build_command.rbAlready covered:
--ruby-abioption parsing, filename generation for CA gems, basic--executewith ruby_abiAdded — 6 tests:
X.Y) raises with message--ruby-abirejectedrequired_ruby_versionand--ruby-abiraisesrequired_ruby_versionto~> X.Y.0when--ruby-abiset and RRV unset--output+--ruby-abiraises (CA gems must use generated filename)test/rubygems/test_gem_commands_push_command.rbAlready covered: platform selector matching, ruby_abi selector matching, invalid ruby_abi rejection, invalid gem package skipping, combined platform + ruby selectors, same name/version/platform selecting by ruby_abi, non-CA vs CA candidate selection, broad ruby requirement rejection, no match error, multiple match error, multiple ruby_abis error, multiple platforms error, exact filename suggestion, no selectors rejecting multiple gems
Added — 4 tests:
test/rubygems/test_gem_package.rbAlready covered: CA file creation with ruby_abi, no CA file without ruby_abi, RRV set by ruby_abi if default, RRV not modified if build fails, RRV/ruby_abi conflict raises, invalid ruby_abi format raises, non-platformed spec with ruby_abi raises, explicit output keeps filename, explicit output + ruby_abi raises
Added — 2 tests:
required_ruby_versionunchanged after successful matching buildtest/rubygems/test_gem_installer.rbAlready covered: content_address assigned from filename, mismatched content_address raises, non-CA gems install normally, numeric version not treated as CA, hex suffix without matching spec prefix not CA, CA not set without RRV + platform, CA not set with only RRV, CA not set with only platform, require works after CA install, reinstalling CA gem is idempotent
Added — 3 tests:
test_install_assigns_content_address_from_filenametest/rubygems/test_gem_commands_list_command.rbAlready covered: unscoped CA gems don't fetch metadata, CA gem displays real platform + ruby_abi, multiple ruby_abis on same line, multiple versions on separate lines, CA + platform gems display together, platform gem displays version once for multiple platforms
Added — 1 test:
test/rubygems/test_gem_commands_search_command.rbAlready covered: unscoped CA gems don't fetch metadata, CA gem displays real platform + ruby_abi, CA + platform gems display together
Added — 5 tests:
test/rubygems/test_gem_commands_info_command.rbAlready covered: unscoped CA gems don't fetch metadata, CA gem displays real platform + ruby_abi, ruby_abis next to platforms, CA + platform gems display together
Added — 3 tests:
test/rubygems/test_gem_source.rbAlready covered: compact index loads CA metadata, skips CA rows without metadata, skips CA rows without required platform, doesn't infer ruby_abi from broad ruby requirement, latest keeps CA ruby_abi variants, decode groups by gem name before platform, latest per platform
Added — 2 tests:
ruby:field (inconsistent candidate prevention)test/rubygems/test_gem_resolver.rbAlready covered: prefers CA gem for same platform, falls back to non-CA when CA requires other rubygems version, falls back to source when CA requires other ruby, falls back to non-CA before source when CA requires other ruby
Added — 2 tests:
test/rubygems/test_gem_specification.rbAlready covered: ruby_abi derived from RRV (
~> 3.4.0→3.4), nil for pessimistic without patch segment, nil for nonzero patch segment, nil for multiple requirements, nil for dev version, nil for non-pessimistic operator, CA full_name, CA specs distinct by ==, CA to_ruby serializationAdded — 1 test:
ruby_abireturns nil for defaultrequired_ruby_version(>= 0)test/rubygems/test_gem_name_tuple.rbAlready covered: CA metadata (content_address, ruby_abi, full_name), non-CA tuple doesn't store nil ivars, mixed CA/non-CA sorting
Added — renames only:
test_fat_tuple_*→test_non_content_addressable_tuple_*test_sort_mixed_fat_*→test_sort_mixed_non_content_addressable_*, variablesfat/skinny→non_content_addressable/content_addressableto_a/hashfix lives)test/rubygems/test_gem_commands_yank_command.rbAlready covered: ruby_abi option parsing, invalid ruby_abi format rejection, yank with platform, yank with platform + ruby_abi, yank with ruby_abi but no platform (API error)
Added — 1 test:
spec/install/gemfile/content_addressable_spec.rbAlready covered: installs CA gem when ABI matches (with cache file + lockfile verification), resolves CA binary from local cache after lockfile round-trip, falls back to non-CA on ABI mismatch, hex suffix not treated as CA without platform metadata, falls back to non-CA on platform mismatch, selects matching platform from multiple CA gems, falls back to pure-ruby on ABI mismatch, selects ABI-compatible CA from multiple CA gems, higher non-CA version over lower CA version
Added — 3 tests:
Files reviewed with no gaps found (23 files)
RubyGems tests
test_gem_package_task.rb— no CA-specific behaviortest_gem_commands_install_command.rb— cooldown only; local CA install is generic pass-throughtest_gem_commands_update_command.rb— cooldown onlytest_gem_commands_outdated_command.rb— cooldown onlytest_gem_resolver_api_specification.rb— no actionable CA gapstest_gem_stub_specification.rb— CA stub parsing (init with target, distinctness) well coveredtest_gem_cooldown.rb— cooldown only, CA not yet implemented for cooldowntest_gem_compact_index_client_parser.rb— parser edge cases, CA detection at source leveltest_gem_ext_builder.rb— shellsplit/run, no CA interactiontest_gem_safe_marshal.rb— 2 CA marshal round-trip tests already cover forward/backward compattest_gem_safe_yaml.rb— YAML parsing, no CA testsBundler specs
spec/bundler/endpoint_specification_spec.rb— CA detection, metadata parsing, RRV loading well coveredspec/bundler/lockfile_parser_spec.rb— 2 CA tests for lockfile parsing + false positive preventionspec/bundler/remote_specification_spec.rb— 3 tests confirm content_address always nil on legacy index (by design)spec/install/cooldown_spec.rb— cooldown onlyspec/install/gems/compact_index_spec.rb— cleanup, no CA contentspec/lock/lockfile_spec.rb— refactoring only; CA lockfile format covered incontent_addressable_spec.rbspec/commands/install_spec.rb— CA install covered incontent_addressable_spec.rbspec/commands/lock_spec.rb— CA lock covered incontent_addressable_spec.rbandext_spec.rbspec/install/deploy_spec.rb— CA deploy (lockfile-driven local install) covered incontent_addressable_spec.rbspec/install/global_cache_spec.rb— cache path refactoring; CA cache filenames covered incontent_addressable_spec.rbspec/bundler/fetcher_spec.rb— SSL/proxy refactoring; fetcher is CA-agnosticNaming convention
Renamed all occurrences of
fat/skinny/broadtonon_content_addressable/content_addressable(full words, no abbreviations) in test names, variables, and gem content strings across all affected files.