Skip to content

Conversation

yahonda
Copy link
Member

@yahonda yahonda commented Oct 13, 2025

This pull request skips Railties test failures/errors reported at
https://buildkite.com/rails/rails-nightly/builds/2969#0199cfed-068b-44bc-9ec4-b48d4395d8e3/1403-1454
to follow up #55883

Motivation / Background

This commit skips ApiAppGeneratorTest and AppGeneratorTest for Ruby 3.5.0dev until budndler-audit support it.

Detail

Steps to reproduce

  1. Update railties/test/generators/generators_test_helper.rb not to silence errors
$ bundle update --bundler
$ git diff
diff --git a/Gemfile.lock b/Gemfile.lock
index 7a4c7e53ed..0b4bf3e94d 100644
--- a/Gemfile.lock
+++ b/Gemfile.lock
@@ -151,9 +151,6 @@ GEM
     brakeman (7.0.0)
       racc
     builder (3.3.0)
-    bundler-audit (0.9.2)
-      bundler (>= 1.2.0, < 3)
-      thor (~> 1.0)
     bunny (2.23.0)
       amq-protocol (~> 2.3, >= 2.3.1)
       sorted_set (~> 1, >= 1.0.2)
@@ -755,7 +752,6 @@ DEPENDENCIES
   bcrypt (~> 3.1.11)
   bootsnap (>= 1.4.4)
   brakeman
-  bundler-audit
   capybara (>= 3.39)
   connection_pool
   cssbundling-rails
@@ -832,4 +828,4 @@ DEPENDENCIES
   websocket-client-simple

 BUNDLED WITH
-   2.7.0
+   4.0.0.dev
diff --git a/railties/test/generators/generators_test_helper.rb b/railties/test/generators/generators_test_helper.rb
index 6bc08c3ca0..d0522bf878 100644
--- a/railties/test/generators/generators_test_helper.rb
+++ b/railties/test/generators/generators_test_helper.rb
@@ -147,7 +147,7 @@ def run_app_update(app_root = destination_root, flags: "--force")
       gemfile_contents.sub!(/^(gem "rails").*/, "\\1, path: #{File.expand_path("../../..", __dir__).inspect}")
       File.write("Gemfile", gemfile_contents)

-      silence_stream($stdout) { system({ "BUNDLE_GEMFILE" => "Gemfile" }, "bin/rails app:update #{flags}", exception: true) }
+      system({ "BUNDLE_GEMFILE" => "Gemfile" }, "bin/rails app:update #{flags}", exception: true)
     end
   end

Actual result without skipping it

$ bin/test test/generators/api_app_generator_test.rb -n test_app_update_does_not_generate_public_files
Run options: -n test_app_update_does_not_generate_public_files --seed 33882

Could not find gem 'bundler-audit' in locally installed gems.
Run `bundle install` to install missing gems.
E

Error:
ApiAppGeneratorTest#test_app_update_does_not_generate_public_files:
RuntimeError: Command failed with exit 7: bin/rails
    test/generators/generators_test_helper.rb:150:in 'Kernel#system'
    test/generators/generators_test_helper.rb:150:in 'block in GeneratorsTestHelper#run_app_update'
    test/generators/generators_test_helper.rb:145:in 'Dir.chdir'
    test/generators/generators_test_helper.rb:145:in 'GeneratorsTestHelper#run_app_update'
    test/generators/api_app_generator_test.rb:138:in 'ApiAppGeneratorTest#test_app_update_does_not_generate_public_files'

bin/test test/generators/api_app_generator_test.rb:136

Finished in 0.136068s, 7.3493 runs/s, 0.0000 assertions/s.
1 runs, 0 assertions, 0 failures, 1 errors, 0 skips
$

Since more than 20 failures or errors appeared in AppGeneratorTest this commit skips entire AppGeneratorTest for Ruby 3.5.0dev tentatively.

  • Number of failures or errors to be skipped
$ bin/test test/generators/app_generator_test.rb
192 runs, 1575 assertions, 2 failures, 23 errors, 0 skips

$ bin/test test/generators/api_app_generator_test.rb
10 runs, 151 assertions, 0 failures, 1 errors, 0 skips

Additional information

Follow up #55883
Related to rubysec/bundler-audit#405

Checklist

Before submitting the PR make sure the following are checked:

  • This Pull Request is related to one change. Unrelated changes should be opened in separate PRs.
  • Commit message has a detailed description of what changed and why. If this PR fixes a related issue include it in the commit message. Ex: [Fix #issue-number]
  • Tests are added or updated if you fix a bug or add a feature.
  • CHANGELOG files are updated for the changed libraries if there is a behavior change or additional feature. Minor bug fixes and documentation changes should not be included.

…tively

This commit skips ApiAppGeneratorTest and AppGeneratorTest for Ruby 3.5.0dev
until budndler-audit support it.

-- Steps to reproduce

1. Update `railties/test/generators/generators_test_helper.rb` not to silence errors

```
$ bundle update --bundler
$ git diff
diff --git a/Gemfile.lock b/Gemfile.lock
index 7a4c7e5..0b4bf3e94d 100644
--- a/Gemfile.lock
+++ b/Gemfile.lock
@@ -151,9 +151,6 @@ GEM
     brakeman (7.0.0)
       racc
     builder (3.3.0)
-    bundler-audit (0.9.2)
-      bundler (>= 1.2.0, < 3)
-      thor (~> 1.0)
     bunny (2.23.0)
       amq-protocol (~> 2.3, >= 2.3.1)
       sorted_set (~> 1, >= 1.0.2)
@@ -755,7 +752,6 @@ DEPENDENCIES
   bcrypt (~> 3.1.11)
   bootsnap (>= 1.4.4)
   brakeman
-  bundler-audit
   capybara (>= 3.39)
   connection_pool
   cssbundling-rails
@@ -832,4 +828,4 @@ DEPENDENCIES
   websocket-client-simple

 BUNDLED WITH
-   2.7.0
+   4.0.0.dev
diff --git a/railties/test/generators/generators_test_helper.rb b/railties/test/generators/generators_test_helper.rb
index 6bc08c3..d0522bf878 100644
--- a/railties/test/generators/generators_test_helper.rb
+++ b/railties/test/generators/generators_test_helper.rb
@@ -147,7 +147,7 @@ def run_app_update(app_root = destination_root, flags: "--force")
       gemfile_contents.sub!(/^(gem "rails").*/, "\\1, path: #{File.expand_path("../../..", __dir__).inspect}")
       File.write("Gemfile", gemfile_contents)

-      silence_stream($stdout) { system({ "BUNDLE_GEMFILE" => "Gemfile" }, "bin/rails app:update #{flags}", exception: true) }
+      system({ "BUNDLE_GEMFILE" => "Gemfile" }, "bin/rails app:update #{flags}", exception: true)
     end
   end
```

- Actual result without skipping it

```ruby
$ bin/test test/generators/api_app_generator_test.rb -n test_app_update_does_not_generate_public_files
Run options: -n test_app_update_does_not_generate_public_files --seed 33882

Could not find gem 'bundler-audit' in locally installed gems.
Run `bundle install` to install missing gems.
E

Error:
ApiAppGeneratorTest#test_app_update_does_not_generate_public_files:
RuntimeError: Command failed with exit 7: bin/rails
    test/generators/generators_test_helper.rb:150:in 'Kernel#system'
    test/generators/generators_test_helper.rb:150:in 'block in GeneratorsTestHelper#run_app_update'
    test/generators/generators_test_helper.rb:145:in 'Dir.chdir'
    test/generators/generators_test_helper.rb:145:in 'GeneratorsTestHelper#run_app_update'
    test/generators/api_app_generator_test.rb:138:in 'ApiAppGeneratorTest#test_app_update_does_not_generate_public_files'

bin/test test/generators/api_app_generator_test.rb:136

Finished in 0.136068s, 7.3493 runs/s, 0.0000 assertions/s.
1 runs, 0 assertions, 0 failures, 1 errors, 0 skips
$
```

Since more than 20 failures or errors appeared in AppGeneratorTest
this commit skips entire AppGeneratorTest for Ruby 3.5.0dev tentatively.

- Number of failures or errors to be skipped
```
$ bin/test test/generators/app_generator_test.rb
192 runs, 1575 assertions, 2 failures, 23 errors, 0 skips

$ bin/test test/generators/api_app_generator_test.rb
10 runs, 151 assertions, 0 failures, 1 errors, 0 skips
```

Follow up rails#55883
Related to rubysec/bundler-audit#405
@rails-bot rails-bot bot added the railties label Oct 13, 2025
@yahonda yahonda merged commit 252ee6c into rails:main Oct 13, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant