Retry ActionableErrors when running tests

[andrewn617]

Motivation / Background

This Pull Request has been created because I want to make it easier to retry actionable errors that occur when running tests. Rails already knows how to recover these errors, so lets make it possible for the user to recover from them and run their tests without exiting the process.

Detail

Allow Actionable Errors encountered when running tests to be retried.

Migrations are pending. To resolve this issue, run:

        bin/rails db:migrate

You have 1 pending migration:

db/migrate/20240201213806_add_a_to_b.rb
Run pending migrations? [Yn] Y
== 20240201213806 AddAToB: migrating =========================================
== 20240201213806 AddAToB: migrated (0.0000s) ================================

Running 7 tests in a single process (parallelization threshold is 50)
Run options: --seed 22200

# Running:

.......

Finished in 0.243394s, 28.7600 runs/s, 45.1942 assertions/s.
7 runs, 11 assertions, 0 failures, 0 errors, 0 skips

This feature is only available in an interactive terminal.

Additional information

Right now this is only implemented in the Test Runner, but I'd also like to follow up with a similar implementation for Rails::Command::CorrectableNameErrors.

Checklist

Before submitting the PR make sure the following are checked:

• This Pull Request is related to one change. Changes that are unrelated should be opened in separate PRs.
• Commit message has a detailed description of what changed and why. If this PR fixes a related issue include it in the commit message. Ex: [Fix #issue-number]
• Tests are added or updated if you fix a bug or add a feature.
• CHANGELOG files are updated for the changed libraries if there is a behavior change or additional feature. Minor bug fixes and documentation changes should not be included.

[gmcgibbon]

Looks like we have some railties tests continuously timing out. We'll look into that ASAP.

[andrewn617]

Looks like we have some railties tests continuously timing out. We'll look into that ASAP.

Needed to turn off the flag in the generated app in a few more tests 👍

[gmcgibbon]

@andrewn617 Let's go ahead and rebase + add the new framework default, so new apps can onboard to this behaviour.

[andrewn617]

Ultimately @gmcgibbon and I decided against a framework configuration for this. The other use case we are working on is for "Did you mean" errors when the command name is typoed. In that case, the app is not even loaded yet, so the configuration is not available to be checked. So instead, we are going with STDOUT.tty? for determining if the console is interactive.

[gmcgibbon]

LGTM, @andrewn617 can you please update the description of the PR and git commit message so this is accurate to the new approach. I'll merge after that.

[andrewn617]

@gmcgibbon Done.

[gmcgibbon]

Thanks! ❤️