Use test_runner for tests

[MSP-Greg]

Description

Currently rake is used to run tests. rake creates a main process, and runs tasks in sub-processes. This allows chaining tasks, as we do, allowing compile to run before test.

CI systems' resources are limited, and probably much smaller than most dev's local PC's.

The PR creates two files for test runners (test_runner.cmd is used only on Windows) and uses them in the Actions CI test.yaml script.

Note that test_runner runs the full test suite. So, for local testing, using other code should still be used, unless one is very familiar with minitest, etc. More functionality can be added in the future.

Your checklist for this pull request

• I have reviewed the guidelines for contributing to this repository.
• I have added (or updated) appropriate tests if this PR fixes a bug or adds a feature.
• My pull request is 100 lines added/removed or less so that it can be easily reviewed.
• If this PR doesn't need tests (docs change), I added [ci skip] to the title of the PR.
• If this closes any issues, I have added "Closes #issue" to the PR description or my commit messages.
• I have updated the documentation accordingly.
• All new and existing tests passed, including Rubocop.

[dentarg]

What magic is this? 😆

[MSP-Greg]

Windows fun!

Actually, something very similar is generated by RubyGems when it creates Windows bin stubs.

With *nix systems, one has executable permissions and shebang lines.

On Windows, the extension determines whether a file is executable.

%* is equivalent to ARGV, and %~dpn0 is equivalent to File.basename(__FILE__, '.*')

So, in this instance it is equivalent to ruby.exe -x "test_runner" ARGV

[MSP-Greg]

Apologies for making this a draft PR. GitHub desktop retains PR settings, and my last PR (#3267) was a draft.

Rebased, and fixed a double space in test.yaml:

./test_runner  --verbose  # initial commit
./test_runner --verbose   # force-pushed update

[MSP-Greg]

Ok, I did say "More functionality can be added in the future.". One question, one update...

1. File name & location. I'm ok with it as is, wondering about whether it should be in the test folder, and rename it 'runner'?

2. I've got additions that allow a single file name or a glob, like test_puma_server* or test_integration_*. It has to be the last argument.

[dentarg]

1. Maybe place it in the test directory, the root is kinda clean right now
2. That's sounds like nice additions

[MSP-Greg]

@dentarg

Moved the file and added ability to add a files argument. See comments in the file. Can be a single file, a glob, or a list...

[MSP-Greg]

Sorry. I didn't set the executable bit, as Git has a special command for setting it on Windows systems. I've always used GitHub desktop on Windows for most operations, along with GIT cli...

[dentarg]

👍🏻👌🏻

[MSP-Greg]

With the new runner, running tests shows one main process:

ruby test/runner --verbose

With bundle exec rake test TESTOPTS=--verbose, the two processes are:

/usr/vendor/bundle/puma/22mri/ruby/3.3.0+0/bin/rake test TESTOPTS=--verbose
sh -c /usr/local/bin/ruby -w -I"lib" /usr/vendor/bundle/puma/22mri/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/rake_test_loader.rb "test/test_app_status.rb" "test/test_binder.rb" "test/test_bundle_pruner.rb" "test/test_busy_worker.rb" "test/test_cli.rb" "test/test_config.rb" "test/test_error_logger.rb" "test/test_events.rb" "test/test_example_cert_expiration.rb" "test/test_http10.rb" "test/test_http11.rb" "test/test_integration_cluster.rb" "test/test_integration_pumactl.rb" "test/test_integration_single.rb" "test/test_integration_ssl.rb" "test/test_integration_ssl_session.rb" "test/test_iobuffer.rb" "test/test_json_serialization.rb" "test/test_launcher.rb" "test/test_log_writer.rb" "test/test_minissl.rb" "test/test_null_io.rb" "test/test_out_of_band_server.rb" "test/test_persistent.rb" "test/test_plugin.rb" "test/test_plugin_systemd.rb" "test/test_plugin_systemd_jruby.rb" "test/test_preserve_bundler_env.rb" "test/test_puma_localhost_authority.rb" "test/test_puma_server.rb" "test/test_puma_server_hijack.rb" "test/test_puma_server_ssl.rb" "test/test_pumactl.rb" "test/test_rack_handler.rb" "test/test_rack_server.rb" "test/test_redirect_io.rb" "test/test_request_invalid.rb" "test/test_response_header.rb" "test/test_state_file.rb" "test/test_thread_pool.rb" "test/test_unix_socket.rb" "test/test_url_map.rb" "test/test_web_server.rb" "test/test_worker_gem_independence.rb"

Saving people the time of scrolling the above script window, the 2nd process is:

sh -c /usr/local/bin/ruby -w -I"lib" /usr/vendor/bundle/puma/22mri/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/rake_test_loader.rb

With every test file (in quotes) following it as arguments. Or, it's really pretty in a console window when it wraps...

I've also found this helps speed things up in a shell script I use to run each test file individually, in particular with JRuby.

@nateberkopec Any thoughts?

[MSP-Greg]

I updated the code to properly handle arguments like test/runner -vs33388 and added more notes...