Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minitest - Add code to group & summarize test skips [changelog skip] #2577

Merged
merged 1 commit into from
Mar 15, 2021

Conversation

MSP-Greg
Copy link
Member

Description

Many CI jobs have a large number of skips, and Minitest's verbose output lists skips, failures, & errors in the order they occurred.

This adds code to helper.rb to group the skips by their skip message, making it easier to see better information about the skips, and places the failure & error logging at the end of the summary output.

Your checklist for this pull request

  • I have reviewed the guidelines for contributing to this repository.
  • I have added an entry to History.md if this PR fixes a bug or adds a feature. If it doesn't need an entry to HISTORY.md, I have added [changelog skip] or [ci skip] to the pull request title.
  • I have added (or updated) appropriate tests if this PR fixes a bug or adds a feature.
  • My pull request is 100 lines added/removed or less so that it can be easily reviewed.
  • If this PR doesn't need tests (docs change), I added [ci skip] to the title of the PR.
  • If this closes any issues, I have added "Closes #issue" to the PR description or my commit messages.
  • I have updated the documentation accordingly.
  • All new and existing tests passed, including Rubocop.

@nateberkopec
Copy link
Member

Gorgeous!

Skips:
────────────────────────────────────────── Kernel.fork isn't available on jruby on java ──
  test_cli.rb
     1) :107  TestCLI test_control_clustered
     2) :358  TestCLI test_state_file_callback_filtering
     3) :375  TestCLI test_log_formatter_default_clustered
     4) :388  TestCLI test_log_formatter_custom_clustered

  test_integration_pumactl.rb
     5) :61   TestIntegrationPumactl test_phased_restart_cluster
     6) :96   TestIntegrationPumactl test_prune_bundler_with_multiple_workers

  test_launcher.rb
     7) :141  TestLauncher test_puma_stats_clustered

  test_preserve_bundler_env.rb
     8) :17   TestPreserveBundlerEnv test_usr2_restart_preserves_bundler_environment
     9) :42   TestPreserveBundlerEnv test_phased_restart_preserves_unspecified_bundle_gemfile

  test_redirect_io.rb
    10) :57   TestRedirectIO test_sighup_redirects_io_cluster

  test_worker_gem_independence.rb
    11) :16   TestWorkerGemIndependence test_changing_nio4r_version_during_phased_restart
    12) :23   TestWorkerGemIndependence test_changing_json_version_during_phased_restart
    13) :30   TestWorkerGemIndependence test_changing_json_version_during_phased_restart_after_querying_stats_from_status_server
    14) :45   TestWorkerGemIndependence test_changing_json_version_during_phased_restart_after_querying_gc_stats_from_status_server
    15) :60   TestWorkerGemIndependence test_changing_json_version_during_phased_restart_after_querying_thread_backtraces_from_status_server
    16) :75   TestWorkerGemIndependence test_changing_json_version_during_phased_restart_after_accessing_puma_stats_directly

────────────────────────────────────── Signal INFO isn't available on the java platform ──
  test_integration_single.rb
    17) :113  TestIntegrationSingle test_siginfo_thread_print

─────────────────────────────────────────────────────────────────────── Skip unless MRI ──
  test_busy_worker.rb
    18) :65   TestBusyWorker test_multiple_requests_waiting_on_less_busy_worker
    19) :85   TestBusyWorker test_multiple_requests_processing_in_parallel

────────────────────────────────────────────────────────────────────── Skipped on JRuby ──
  test_binder.rb
    20) :165  TestBinder test_allows_both_unix_and_tcp
    21) :345  TestBinder test_socket_activation_unix

  test_config.rb
    22) :62   TestConfigFile test_ssl_bind
    23) :106  TestConfigFile test_ssl_bind_no_tlsv1_1
    24) :125  TestConfigFile test_ssl_bind_with_cipher_filter
    25) :144  TestConfigFile test_ssl_bind_with_verification_flags

  test_http11.rb
    26) :116  Http11ParserTest test_semicolon_in_path

  test_integration_pumactl.rb
    27) :25   TestIntegrationPumactl test_stop_tcp
    28) :116  TestIntegrationPumactl test_kill_unknown

  test_integration_single.rb
    29) :24   TestIntegrationSingle test_usr2_restart_restores_environment
    30) :37   TestIntegrationSingle test_term_exit_code
    31) :66   TestIntegrationSingle test_term_not_accepts_new_connections
    32) :95   TestIntegrationSingle test_int_refuse

  test_integration_systemd.rb
    33) :38   TestIntegrationSystemd test_systemd_integration
    34) :51   TestIntegrationSystemd test_systemd_watchdog

  test_redirect_io.rb
    35) :30   TestRedirectIO test_sighup_redirects_io_single

@nateberkopec nateberkopec merged commit 5e6e073 into puma:master Mar 15, 2021
@MSP-Greg MSP-Greg deleted the 00-skip-summary branch March 15, 2021 14:50
JuanitoFatas pushed a commit to JuanitoFatas/puma that referenced this pull request Sep 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants