Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase assert_output timeout for FullStackConsoleTest and ApplicationTests::ServerTest #51148

Merged
merged 1 commit into from Feb 21, 2024

Conversation

yahonda
Copy link
Member

@yahonda yahonda commented Feb 20, 2024

Motivation / Background

This pull request addresses the following failures at Rails Nightly CI with assertion enabled Ruby. that has been built with cppflags="-DENABLE_PATH_CHECK=0 -DRUBY_DEBUG=1" optflags="-O3 -fno-inline"

https://buildkite.com/rails/rails-nightly/builds/191#018dc3d1-9032-4baa-ae8a-3c630889ab5f/1342-1348 https://buildkite.com/rails/rails-nightly/builds/191#018dc3d1-9033-4c0c-b61c-00f26d3a63fb/1198-1204

Related to #51140

Detail

  • Failures fixed by this commit:
$ ruby -v
ruby 3.4.0dev (2024-02-20T11:52:09Z master c22cb960cf) [x86_64-linux]
$ bin/test test/application/console_test.rb -n test_sandbox
Run options: -n test_sandbox --seed 3666

F

Failure:
FullStackConsoleTest#test_sandbox [test/console_helpers.rb:19]:
"=> 0" expected, but got:

app-template(dev)> quapp-template(dev)> quiapp-template(dev)> quit.
Expected "\r\napp-template(dev)> quapp-template(dev)> quiapp-template(dev)> quit" to include "=> 0".

bin/test test/application/console_test.rb:142

Finished in 32.180314s, 0.0311 runs/s, 0.7458 assertions/s.
1 runs, 24 assertions, 1 failures, 0 errors, 0 skips
$
$ bin/test test/application/server_test.rb
Run options: --seed 64559

F

Failure:
ApplicationTests::ServerTest#test_restart_rails_server_with_custom_pid_file_path [test/console_helpers.rb:19]:
"Listening" expected, but got:

.
Expected "" to include "Listening".

bin/test test/application/server_test.rb:19

F

Failure:
ApplicationTests::ServerTest#test_run_+server+_blocks_after_the_server_starts [test/console_helpers.rb:19]:
"Hello world" expected, but got:

.
Expected "" to include "Hello world".

bin/test test/application/server_test.rb:43

Finished in 2.117413s, 0.9445 runs/s, 1.8891 assertions/s.
2 runs, 4 assertions, 2 failures, 0 errors, 0 skips
$

Additional information

Note: These failures depend on assert_timeout and test environment. If these failures do not reproduce locally, setting short timeout = 1 should reproduce them.

$ git diff
diff --git a/railties/test/console_helpers.rb b/railties/test/console_helpers.rb
index 10d3a54602..32d333ed99 100644
--- a/railties/test/console_helpers.rb
+++ b/railties/test/console_helpers.rb
@@ -6,7 +6,7 @@
 end

 module ConsoleHelpers
-  def assert_output(expected, io, timeout = 10)
+  def assert_output(expected, io, timeout = 1)
     timeout = Time.now + timeout

     output = +""
$

Checklist

Before submitting the PR make sure the following are checked:

  • This Pull Request is related to one change. Changes that are unrelated should be opened in separate PRs.
  • Commit message has a detailed description of what changed and why. If this PR fixes a related issue include it in the commit message. Ex: [Fix #issue-number]
  • Tests are added or updated if you fix a bug or add a feature.
  • CHANGELOG files are updated for the changed libraries if there is a behavior change or additional feature. Minor bug fixes and documentation changes should not be included.

…ationTests::ServerTest`

This commit addresses the following failures at Rails Nightly CI with assertion enabled Ruby.
that has been built with `cppflags="-DENABLE_PATH_CHECK=0 -DRUBY_DEBUG=1" optflags="-O3 -fno-inline"`

https://buildkite.com/rails/rails-nightly/builds/191#018dc3d1-9032-4baa-ae8a-3c630889ab5f/1342-1348
https://buildkite.com/rails/rails-nightly/builds/191#018dc3d1-9033-4c0c-b61c-00f26d3a63fb/1198-1204

Related to rails#51140

- Failures fixed by this commit:

```
$ ruby -v
ruby 3.4.0dev (2024-02-20T11:52:09Z master c22cb960cf) [x86_64-linux]
$ bin/test test/application/console_test.rb -n test_sandbox
Run options: -n test_sandbox --seed 3666

F

Failure:
FullStackConsoleTest#test_sandbox [test/console_helpers.rb:19]:
"=> 0" expected, but got:

app-template(dev)> quapp-template(dev)> quiapp-template(dev)> quit.
Expected "\r\napp-template(dev)> quapp-template(dev)> quiapp-template(dev)> quit" to include "=> 0".

bin/test test/application/console_test.rb:142

Finished in 32.180314s, 0.0311 runs/s, 0.7458 assertions/s.
1 runs, 24 assertions, 1 failures, 0 errors, 0 skips
$
```

```
$ bin/test test/application/server_test.rb
Run options: --seed 64559

F

Failure:
ApplicationTests::ServerTest#test_restart_rails_server_with_custom_pid_file_path [test/console_helpers.rb:19]:
"Listening" expected, but got:

.
Expected "" to include "Listening".

bin/test test/application/server_test.rb:19

F

Failure:
ApplicationTests::ServerTest#test_run_+server+_blocks_after_the_server_starts [test/console_helpers.rb:19]:
"Hello world" expected, but got:

.
Expected "" to include "Hello world".

bin/test test/application/server_test.rb:43

Finished in 2.117413s, 0.9445 runs/s, 1.8891 assertions/s.
2 runs, 4 assertions, 2 failures, 0 errors, 0 skips
$
```

Note: These failures depend on assert_timeout and test environment. If these failures do not reproduce locally, setting short timeout = 1 should reproduce them.
```
$ git diff
diff --git a/railties/test/console_helpers.rb b/railties/test/console_helpers.rb
index 10d3a54602..32d333ed99 100644
--- a/railties/test/console_helpers.rb
+++ b/railties/test/console_helpers.rb
@@ -6,7 +6,7 @@
 end

 module ConsoleHelpers
-  def assert_output(expected, io, timeout = 10)
+  def assert_output(expected, io, timeout = 1)
     timeout = Time.now + timeout

     output = +""
$
```
@rails-bot rails-bot bot added the railties label Feb 20, 2024
@yahonda yahonda merged commit 3528b9d into rails:main Feb 21, 2024
4 checks passed
@yahonda yahonda deleted the increase_timeout_for_assert_output_2 branch April 19, 2024 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant