Skip to content

Comments

Move printing test seed to beginning of test run#10399

Closed
jrusso1020 wants to merge 1 commit intoelixir-lang:masterfrom
jrusso1020:move-seed-to-beginning
Closed

Move printing test seed to beginning of test run#10399
jrusso1020 wants to merge 1 commit intoelixir-lang:masterfrom
jrusso1020:move-seed-to-beginning

Conversation

@jrusso1020
Copy link
Contributor

@jrusso1020 jrusso1020 commented Oct 6, 2020

Came across this proposal on the forums to move printing the seed of the
test suite to the beginning of the test run. This was actually
something I wished for recently when encountering a weird issue
in our codebase which caused all of our non async tests to fail and
each synchronous test would take 60 seconds before timing out. With
a few hundred synchronous tests this took quite some time for the
suite to finish and I wanted to grab the seed to replicate it.

https://groups.google.com/g/elixir-lang-core/c/sP8DiY1zMFQ

Example

==> ex_unit (ex_unit)

Randomized with seed 411332
................................................................................................................................................................................................................................................................................................................................................................................

Came across this proposal on the forums to move the seed of the
test suite to the beginning of the test run. This was actually
something I wished for recently when encountering a weird issue
in our codebase which caused all of our non async tests to fail and
each synchronous test would take 60 seconds before timing out. With
a few hundred synchronous tests this took quite some time for the
suite to finish and I wanted to grab the seed to replicate it.

https://groups.google.com/g/elixir-lang-core/c/sP8DiY1zMFQ
@jrusso1020 jrusso1020 force-pushed the move-seed-to-beginning branch from d7f9386 to 007f758 Compare October 6, 2020 05:15
@smaximov
Copy link
Contributor

smaximov commented Oct 6, 2020

I think it would be better if ex_unit printed the seed both at the beginning and at the end of the test suite. For instance, if you have a bunch of failing tests, you wouldn't want to scroll all the way back to where tests started just to find the seed (assuming the test run has already finished).

==> ex_unit (ex_unit)

Randomized with seed 411332

................................................................................................................................................................................................................................................................................................................................................................................
# [INSERT SEVERAL SCREENS OF FAILING TESTS HERE]
..................................................................................................................................................................

Finished in 100.5 seconds
3 doctests, 129 tests, 15 failures

Randomized with seed 411332

For example, Ruby's RSpec does exactly that.

@josevalim
Copy link
Member

Thanks @jrusso1020 for reminding me of this issue. I have gone with a different approach in 285a6a6. If your test suite gets stuck, ExUnit now intercepts sigquit and it will show which tests have been aborted as well as a summary of the run alongside the seed. :) In any case, thanks for the PRs! ❤️

@josevalim josevalim closed this Oct 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants