Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Random order seed should be printed in non-verbose mode as well #3515

Closed
Majkl578 opened this issue Feb 6, 2019 · 4 comments
Closed

Random order seed should be printed in non-verbose mode as well #3515

Majkl578 opened this issue Feb 6, 2019 · 4 comments

Comments

@Majkl578
Copy link

@Majkl578 Majkl578 commented Feb 6, 2019

Q A
PHPUnit version 8.0.1
PHP version irrelevant
Installation Method irrelevant

Right now, when tests are executed in random order with implicitly generated seed, the seed is only printed out in verbose mode.
Since verbose mode is way too verbose in some test suites (e.g. Doctrine's, as it prints skipped/incomplete tests too), I'd like to suggest always printing the order seed, regardless of whether verbose mode is enabled or not.

@epdenouden
Copy link
Collaborator

@epdenouden epdenouden commented Feb 6, 2019

@Majkl578 Thanks for your suggestion. I will have it implemented next few days

Loading

@sebastianbergmann
Copy link
Owner

@sebastianbergmann sebastianbergmann commented Feb 7, 2019

The "verbose output" setting is only used in very few places. I am wondering whether we should not just remove it and always print the information that is currently gated behind the "verbose output" setting.

Loading

@Majkl578
Copy link
Author

@Majkl578 Majkl578 commented Feb 7, 2019

FWIW the reason why we originally used verbose mode but then disabled it was that it was printing way too much output (see doctrine/dbal#3085). I.e. we had 649 skipped tests (because they only run when specific database driver/version is used and are skipped otherwise) and all of them were printed in the output.
I would prefer not doing this by default (by removing the -v opt-in) but rather with separate opt-in switch (--print-incomplete).

Loading

@epdenouden
Copy link
Collaborator

@epdenouden epdenouden commented Feb 7, 2019

@sebastianbergmann Thanks for putting that into words so clearly, this topic has been in the back of my mind a lot. While working on the TestDox e.g., I wouldn't have minded having a "recently added tests" header; do I want to print so many lines when just skipping because of @require etc.

@majkel89 I'll add the random seed to the default output for reproducibility of tests runs and think some more about verbosity. The Basic test suite I am working on is designed to trigger output events and will be a good sandbox for experimenting with this.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants