Right now, when tests are executed in random order with implicitly generated seed, the seed is only printed out in verbose mode.
Since verbose mode is way too verbose in some test suites (e.g. Doctrine's, as it prints skipped/incomplete tests too), I'd like to suggest always printing the order seed, regardless of whether verbose mode is enabled or not.
The text was updated successfully, but these errors were encountered:
The "verbose output" setting is only used in very few places. I am wondering whether we should not just remove it and always print the information that is currently gated behind the "verbose output" setting.
FWIW the reason why we originally used verbose mode but then disabled it was that it was printing way too much output (see doctrine/dbal#3085). I.e. we had 649 skipped tests (because they only run when specific database driver/version is used and are skipped otherwise) and all of them were printed in the output.
I would prefer not doing this by default (by removing the -v opt-in) but rather with separate opt-in switch (--print-incomplete).
@sebastianbergmann Thanks for putting that into words so clearly, this topic has been in the back of my mind a lot. While working on the TestDox e.g., I wouldn't have minded having a "recently added tests" header; do I want to print so many lines when just skipping because of @require etc.
@majkel89 I'll add the random seed to the default output for reproducibility of tests runs and think some more about verbosity. The Basic test suite I am working on is designed to trigger output events and will be a good sandbox for experimenting with this.