Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Buffered TestDox printer #3417


Copy link

@epdenouden epdenouden commented Nov 26, 2018

This addition provides a working solution to #3380. For Nikolaus you wanted:

The TestDox result printer(s) should be changed to buffer test results in order to ensure the right order and then emit all output after the last test was run.

Refactoring the TextDox printer turned out be a bit of work. To clean up the internal TestResult magic I ended up rewriting large parts. Still some minor cleaning up to do, but it's good enough for daily use.


  • TestSuiteSorter now calculates and stores the original execution order before a run
  • naming scheme logic for test reordering consolidated in TestSuiteSorter::getTestSorterUID
  • main runner passes this original order into a CliTestDoxPrinter extension when required
  • prints unbuffered when original execution order is not provided
  • when buffering the printer will flush test results in the original order as soon as it can
  • new PHPT end-to-end test keeps an eye on functionality
  • unit test fixes to match test naming scheme

How to test

Edit phpunit.xml and switch to verbose=false. Then run the test suite in both buffered and unbuffered TestDox mode:

./phpunit --testdox | tee /tmp/new.unbuffered
./phpunit --testdox --order-by=defects,depends | tee /tmp/new.buffered
diff /tmp/new.unbuffered /tmp/new.buffered


  • output diff: when buffering the names of some tests have a bit of extra detail
  • could clean up unbuffered message formatting and writing; rather do that after the test Hooks are done

@epdenouden epdenouden force-pushed the issue-3380-v75-buffered-testdox-printer branch from 9a65fa2 to da99a17 Compare November 26, 2018 11:42
Copy link

codecov bot commented Nov 26, 2018

Codecov Report

Merging #3417 into 7.5 will decrease coverage by 0.3%.
The diff coverage is 97.12%.

Impacted file tree graph

@@             Coverage Diff              @@
##                7.5    #3417      +/-   ##
- Coverage     83.49%   83.18%   -0.31%     
- Complexity     3575     3620      +45     
  Files           143      143              
  Lines          9513     9637     +124     
+ Hits           7943     8017      +74     
- Misses         1570     1620      +50
Impacted Files Coverage Δ Complexity Δ
src/Util/TestResultCache.php 91.22% <ø> (ø) 29 <0> (ø) ⬇️
src/TextUI/TestRunner.php 68.9% <100%> (+0.14%) 288 <0> (+2) ⬆️
src/Runner/TestSuiteSorter.php 99.16% <96.87%> (+0.13%) 54 <18> (+7) ⬆️
src/Util/TestDox/CliTestDoxPrinter.php 95.62% <97.12%> (+8.28%) 63 <37> (+36) ⬆️
src/Util/TestDox/TestResult.php 0% <0%> (-94.55%) 23% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b1ccace...ca44ddf. Read the comment docs.

@epdenouden epdenouden force-pushed the issue-3380-v75-buffered-testdox-printer branch from da99a17 to ca44ddf Compare November 26, 2018 11:50
Copy link
Contributor Author

@sebastianbergmann Working on the TestDox has given me good insight in what is required for improving the TestHook interface as mentioned in #3390

@sebastianbergmann sebastianbergmann merged commit 8682a3f into sebastianbergmann:7.5 Nov 26, 2018
@epdenouden epdenouden deleted the issue-3380-v75-buffered-testdox-printer branch November 26, 2018 14:17
@epdenouden epdenouden mentioned this pull request Dec 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet

Successfully merging this pull request may close these issues.

None yet

2 participants