Catch afterTestMethod() output; fixes #820

[cedric-anne]

I added this piece of code in `atoum\atoum\tests\units\annotations\extractor``test class:

    public function beforeTestMethod($method)
    {
        echo "start\n";
    }

    public function afterTestMethod($method)
    {
        echo "end\n";
    }

It produces the following result:

> Total tests duration: 12.81 seconds.
> Total tests memory usage: 500.94 Mb.
> Running duration: 20.59 seconds.
Success (230 tests, 1825/1827 methods, 0 void method, 2 skipped methods, 28614 assertions)!
> There are 16 outputs:
=> In atoum\atoum\tests\units\annotations\extractor::testResetHandlers():
start
=> In atoum\atoum\tests\units\annotations\extractor::testResetHandlers():
end
=> In atoum\atoum\tests\units\annotations\extractor::testUnsetHandler():
start
=> In atoum\atoum\tests\units\annotations\extractor::testUnsetHandler():
end
=> In atoum\atoum\tests\units\annotations\extractor::test__construct():
start
=> In atoum\atoum\tests\units\annotations\extractor::test__construct():
end
=> In atoum\atoum\tests\units\annotations\extractor::testExtract():
start
=> In atoum\atoum\tests\units\annotations\extractor::testExtract():
end
=> In atoum\atoum\tests\units\annotations\extractor::testSpace():
start
=> In atoum\atoum\tests\units\annotations\extractor::testSpace():
end
=> In atoum\atoum\tests\units\annotations\extractor::testStar():
start
=> In atoum\atoum\tests\units\annotations\extractor::testStar():
end
=> In atoum\atoum\tests\units\annotations\extractor::testToBoolean():
start
=> In atoum\atoum\tests\units\annotations\extractor::testToBoolean():
end
=> In atoum\atoum\tests\units\annotations\extractor::testToArray():
start
=> In atoum\atoum\tests\units\annotations\extractor::testToArray():
end

So it works as expected, but I have no idea how I can test this.

[cedric-anne]

So it works as expected, but I have no idea how I can test this.

I added a functionnal test. Running it without this patch will result in a test failure.

Without patch:

$ bin/atoum -d tests/functionals
> atoum path: /app/bin/atoum
> atoum version: dev-master
> PHP path: /usr/local/bin/php
> PHP version:
=> PHP 8.1.0RC3 (cli) (built: Sep 30 2021 22:28:27) (NTS)
=> Copyright (c) The PHP Group
=> Zend Engine v4.1.0RC3, Copyright (c) Zend Technologies
> atoum\atoum\tests\functionals\asserters\generator...
[..__________________________________________________________][0/2]
[S.__________________________________________________________][1/2]
[SS__________________________________________________________][2/2]
=> Test duration: 0.05 second.
=> Memory usage: 0.50 Mb.
> atoum\atoum\tests\functionals\mock\controller...
[..__________________________________________________________][0/2]
[S.__________________________________________________________][1/2]
[SS__________________________________________________________][2/2]
=> Test duration: 0.01 second.
=> Memory usage: 0.57 Mb.
> atoum\atoum\tests\functionals\test...
[.___________________________________________________________][0/1]
[U___________________________________________________________][1/1]
=> Test duration: 0.00 second.
=> Memory usage: 0.00 Mb.
> atoum\atoum\tests\functionals\test\annotations\ignore...
[.___________________________________________________________][0/1]
[S___________________________________________________________][1/1]
=> Test duration: 0.00 second.
=> Memory usage: 0.07 Mb.
> atoum\atoum\tests\functionals\test\data\providers\mock...
[.___________________________________________________________][0/1]
[S___________________________________________________________][1/1]
=> Test duration: 0.00 second.
=> Memory usage: 0.31 Mb.
> Total tests duration: 0.06 second.
> Total tests memory usage: 1.46 Mb.
> Running duration: 0.21 second.
Failure (5 tests, 6/7 methods, 0 void method, 0 skipped method, 1 uncompleted method, 0 failure, 0 error, 0 exception)!
> There is 1 uncompleted method:
=> atoum\atoum\tests\functionals\test::testSuccess() with exit code 0:
==> output(1187) "end
==> O:22:"atoum\atoum\test\score":15:{s:13:"*passNumber";i:0;s:17:"*failAssertions";a:0:{}s:13:"*exceptions";a:0:{}s:20:"*runtimeExceptions";a:0:{}s:9:"*errors";a:0:{}s:10:"*outputs";a:1:{i:0;a:3:{s:5:"class";s:34:"atoum\atoum\tests\functionals\test";s:6:"method";s:11:"testSuccess";s:5:"value";s:6:"start
==> ";}}s:12:"*durations";a:1:{i:0;a:4:{s:5:"class";s:34:"atoum\atoum\tests\functionals\test";s:6:"method";s:11:"testSuccess";s:5:"value";d:5.9604644775390625E-6;s:4:"path";s:44:"/app/tests/functionals/classes/extractor.php";}}s:15:"*memoryUsages";a:0:{}s:14:"*voidMethods";a:1:{i:0;a:3:{s:4:"file";s:44:"/app/tests/functionals/classes/extractor.php";s:5:"class";s:34:"atoum\atoum\tests\functionals\test";s:6:"method";s:11:"testSuccess";}}s:21:"*uncompletedMethods";a:0:{}s:17:"*skippedMethods";a:0:{}s:11:"*coverage";O:26:"atoum\atoum\score\coverage":7:{s:7:"classes";a:0:{}s:7:"methods";a:0:{}s:5:"paths";a:0:{}s:8:"branches";a:0:{}s:15:"excludedClasses";a:0:{}s:18:"excludedNamespaces";a:0:{}s:19:"excludedDirectories";a:0:{}}s:28:"atoum\atoum\test\scorecase";N;s:34:"atoum\atoum\test\scoredataSetKey";N;s:39:"atoum\atoum\test\scoredataSetProvider";N;}"

With patch:

$ bin/atoum -d tests/functionals
> atoum path: /app/bin/atoum
> atoum version: dev-master
> PHP path: /usr/local/bin/php
> PHP version:
=> PHP 8.1.0RC3 (cli) (built: Sep 30 2021 22:28:27) (NTS)
=> Copyright (c) The PHP Group
=> Zend Engine v4.1.0RC3, Copyright (c) Zend Technologies
> atoum\atoum\tests\functionals\asserters\generator...
[..__________________________________________________________][0/2]
[S.__________________________________________________________][1/2]
[SS__________________________________________________________][2/2]
=> Test duration: 0.05 second.
=> Memory usage: 0.50 Mb.
> atoum\atoum\tests\functionals\mock\controller...
[..__________________________________________________________][0/2]
[S.__________________________________________________________][1/2]
[SS__________________________________________________________][2/2]
=> Test duration: 0.01 second.
=> Memory usage: 0.57 Mb.
> atoum\atoum\tests\functionals\test...
[.___________________________________________________________][0/1]
[S___________________________________________________________][1/1]
=> Test duration: 0.00 second.
=> Memory usage: 0.05 Mb.
> atoum\atoum\tests\functionals\test\annotations\ignore...
[.___________________________________________________________][0/1]
[S___________________________________________________________][1/1]
=> Test duration: 0.00 second.
=> Memory usage: 0.07 Mb.
> atoum\atoum\tests\functionals\test\data\providers\mock...
[.___________________________________________________________][0/1]
[S___________________________________________________________][1/1]
=> Test duration: 0.00 second.
=> Memory usage: 0.31 Mb.
> Total tests duration: 0.06 second.
> Total tests memory usage: 1.51 Mb.
> Running duration: 0.21 second.
Success (5 tests, 7/7 methods, 0 void method, 0 skipped method, 53 assertions)!
> There are 2 outputs:
=> In atoum\atoum\tests\functionals\test::testOutputFromBeforeAndAfterTestMethod():
start
=> In atoum\atoum\tests\functionals\test::testOutputFromBeforeAndAfterTestMethod():
end

[Grummfy]

Seems nice
did you try on all the test engine?
to be sure it's working properly?

[cedric-anne]

Seems nice did you try on all the test engine? to be sure it's working properly?

The functionnal test suite works properly and the atoum test suite is still working, so I guess it does not break everything. I will run the Glpi project test suite using this patch, to ensure everything is ok, and give a feedback soon.

[cedric-anne]

The full GLPI project tests suite produces expected results with this patch, so I guess it has no side-effects.

[cedric-anne]

Seems nice
did you try on all the test engine?
to be sure it's working properly?

Not sure you were talking about that, but I added output catching on setUp and tearDown methods.