Allow TestCase to assess if STDERR buffer is a real error when running in process isolation

[gypsy]

Hello,

I have been using PHPUnit and to be honest I really like it. I just had a question:

When running a test in process isolation:

===== SYSTEM UNDER TEST.php =====

<?php
class MyHello {
    public function sayHello() {
        system('echo hey >&2');
        return 'I feel good';
    }
}
?>

==== TEST RUNNER FILE.php ======

<?php
require_once 'PHPUnit/Framework.php';

/**
 * @runTestsInSeparateProcesses
 */
class MyHelloTest extends PHPUnit_Framework_TestCase {

    public function testSayHello() {
        require_once 'test.php';
        $myHello = new MyHello();
        $out = $myHello->sayHello();
        $this->assertTrue($out === 'I feel good');
    }
}
?>

====== OUTPUT =====

localhost 21:34:29 ~ $ phpunit runtest.php 
PHPUnit 3.4.12 by Sebastian Bergmann.

E

Time: 0 seconds, Memory: 4.50Mb

There was 1 error:

1) MyHelloTest::testSayHello
RuntimeException: hey


FAILURES!
Tests: 1, Assertions: 0, Errors: 1.

If there is anything written to the STDERR, the test is considered to be a failure. Is there any way to intersect line 179 in

excerpt from http://github.com/sebastianbergmann/phpunit/blob/3.5/PHPUnit/Util/PHP.php

173 * @param string $stdout
174 * @param string $stderr
175 * @since Method available since Release 3.5.0
176 */
177 protected static function processChildResult(PHPUnit_Framework_Test $test, PHPUnit_Framework_TestResult $result, $stdout, $stderr)
178    {
179        if (!empty($stderr)) {
180            $time = 0;
181            $result->addError(
182              $test,
183              new RuntimeException(trim($stderr)), $time
184            );
185        } else {

I couldn't find a way to do this in an object oriented way. Could you allow a hook on line 605 of TestCase.php

excerpt from http://github.com/sebastianbergmann/phpunit/blob/3.5/PHPUnit/Framework/TestCase.php

603       $this->prepareTemplate($template);
604
605            PHPUnit_Util_PHP::runJob($template->render(), $this, $result);
606        } else {
607            $result->run($this);

I don't want to copy paste the entire run() method in TestCase.php in my class that extends PHPUnit_Framework_TestCase and modify line 605

excerpt from
http://github.com/sebastianbergmann/phpunit/blob/3.5/PHPUnit/Framework/TestCase.php

488 public function run(PHPUnit_Framework_TestResult $result = NULL)

just to be able to judge if a test is actually a success and the output in STDERR buffer is just extra noise. I couldn't find a way as I said, because

a) PHPUnit_Util_PHP::runJob() is called within run()
b) after it's called, I have no way to intersect stderr returned by the result from test in separate process

Any help is really appreciated. Thank you.

[whatthejeff]

Since this is sort of a rare situation, I don't really think it warrants changes to the core libraries. You can actually handle this in a couple of different ways:

1. Trap STDERR in your application code. For me this makes the most sense. Then you can differentiate between actual errors and just noise in your application code and use your tests to ensure that your application is handling the errors correctly.
2. If you really must allow STDERR to bubble up through your tests, redirect it to STDOUT or to a file in your application code so that your tests can process it without triggering an error.
3. If neither of the above options work for you, as a last resort you can subclass PHPUnit_Framework_TestCase and call a version of PHPUnit_Util_PHP ::runJobs() that is customized for your exact specifications.

[sebastianbergmann]

Closing.