Show stderr in test output

[MartinMystikJonas]

Is there reason why tester shows only content of stdout of test execution after test failure?

I just spent about hour trying to figure out why my tests suddenly failing with no information at all, because causing exception was written to stderr.

It should be sufficient to append "2>&1" to Job command at Job.php:77

Or is there reason for not doing so?

[milo]

It is a question how stderr should be handled. It should be cross-platform solution and `2>&1` is not. And one problem to resolv. Normally, stdout interlaces with stderr. If it should be kept in `Job::getOutput()`, how to implement it? ```php $mem = fopen('php://memory', 'r+'); $descriptors = array( array('pipe', 'r'), $mem, $mem, ); ``` seems nice, but `php://memory` cannot be used with `proc_open()`. The `php://temp` works well but it always creates temporary file. Solution can be catch stderr and add `Job::getErrorOutput()` and add `TestHandler::assertErrorOutput()` plus other things. Other ideas?

[JanTvrdik]

It should be cross-platform solution

On what platform it does not work?

[milo]

No way! It works for Windows too! 😃 Well, I'm shut up. I don't see a reason why not to do that in this way.

[MartinMystikJonas]

Sent pull request #89

[milo]

We were discussing it with @dg and we agreed that stderr should be captured by Job separately. I tried to implement it, but there is really a lot of proc_open()'s bugs espacially on Win.

The main problem is with stderr as pipe and large output. Bugs #65650, #51800, #64438, #60120. For some I found workaround, but for some not on PHP 5.3.3. The bug behavior is that the process get stuck when reading even single byte from stdout or stderr.

One solution is use php://temp for stderr, but it is not the way since temp file is opened and discarted with every Job. But it works.

Next solution is use it and just pray for stderr shorter 2048 bytes ;-)

The safest way is close the stderr immediately after proc_open() as it is done now.

The next is use proposed 2>&1, mix the stdout with stderr and somehow parse valid HTTP headers.

I have no more ideas...

[milo]

Btw. @MartinMystikJonas When is the stderr hidden? If I use file_put_contents('php://stderr', 'test'); in tests now, I get E_WARNING: file_put_contents(): Only 0 of 4 bytes written, possibly out of free disk space because stderr is closed.

[MartinMystikJonas]

@milo Did you manage to solve this issus on Windows or did you give up?

I have one idea how to solve this. It's not strictly clean but if we need workaround bug I think it is acceptable.

My idea is to output stderr to custom temp file (similarly to lock file used in Environment::lock) and then just read this file.

What do you think? Could this solution work?

[milo]

@MartinMystikJonas I didn't solve this, din't give up yet.

My idea is to output stderr to custom temp file

That's I wrote above with the php://temp. Problem is, this temporary file is created by every job, even stderr is not used.

Maybe solution is just mix stdin and stdout together as proposed. Who cares about wrong HTTP headers parsing. If is header needed by test, it just fails.

[MartinMystikJonas]

Well what about use just 2> stderr-uniqid? File will be created only if there is some output

[milo]

@MartinMystikJonas Not so on Linux. But interesting idea...

[MartinMystikJonas]

@milo My bad. I did not try it :-)