phpunit runs one or two tests and shut down with no message

[garabiriel]

I'm using laravel 5.1 and homestead 2 and having the following issue:
If I run each test individually the test passes or fail with a error message as expected:

$ phpunit tests/API/CustomerTest.php --filter GetCustomer
PHPUnit 5.6.1 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 2.06 seconds, Memory: 29.50MB

OK (1 test, 15 assertions)

but if I run a test suite with more then one test, phpunit exits suddenly with no message:

$ phpunit tests/API/CustomerTest.php --debug
PHPUnit 5.6.1 by Sebastian Bergmann and contributors.


Starting test 'Tests\API\CustomerTest::testGetCustomer'.
.
Starting test 'Tests\API\CustomerTest::testCustomerGetOrder'.
$

It does not matter which test, if I comment out the one phpunit stopped at it stops in the next one:

 $ phpunit tests/API/CustomerTest.php --debug
PHPUnit 5.6.1 by Sebastian Bergmann and contributors.


Starting test 'Tests\API\CustomerTest::testGetCustomer'.
.
Starting test 'Tests\API\CustomerTest::testCustomerSubcriptions'.
$

If I comment out the whole file, it goes to the next one and dies in the same way:
$ phpunit --debug

PHPUnit 5.6.1 by Sebastian Bergmann and contributors.


Starting test 'Tests\API\CartTest::testShoppingCart'.
.
Starting test 'Warning'.
W
Starting test 'Tests\API\OrderTest::testOrder'.
$

[vbashkirtsev]

Does not look as PHPUnit problem to me. Rather an issue with test isolation. Perhaps first thing to try would be to use @runTestsInSeparateProcesses . Or you may use strace to see when exactly your testing bails out: it may give you a clue why it does not work as expected.

[garabiriel]

@runTestsInSeparateProcesses in the CustomerTest.php file solves the exiting problem but creates a problem with the Laravel loaded classes:

$ phpunit --debug
PHPUnit 4.8.27 by Sebastian Bergmann and contributors.


Starting test 'Tests\API\CartTest::testShoppingCart'.
.
Starting test 'Tests\API\CustomerTest::testGetCustomer'.
E
Starting test 'Tests\API\CustomerTest::testCustomerGetOrder'.
E
Starting test 'Tests\API\CustomerTest::testCustomerSubcriptions'.
E
Starting test 'Tests\API\CustomerTest::testCustomerAddresses'.
E
Starting test 'Tests\API\CustomerTest::testCustomerPaymentProfiles'.
E
Starting test 'Tests\API\OrderTest::testOrder'.
.

Time: 11.01 seconds, Memory: 40.00MB

There were 5 errors:


1) Tests\API\CustomerTest::testGetCustomer
PHPUnit_Framework_Exception: PHP Fatal error:  Class 'Route' not found in /home/vagrant/Code/fg-rest-api/app/Http/routes.php on line 15
PHP Stack trace:
PHP   1. {main}() -:0
PHP   2. require_once() -:649

Fatal error: Class 'Route' not found in /home/vagrant/Code/fg-rest-api/app/Http/routes.php on line 15

Call Stack:
    0.0013     614616   1. {main}() -:0
    0.7052   12198728   2. require_once('/home/vagrant/Code/fg-rest-api/app/Http/routes.php') -:649
(... All the errors are the same ...)

But if I run only CustomerTest.php the tests pass:

$ phpunit tests/API/CustomerTest.php --debug
PHPUnit 4.8.27 by Sebastian Bergmann and contributors.


Starting test 'Tests\API\CustomerTest::testGetCustomer'.
.
Starting test 'Tests\API\CustomerTest::testCustomerGetOrder'.
.
Starting test 'Tests\API\CustomerTest::testCustomerSubcriptions'.
.
Starting test 'Tests\API\CustomerTest::testCustomerAddresses'.
.
Starting test 'Tests\API\CustomerTest::testCustomerPaymentProfiles'.
.

Time: 11.98 seconds, Memory: 8.75MB

OK (5 tests, 109 assertions)

[vbashkirtsev]

Still looks like test isolation problem. Perhaps you should try to add @runTestsInSeparateProcesses in the rest of your tests. Also it appears that the error with 'Route' class is due to missing autoloader: when you run tests in separate processes your responsibility for setting up the environment is a bit higher.

In the end IMO there no problem with PHPUnit. Just a problem with your particular tests. Please mind that PHPUnit runs as normal PHP process and then passes control to your tests. If your tests fail in some way then PHPUnit tries its best to keep up but it is fairly easy to break execution if there something wrong - in this case unexpected termination is not uncommon. It is recommended to run tests in separate processes in this case (albeit it is slower) as only child processes will die unexpectedly while main PHPUnit process will remain. But as downside you will need to take care of your setup routines: there nothing will be available from previous tests to any of the following tests and generally it is an issue: tests should not expect anything left from previous test.

[sandervanhooft]

@garabiriel Where you able to fix this problem? I'm having similar issues with Laravel 5.3.

[andreshg112]

Same problem here.

[andreshg112]

If I change processIsolation from "false to "true", It works, but it is slower.

[andreshg112]

It doesn't really work. Now I got this error. If I change it back to processIsolation=false, it works but just for one test.

[andreshg112]

Solved! The problem happened to me because I renamed some classes but forgot to change the name of the files. So, there were "duplicated classes" in cached files. I realized about it when I executed: php artisan config:cache.

[Pointotech]

I managed to fix this by adding this tag inside of the <phpunit> tag in phpunit.xml:

<php>
  <ini name="error_reporting" value="E_ALL" />
</php>