Generate JUnit XML Log will failed when Selenium server can’t be connected #3653

Closed
eXorus opened this Issue Oct 28, 2016 · 8 comments

Comments

Projects
None yet
4 participants
@eXorus
Contributor

eXorus commented Oct 28, 2016

Hello,

I think that with the different change about improving WebDriver in 2.2.6 we break something when we are exporting xml file.

Run test without exporting => OK it will print an error

./vendor/bin/codecept run tests/acceptance/minimumRequirementsCest.php --env ie8                                                               
Codeception PHP Testing Framework v2.2.6
Powered by PHPUnit 5.6.2 by Sebastian Bergmann and contributors.

Acceptance (ie8) Tests (1) ------------------------------------------------------------------------------------------------------------------------------------------
E minimumRequirementsCest: Test helium on internet explorer8 
---------------------------------------------------------------------------------------------------------------------------------------------------------------------


Time: 594 ms, Memory: 14.25MB

There was 1 error:

---------
1) minimumRequirementsCest: Test helium on internet explorer8
 Test  tests/acceptance/minimumRequirementsCest.php:testHeliumOnInternetExplorer8
JSON decoding of remote response failed.
Error code: 4
The response: 'Uh oh, you've run out of minutes!
Please visit https://saucelabs.com/pricing to purchase a subscription.'


ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

Run test with xml export => It will fail in JUnit.php

./vendor/bin/codecept run tests/acceptance/minimumRequirementsCest.php --env ie8 --xml result_acceptance_minrequirement.xml          
Codeception PHP Testing Framework v2.2.6
Powered by PHPUnit 5.6.2 by Sebastian Bergmann and contributors.

Acceptance (ie8) Tests (1) ------------------------------------------------------------------------------------------------------------------------------------------
E minimumRequirementsCest: Test helium on internet explorer8 
PHP Fatal error:  Call to a member function setAttribute() on null in /home/vdauce/Code/iwqa/vendor/phpunit/phpunit/src/Util/Log/JUnit.php on line 375



FATAL ERROR. TESTS NOT FINISHED.
Call to a member function setAttribute() on null 
in /vendor/phpunit/phpunit/src/Util/Log/JUnit.php:375

The issue is about $this->currentTestCase that is null at the end of the test.

It was working well in 2.2.5.

@DavertMik I'm don't know JUnit.php so I will not fix it.

@DavertMik DavertMik added the BUG label Oct 29, 2016

@DavertMik

This comment has been minimized.

Show comment
Hide comment
@DavertMik

DavertMik Oct 29, 2016

Member

Thanks for the report. Probably I can handle it.

Member

DavertMik commented Oct 29, 2016

Thanks for the report. Probably I can handle it.

@Naktibalda

This comment has been minimized.

Show comment
Hide comment
@Naktibalda

Naktibalda Nov 1, 2016

Member

I got the same error with PHPUnit 4.8.27

PHP Fatal error: Call to a member function setAttribute() on null in /home/.../vendor/phpunit/phpunit/src/Util/Log/JUnit.php on line 400

Member

Naktibalda commented Nov 1, 2016

I got the same error with PHPUnit 4.8.27

PHP Fatal error: Call to a member function setAttribute() on null in /home/.../vendor/phpunit/phpunit/src/Util/Log/JUnit.php on line 400

@Naktibalda

This comment has been minimized.

Show comment
Hide comment
@Naktibalda

Naktibalda Nov 1, 2016

Member

It is an odd issue - it happens, because Codeception\PHPUnit\Log\JUnit test listener is not yet registered when startTest method is called, but it becomes registered before endTest in https://github.com/Codeception/Codeception/blob/2.2.6/src/Codeception/Test/Test.php#L74-L112

Member

Naktibalda commented Nov 1, 2016

It is an odd issue - it happens, because Codeception\PHPUnit\Log\JUnit test listener is not yet registered when startTest method is called, but it becomes registered before endTest in https://github.com/Codeception/Codeception/blob/2.2.6/src/Codeception/Test/Test.php#L74-L112

@eXorus

This comment has been minimized.

Show comment
Hide comment
@eXorus

eXorus Nov 3, 2016

Contributor

I don't think it's an old issue @Naktibalda

[Codeception 2.2.5] codecept run acceptance --xml result.xml => result.xml generated

Codeception PHP Testing Framework v2.2.5
Powered by PHPUnit 5.6.2 by Sebastian Bergmann and contributors.


  [Codeception\Exception\ConnectionException]                                                                 
  Curl error thrown for http POST to /session with params: {"desiredCapabilities":{"browserName":"firefox"}}  
  Failed to connect to 127.0.0.1 port 4444: Connection refused                                                

  Please make sure that Selenium Server or PhantomJS is running.                                              


run [--report] [--html [HTML]] [--xml [XML]] [--tap [TAP]] [--json [JSON]] [--colors] [--no-colors] [--silent] [--steps] [-d|--debug] [--coverage [COVERAGE]] [--coverage-html [COVERAGE-HTML]] [--coverage-xml [COVERAGE-XML]] [--coverage-text [COVERAGE-TEXT]] [--no-exit] [-g|--group GROUP] [-s|--skip SKIP] [-x|--skip-group SKIP-GROUP] [--env ENV] [-f|--fail-fast] [--no-rebuild] [--] [<suite>] [<test>]

[Codeception 2.2.6] codecept run acceptance --xml result.xml => result.xml NOT generated

Codeception PHP Testing Framework v2.2.6
Powered by PHPUnit 5.6.2 by Sebastian Bergmann and contributors.

Acceptance Tests (1) -----------------------------------------------------------------------------------------------------------------------------------------
E WelcomeCest: Try to test 
PHP Fatal error:  Call to a member function setAttribute() on null in /////vendor/phpunit/phpunit/src/Util/Log/JUnit.php on line 375



FATAL ERROR. TESTS NOT FINISHED.
Call to a member function setAttribute() on null 
in /////vendor/phpunit/phpunit/src/Util/Log/JUnit.php:375

Contributor

eXorus commented Nov 3, 2016

I don't think it's an old issue @Naktibalda

[Codeception 2.2.5] codecept run acceptance --xml result.xml => result.xml generated

Codeception PHP Testing Framework v2.2.5
Powered by PHPUnit 5.6.2 by Sebastian Bergmann and contributors.


  [Codeception\Exception\ConnectionException]                                                                 
  Curl error thrown for http POST to /session with params: {"desiredCapabilities":{"browserName":"firefox"}}  
  Failed to connect to 127.0.0.1 port 4444: Connection refused                                                

  Please make sure that Selenium Server or PhantomJS is running.                                              


run [--report] [--html [HTML]] [--xml [XML]] [--tap [TAP]] [--json [JSON]] [--colors] [--no-colors] [--silent] [--steps] [-d|--debug] [--coverage [COVERAGE]] [--coverage-html [COVERAGE-HTML]] [--coverage-xml [COVERAGE-XML]] [--coverage-text [COVERAGE-TEXT]] [--no-exit] [-g|--group GROUP] [-s|--skip SKIP] [-x|--skip-group SKIP-GROUP] [--env ENV] [-f|--fail-fast] [--no-rebuild] [--] [<suite>] [<test>]

[Codeception 2.2.6] codecept run acceptance --xml result.xml => result.xml NOT generated

Codeception PHP Testing Framework v2.2.6
Powered by PHPUnit 5.6.2 by Sebastian Bergmann and contributors.

Acceptance Tests (1) -----------------------------------------------------------------------------------------------------------------------------------------
E WelcomeCest: Try to test 
PHP Fatal error:  Call to a member function setAttribute() on null in /////vendor/phpunit/phpunit/src/Util/Log/JUnit.php on line 375



FATAL ERROR. TESTS NOT FINISHED.
Call to a member function setAttribute() on null 
in /////vendor/phpunit/phpunit/src/Util/Log/JUnit.php:375

@Naktibalda

This comment has been minimized.

Show comment
Hide comment
@Naktibalda

Naktibalda Nov 3, 2016

Member

I wrote odd, not old.

Member

Naktibalda commented Nov 3, 2016

I wrote odd, not old.

@eXorus

This comment has been minimized.

Show comment
Hide comment
@eXorus

eXorus Nov 3, 2016

Contributor

Sorry :)

Contributor

eXorus commented Nov 3, 2016

Sorry :)

@marten-cz

This comment has been minimized.

Show comment
Hide comment
@marten-cz

marten-cz Feb 6, 2017

The problem is that phpunit/phpunit/src/Framework/TestResult.php::$listeners has Codeception\PHPUnit\Listener on first place. startTest method in this listener will throw exception when the selenium is not running. It will fail in method startTest in TestResult class.

It will be ok with this type of code

public function startTest(PHPUnit_Framework_Test $test)
{
    $this->lastTestFailed = false;
    $this->runTests      += count($test);

    foreach (array_reverse($this->listeners) as $listener) {
        $listener->startTest($test);
    }
}

One option is to change order of the listeners (temporary solution), or pull request was created for PHPUnit which should fix the order sensitivity - sebastianbergmann/phpunit#2476

The problem is that phpunit/phpunit/src/Framework/TestResult.php::$listeners has Codeception\PHPUnit\Listener on first place. startTest method in this listener will throw exception when the selenium is not running. It will fail in method startTest in TestResult class.

It will be ok with this type of code

public function startTest(PHPUnit_Framework_Test $test)
{
    $this->lastTestFailed = false;
    $this->runTests      += count($test);

    foreach (array_reverse($this->listeners) as $listener) {
        $listener->startTest($test);
    }
}

One option is to change order of the listeners (temporary solution), or pull request was created for PHPUnit which should fix the order sensitivity - sebastianbergmann/phpunit#2476

@Naktibalda Naktibalda self-assigned this Mar 7, 2017

Naktibalda added a commit to Naktibalda/Codeception that referenced this issue Mar 9, 2017

Naktibalda added a commit to Naktibalda/Codeception that referenced this issue Mar 9, 2017

@Naktibalda

This comment has been minimized.

Show comment
Hide comment
@Naktibalda

Naktibalda Mar 10, 2017

Member

@marten-cz thanks for explaining the root cause of this issue and suggestion how to fix it.

Member

Naktibalda commented Mar 10, 2017

@marten-cz thanks for explaining the root cause of this issue and suggestion how to fix it.

@DavertMik DavertMik closed this in #4055 Mar 13, 2017

DavertMik added a commit that referenced this issue Mar 13, 2017

Fixed issue #3653 (#4055)
* Reproduced issue #3653

* Execute Codeception\PHPUnit\Listener last to make sure that other listeners are executed

Fixes #3653

* Catch exception in listener

chris1312 added a commit to chris1312/Codeception that referenced this issue Jun 16, 2017

Fixed issue #3653 (#4055)
* Reproduced issue #3653

* Execute Codeception\PHPUnit\Listener last to make sure that other listeners are executed

Fixes #3653

* Catch exception in listener
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment