New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incorrect steps shown when having multiple canXXX conditional assertion failures #3225

Closed
Tenzian opened this Issue Jun 15, 2016 · 3 comments

Comments

Projects
None yet
3 participants
@Tenzian
Contributor

Tenzian commented Jun 15, 2016

When using the conditional assertion canXXX and having multiple failures there's a problem where the console results only show the initial step where a fail occurred.

There's also an issue with the html report: Each failed step is correctly highlighted but the report only shows the details for the final fail.

Example:

Creating the following acceptance test FailsCept.php:

<?php
$I = new AcceptanceTester($scenario);

$I->wantTo('generate some fails');

$I->amOnPage('/');

$I->canSeeElement('//nothere');
$I->canSeeElement('//nothereeither');
$I->canSeeElement('//doesntexist');

In the 'There were 3 failures:' section, each resulting fail after the first misreports the step that caused the fail:

codecept run acceptance FailsCept --html --steps --no-colors

Codeception PHP Testing Framework v2.2.1
Powered by PHPUnit 5.4.4 by Sebastian Bergmann and contributors.

Acceptance Tests (1) -------------------------------------------------------------------------------------------------------------
FailsCept: Generate some fails
Signature: FailsCept
Test: tests/acceptance/FailsCept.php
Scenario --
 I am on page "/"
 I can see element "//nothere"
 FAIL 

 I can see element "//nothereeither"
 FAIL 

 I can see element "//doesntexist"
 FAIL 

 PASSED 

----------------------------------------------------------------------------------------------------------------------------------


Time: 706 ms, Memory: 13.00MB

There were 3 failures:

---------
1) FailsCept: Generate some fails
 Test  tests/acceptance/FailsCept.php
 Step  Can see element "//nothere"
 Fail  Element located either by name, CSS or XPath element with '//nothere' was not found.

Scenario Steps:

 4. $I->canSeeElement("//doesntexist")
 3. $I->canSeeElement("//nothereeither")
 2. $I->canSeeElement("//nothere")
 1. $I->amOnPage("/")


---------
2) FailsCept: Generate some fails
 Test  tests/acceptance/FailsCept.php
 Step  Can see element "//nothere"
 Fail  Element located either by name, CSS or XPath element with '//nothereeither' was not found.

Scenario Steps:

 4. $I->canSeeElement("//doesntexist")
 3. $I->canSeeElement("//nothereeither")
 2. $I->canSeeElement("//nothere")
 1. $I->amOnPage("/")


---------
3) FailsCept: Generate some fails
 Test  tests/acceptance/FailsCept.php
 Step  Can see element "//nothere"
 Fail  Element located either by name, CSS or XPath element with '//doesntexist' was not found.

Scenario Steps:

 4. $I->canSeeElement("//doesntexist")
 3. $I->canSeeElement("//nothereeither")
 2. $I->canSeeElement("//nothere")
 1. $I->amOnPage("/")


FAILURES!
Tests: 1, Assertions: 3, Failures: 3.
- HTML report generated in file:///tests/_output/report.html

@Tenzian Tenzian changed the title from having multiple canXXX conditional assertion failures show incorrect steps to incorrect steps shown when having multiple canXXX conditional assertion failures Jun 15, 2016

@DavertMik DavertMik added the BUG label Jun 17, 2016

@DavertMik

This comment has been minimized.

Show comment
Hide comment
@DavertMik

DavertMik Jun 17, 2016

Member

Thanks, I will take a look

Member

DavertMik commented Jun 17, 2016

Thanks, I will take a look

@Naktibalda

This comment has been minimized.

Show comment
Hide comment
@Naktibalda

Naktibalda Sep 25, 2016

Member

@DavertMik you promised to take a look.

Member

Naktibalda commented Sep 25, 2016

@DavertMik you promised to take a look.

@Mitrichius Mitrichius referenced this issue Nov 5, 2016

Merged

Fix #3225 #3686

@DavertMik

This comment has been minimized.

Show comment
Hide comment
@DavertMik
Member

DavertMik commented Nov 7, 2016

@DavertMik DavertMik closed this in #3686 Nov 7, 2016

DavertMik added a commit that referenced this issue Nov 7, 2016

Fix #3225 (#3686)
* #3325 fix html part

* #3325 fix console part

chris1312 added a commit to chris1312/Codeception that referenced this issue Jun 16, 2017

Fix #3225 (#3686)
* #3325 fix html part

* #3325 fix console part
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment