Failure output displays wrong "Step" when multiple tests fail #3900

mattmorgante opened this Issue Jan 9, 2017 · 1 comment


None yet

2 participants

mattmorgante commented Jan 9, 2017 edited

What are you trying to achieve?

I want to see on which step my tests are failing.

What do you get instead?

If multiple tests fail, I receive different error messages for each one BUT they each have the same "Step" within the error message.

Console Output:

Notice that in each "Step", it says the test is looking for "trike", but that keyword is only mentioned in one of the tests (the third one).

There were 3 failures:

  1. ListingCept: See the right information in the listing
    Test tests/acceptance/ListingCept.php
    Step See "trike","(descendant-or-self::[@class and contains(concat(' ', normalize-space(@class), ' '), ' occ_cartitle ')])[position()=1]"
    Fail Element located either by name, CSS or XPath element with '(descendant-or-self::
    [@class and contains(concat(' ', normalize-space(@class), ' '), ' occ_extrainfo ')])[position()=1]' was not found.

Scenario Steps:

  1. $I->see("chopper","(descendant-or-self::*[@class and contains(concat(' ', normalize-space(@class), ' '), ' occ_extrainfo '...") at tests/acceptance/ListingCept.php:9
  2. // I expect to see only choppers in the listing of this URL
  3. $I->wait(2) at tests/acceptance/ListingCept.php:7
  4. $I->click(".CookiesOK") at tests/acceptance/ListingCept.php:6
  5. $I->amOnPage("/search?crs=346") at tests/acceptance/ListingCept.php:5

  1. boujwaarCept: Perform actions and see result
    Test tests/acceptance/boujwaarCept.php
    Step See "trike","(descendant-or-self::[@class and contains(concat(' ', normalize-space(@class), ' '), ' occ_cartitle ')])[position()=1]"
    Fail Element located either by name, CSS or XPath element with '(descendant-or-self::
    [@class and contains(concat(' ', normalize-space(@class), ' '), ' occ_bouwjaar_kmstand ')])[position()=1]' was not found.

Scenario Steps:

  1. $I->see("Bouwjaar: 2010","(descendant-or-self::*[@class and contains(concat(' ', normalize-space(@class), ' '), ' occ_bouw...") at tests/acceptance/boujwaarCept.php:9
  2. $I->wait(2) at tests/acceptance/boujwaarCept.php:8
  3. $I->click(".CookiesOK") at tests/acceptance/boujwaarCept.php:7
  4. $I->wait(2) at tests/acceptance/boujwaarCept.php:6
  5. $I->amOnPage("/search?bmin=2010&bmax=2010") at tests/acceptance/boujwaarCept.php:5

  1. trefwoordCept: See motors based on a specific keyword
    Test tests/acceptance/trefwoordCept.php
    Step See "trike","(descendant-or-self::[@class and contains(concat(' ', normalize-space(@class), ' '), ' occ_cartitle ')])[position()=1]"
    Fail Failed asserting that any element by '(descendant-or-self::
    [@class and contains(concat(' ', normalize-space(@class), ' '), ' occ_cartitle ')])[position()=1]' on page /search?merk=0&model=0&crs=0&bmin=0&bmax=0&kmin=0&kmax=0&pmin=0&pmax=0&cmin=0&cmax=0&trefw=trike&pc=&strl=0

contains text 'trike'

Scenario Steps:

  1. $I->see("trike","(descendant-or-self::*[@class and contains(concat(' ', normalize-space(@class), ' '), ' occ_cartitle ')]...") at tests/acceptance/trefwoordCept.php:14
  2. $I->seeInCurrentUrl("trefw=trike") at tests/acceptance/trefwoordCept.php:13
  3. $I->wait(2) at tests/acceptance/trefwoordCept.php:12
  4. $I->click("#search_form_btn") at tests/acceptance/trefwoordCept.php:11
  5. $I->fillField("Trefwoord","trike") at tests/acceptance/trefwoordCept.php:10
  6. $I->wait(2) at tests/acceptance/trefwoordCept.php:9

Tests: 17, Assertions: 35, Failures: 3.

  • Codeception version: 2.2.7
  • PHP Version: 7.0.8
  • Operating System: Mac OS El Capitan 10.11.6
  • Installation type: Composer
  • List of installed packages (composer show)

$ composer show
You are running composer with xdebug enabled. This has a major impact on runtime performance. See
behat/gherkin v4.4.5 Gherkin DSL parser for PHP 5.3
classpreloader/classpreloader 3.0.0 Helps class loading performance by generating a single PHP file containing all of ...
codeception/codeception 2.2.7 BDD-style testing framework
codeception/robo-paracept 0.2.0 Codeception Parallel Execution Tasks via Robo Task Runner
consolidation/annotated-command 2.2.2 Initialize Symfony Console commands from annotated command class methods.
consolidation/log 1.0.3 Improved Psr-3 / Psr\Log logger based on Symfony Console components.
consolidation/output-formatters 3.1.5 Format text by applying transformations provided by plug-in formatters.
consolidation/robo 1.0.5 Modern task runner
container-interop/container-interop 1.1.0 Promoting the interoperability of container objects (DIC, SL, etc.)
dnoegel/php-xdg-base-dir 0.1 implementation of xdg base directory specification for php
doctrine/inflector v1.1.0 Common String Manipulations with regard to casing and singular/plural rules.
doctrine/instantiator 1.0.5 A small, lightweight utility to instantiate objects in PHP without invoking their ...
facebook/webdriver 1.2.0 A PHP client for WebDriver
fzaninotto/faker v1.6.0 Faker is a PHP library that generates fake data for you.
guzzlehttp/guzzle 6.2.2 Guzzle is a PHP HTTP client library
guzzlehttp/promises v1.3.1 Guzzle promises library
guzzlehttp/psr7 1.3.1 PSR-7 message implementation
hamcrest/hamcrest-php v1.2.2 This is the PHP port of Hamcrest Matchers
jakub-onderka/php-console-color 0.1
jakub-onderka/php-console-highlighter v0.3.2
jeremeamia/SuperClosure 2.2.0 Serialize Closure objects, including their context and binding
laravel/framework v5.3.18 The Laravel Framework.
laravel/homestead v4.0.2 A virtual machine for web artisans.
league/container 2.2.0 A fast and intuitive dependency injection container.
league/flysystem 1.0.28 Filesystem abstraction: Many filesystems, one API.
mockery/mockery 0.9.5 Mockery is a simple yet flexible PHP mock object framework for use in unit testing...
monolog/monolog 1.21.0 Sends your logs to files, sockets, inboxes, databases and various web services
mtdowling/cron-expression v1.1.0 CRON for PHP: Calculate the next or previous run date and determine if a CRON expr...
myclabs/deep-copy 1.5.4 Create deep copies (clones) of your objects
nesbot/carbon 1.21.0 A simple API extension for DateTime.
nikic/php-parser v2.1.1 A PHP parser written in PHP
paragonie/random_compat v2.0.2 PHP 5.x polyfill for random_bytes() and random_int() from PHP 7
phpdocumentor/reflection-common 1.0 Common reflection classes used by phpdocumentor to reflect the code structure
phpdocumentor/reflection-docblock 3.1.1 With this component, a library can provide support for annotations via DocBlocks o...
phpdocumentor/type-resolver 0.2.1
phpspec/prophecy v1.6.1 Highly opinionated mocking framework for PHP 5.3+
phpunit/php-code-coverage 4.0.4 Library that provides collection, processing, and rendering functionality for PHP ...
phpunit/php-file-iterator 1.4.2 FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-text-template 1.2.1 Simple template engine.
phpunit/php-timer 1.0.8 Utility class for timing
phpunit/php-token-stream 1.4.9 Wrapper around PHP's tokenizer extension.
phpunit/phpunit 5.6.1 The PHP Unit Testing framework.
phpunit/phpunit-mock-objects 3.4.0 Mock Object library for PHPUnit
phpunit/phpunit-selenium 3.0.2 Selenium Server integration for PHPUnit
psr/http-message 1.0.1 Common interface for HTTP messages
psr/log 1.0.2 Common interface for logging libraries
psy/psysh v0.7.2 An interactive shell for modern PHP.
ramsey/uuid 3.5.1 Formerly rhumsaa/uuid. A PHP 5.4+ library for generating RFC 4122 version 1, 3, 4,...
sebastian/code-unit-reverse-lookup 1.0.0 Looks up which function or method a line of code belongs to
sebastian/comparator 1.2.2 Provides the functionality to compare PHP values for equality
sebastian/diff 1.4.1 Diff implementation
sebastian/environment 2.0.0 Provides functionality to handle HHVM/PHP environments
sebastian/exporter 1.2.2 Provides the functionality to export PHP variables for visualization
sebastian/global-state 1.1.1 Snapshotting of global state
sebastian/object-enumerator 1.0.0 Traverses array structures and object graphs to enumerate all referenced objects
sebastian/recursion-context 1.0.2 Provides functionality to recursively process PHP variables
sebastian/resource-operations 1.0.0 Provides a list of PHP built-in functions that operate on resources
sebastian/version 2.0.1 Library that helps with managing the version number of Git-hosted PHP projects
swiftmailer/swiftmailer v5.4.3 Swiftmailer, free feature-rich PHP mailer
symfony/browser-kit v3.2.1 Symfony BrowserKit Component
symfony/console v3.1.8 Symfony Console Component
symfony/css-selector v3.1.5 Symfony CssSelector Component
symfony/debug v3.1.8 Symfony Debug Component
symfony/dom-crawler v3.1.5 Symfony DomCrawler Component
symfony/event-dispatcher v3.2.1 Symfony EventDispatcher Component
symfony/filesystem v3.2.1 Symfony Filesystem Component
symfony/finder v3.1.8 Symfony Finder Component
symfony/http-foundation v3.1.5 Symfony HttpFoundation Component
symfony/http-kernel v3.1.5 Symfony HttpKernel Component
symfony/polyfill-mbstring v1.3.0 Symfony polyfill for the Mbstring extension
symfony/polyfill-php56 v1.2.0 Symfony polyfill backporting some PHP 5.6+ features to lower PHP versions
symfony/polyfill-util v1.2.0 Symfony utilities for portability of PHP codes
symfony/process v3.1.8 Symfony Process Component
symfony/routing v3.1.5 Symfony Routing Component
symfony/translation v3.1.5 Symfony Translation Component
symfony/var-dumper v3.1.5 Symfony mechanism for exploring and dumping PHP variables
symfony/yaml v3.2.1 Symfony Yaml Component
victorjonsson/markdowndocs 1.3.7 Command line tool for generating markdown-formatted class documentation
vlucas/phpdotenv v2.4.0 Loads environment variables from .env to getenv(), $_ENV and $_SERVER auto...
webmozart/assert 1.2.0 Assertions to validate method input/output with nice error messages.

  • Suite configuration:

class_name: AcceptanceTester
- WebDriver:
- \Helper\Acceptance
url: 'http://localhost:8000'
port: 4445
browser: firefox
javascriptEnabled: true

gimler commented Jan 10, 2017

fixed by PR #3868

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment