Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using environment does not work with code coverage #2634

Closed
drodil opened this issue Dec 15, 2015 · 4 comments
Closed

Using environment does not work with code coverage #2634

drodil opened this issue Dec 15, 2015 · 4 comments
Labels

Comments

@drodil
Copy link

@drodil drodil commented Dec 15, 2015

Hi,

I am trying to run codecoverage for my project with codeception. Here is my build/codeception.yml file:

actor: Tester
paths:
    tests: ../tests
    log: ../tests/_output
    data: ../tests/_data
    helpers: ../tests/_support
settings:
    bootstrap: _bootstrap.php
    colors: false
    memory_limit: 2048M
coverage:
    enabled: true
    remote: false
    remote_context_options:
        http:
            timeout: 60
        ssl:
            verify_peer: false
    blacklist:
        include:
            - ../db/*
            - ../data/*
            - ../config/*
modules:
    config:
        Db:
            dsn: ''
            user: ''
            password: ''
            dump: tests/_data/dump.sql

I have defined the following environments in tests/acceptance.yml:

class_name: AcceptanceTester
modules:
    enabled: [WebDriver, AcceptanceHelper]
    config:
        WebDriver:
            url: 'http://localhost/project/'
            browser: firefox
env:
    ci:
        modules:
            config:
                WebDriver:
                    url: 'https://server1.example.com/project/'
                    host: server1.example.com
                    browser: firefox
                    window_size: 1920x1080
    ci-mbv:
        modules:
            config:
                WebDriver:
                    url: 'https://server1.example.com/project-mbv/'
                    host: server1.example.com
                    browser: firefox
                    window_size: 1920x1080

When I run the following command:

php codecept.phar run -f -vvv -c build/codeception.yml --xml build/logs/junit.xml --html build/logs/junit.html --coverage-html build/coverage/html --coverage-xml build/coverage/converage.xml --env ci-mbv

I will get the error:

The "/var/lib/jenkins/workspace/project/build/../tests/acceptance.ci-mbv/" directory does not exist.  

The tests are created in subfolders under tests folder (acceptance/, unit/ and api/). Why does the codeception add the environment to end of the test folder? How can I prevent that or am I doing something wrong? This does not work even with remote option set as true - the error in that case is "Suite acceptance-ci-mbv was not loaded".

Here is the codeception exception backtrace:

Exception trace:
 () at /var/lib/jenkins/workspace/project/vendor/symfony/finder/Finder.php:656
 Symfony\Component\Finder\Finder->in() at /var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/Lib/TestLoader.php:112
 Codeception\Lib\TestLoader->loadTests() at /var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/SuiteManager.php:95
 Codeception\SuiteManager->loadTests() at /var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/Codecept.php:199
 Codeception\Codecept->runSuite() at /var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/Codecept.php:191
 Codeception\Codecept->run() at /var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/Command/Run.php:262
 Codeception\Command\Run->runSuites() at /var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/Command/Run.php:189
 Codeception\Command\Run->execute() at /var/lib/jenkins/workspace/project/vendor/symfony/console/Command/Command.php:259
 Symfony\Component\Console\Command\Command->run() at /var/lib/jenkins/workspace/project/vendor/symfony/console/Application.php:841
 Symfony\Component\Console\Application->doRunCommand() at /var/lib/jenkins/workspace/project/vendor/symfony/console/Application.php:192
 Symfony\Component\Console\Application->doRun() at /var/lib/jenkins/workspace/project/vendor/symfony/console/Application.php:123
 Symfony\Component\Console\Application->run() at /var/lib/jenkins/workspace/project/vendor/codeception/codeception/codecept:28

Thanks for help!

@drodil
Copy link
Author

@drodil drodil commented Dec 15, 2015

Seems like there is some magic handling of the environment name in SuiteManager.php:136:

$suite->setBaseName($this->env ? substr($name, 0, strpos($name, '-' . $this->env)) : $name);

After changing the environment name to be "cimbv" instead "ci-mbv" something at least happened in the running of the test but still got error:

  [RemoteException] Remote Application Error:
Suite++was+not+loaded  

Exception trace:

/var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/Coverage/Subscriber/LocalServer.php:98
/var/lib/jenkins/workspace/project/vendor/symfony/event-dispatcher/EventDispatcher.php:181
/var/lib/jenkins/workspace/project/vendor/symfony/event-dispatcher/EventDispatcher.php:46
/var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/TestCase/Shared/Actor.php:73
/var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/TestCase/Shared/Actor.php:106
/var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/Scenario.php:101
/var/lib/jenkins/workspace/project/tests/_support/_generated/AcceptanceTesterActions.php:268
/var/lib/jenkins/workspace/project/tests/acceptance/BaseAcceptanceTest.php:78
/var/lib/jenkins/workspace/project/tests/acceptance/BaseAcceptanceTest.php:100
/var/lib/jenkins/workspace/project/tests/acceptance/AdminCest.php:32
/var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/Lib/Di.php:112
/var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/TestCase/Cest.php:127
/var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/TestCase/Cest.php:211
/var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/TestCase/Cest.php:64
/var/lib/jenkins/workspace/project/vendor/phpunit/phpunit/src/Framework/TestCase.php:909
/var/lib/jenkins/workspace/project/vendor/phpunit/phpunit/src/Framework/TestCase.php:768
/var/lib/jenkins/workspace/project/vendor/phpunit/phpunit/src/Framework/TestResult.php:612
/var/lib/jenkins/workspace/project/vendor/phpunit/phpunit/src/Framework/TestCase.php:724
/var/lib/jenkins/workspace/project/vendor/phpunit/phpunit/src/Framework/TestSuite.php:747
/var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/PHPUnit/Runner.php:89
/var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/SuiteManager.php:153
/var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/Codecept.php:203
/var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/Codecept.php:194
/var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/Command/Run.php:262
/var/lib/jenkins/workspace/project/vendor/codeception/codeception/src/Codeception/Command/Run.php:189
/var/lib/jenkins/workspace/project/vendor/symfony/console/Command/Command.php:259
/var/lib/jenkins/workspace/project/vendor/symfony/console/Application.php:841
/var/lib/jenkins/workspace/project/vendor/symfony/console/Application.php:192
/var/lib/jenkins/workspace/project/vendor/symfony/console/Application.php:123
/var/lib/jenkins/workspace/project/vendor/codeception/codeception/codecept:28
@drodil
Copy link
Author

@drodil drodil commented Dec 15, 2015

Seems like the acceptance cookie does not have CodeCoverage_Suite set correctly:

[Cookies] [{"name":"PHP_SESSID","path":"/","value":"6121sul17iolebiqn4ve4354aka3cvf1nn3u6f57g2g70u50rmekgp9d5c084uegi1cu1c0khcn24l55gr3r9j10ihj6e8f5rm0p8r3","secure":true,"domain":"server1.example.com","class":"org.openqa.selenium.Cookie","httpOnly":false,"hCode":1755402541},{"name":"CODECEPTION_CODECOVERAGE","path":"/","value":"{\"CodeCoverage\":\"adminModeTest\",\"CodeCoverage_Suite\":\"\",\"CodeCoverage_Config\":null}","secure":false,"domain":"server1.example.com","class":"org.openqa.selenium.Cookie","httpOnly":false,"hCode":960895263}]

Any idea why this would occur? If I set code coverage disabled, the acceptance tests run ok.

@Naktibalda Naktibalda added the Coverage label Jan 14, 2016
@neronmoon
Copy link
Contributor

@neronmoon neronmoon commented Jan 21, 2016

+1

neronmoon referenced this issue Jan 21, 2016
@DavertMik
Copy link
Member

@DavertMik DavertMik commented Jan 21, 2016

Ok, thanks for the investigation. I will take a detailed look

@DavertMik DavertMik added the BUG label Jan 21, 2016
@DavertMik DavertMik closed this in 25c23b2 Jan 21, 2016
DavertMik added a commit that referenced this issue Jan 21, 2016
DavertMik added a commit that referenced this issue Jan 21, 2016
DavertMik added a commit that referenced this issue Jan 25, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.