Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

loadSessionSnapshot does not work with coverage enabled #2923

Closed
shlainn opened this issue Mar 25, 2016 · 9 comments
Closed

loadSessionSnapshot does not work with coverage enabled #2923

shlainn opened this issue Mar 25, 2016 · 9 comments
Labels

Comments

@shlainn
Copy link

@shlainn shlainn commented Mar 25, 2016

Hiya,

I have some working acceptance tests and am now trying to get code coverage for them. As recommended by the Guides, I use an @before directive to handle login before each test, and use saveSessionSnapshot to avoid having to go through login for each test.

This works nicely until I run codeception with --coverage. What happens then is that the first test in the file goes through nicely, but all subsequent tests, which rely on loadSessionSnapshot to skip login, do not work any more.

Is this behavior known?

@Naktibalda Naktibalda added the Coverage label Mar 25, 2016
@shlainn
Copy link
Author

@shlainn shlainn commented Mar 27, 2016

I did some more digging, maybe someone with more experience with codeception can make sense of this. I am running
Codeception PHP Testing Framework v2.1.5 Powered by PHPUnit 4.8.21 by Sebastian Bergmann and contributors.
I ran one of my Cests, once with the --coverage flag, and once without and compared the -vvv outputs. Without --coverage each subsequent test in the Cest continues where the previous test had left off - same page, same cookies, same everything.
With --coverage it resets to the base url for each test. This difference breaks those of my tests which assume that they are still on the same page - because without --coverage they still would be.

EDIT: Confirmed workaround for my issue is to explicitly set $I->amOnPage("whatever.php"); at the beginning of each test.

EDIT2: The culprit is in Codeception\Coverage\Subscriber\LocalServer (protected function startCoverageCollection) Line 151 which explicitly does $this->module->amOnPage('/');. Why?

EDIT3: Ah, because we can only set cookies for the current domain. But why does it not return to the previous page then?

EDIT4: After some messing around, I think that moving $this->module->amOnPage('/'); from startCoverageCollection to beforeSuite should do the trick

@DavertMik DavertMik added the BUG label Apr 15, 2016
@DavertMik DavertMik closed this in bc79819 Apr 19, 2016
@boboldehampsink
Copy link

@boboldehampsink boboldehampsink commented Feb 17, 2017

This is still not working. I am on Codeception 2.2.9. Commenting lines 161-162 in Codeception\Coverage\Subscriber\LocalServer makes it work again:

$this->module->amOnPage('/');
$this->module->setCookie(self::COVERAGE_COOKIE, $value);
@Naktibalda Naktibalda reopened this Feb 17, 2017
@boboldehampsink
Copy link

@boboldehampsink boboldehampsink commented Feb 17, 2017

The fix mentioned above fixes bc79819 but does not fix the problem.

@boboldehampsink
Copy link

@boboldehampsink boboldehampsink commented Feb 18, 2017

I have found this is because calling $this->module->amOnPage('/') before every suite resets the session on the server. We should somehow restore the session snapshot before calling this (perhaps by emitting a START_COVERAGE event?)

DavertMik added a commit that referenced this issue Mar 8, 2017
@lowerends
Copy link

@lowerends lowerends commented Apr 27, 2017

I can confirm this is still happening in version 2.2.10. This version contains #4020.

@boboldehampsink
Copy link

@boboldehampsink boboldehampsink commented Apr 27, 2017

#4020 is different fix

chris1312 pushed a commit to chris1312/Codeception that referenced this issue Jun 16, 2017
@iBotPeaches
Copy link
Contributor

@iBotPeaches iBotPeaches commented Jul 10, 2017

Thanks for the tips in this. I recently upgraded to 2.3.3 and implemented code coverage and saw my tests failing due to loadSessionSnapshot. I think it was my fault at bad test design, but enforcing all my tests to do $I->amOnPage('/whatever') before execution fixed this.

Prior to this, I was depending on the order of tests of where they ended for the next one to begin. (Yes, I know that sounds horrible, that has been resolved).

@DavertMik
Copy link
Member

@DavertMik DavertMik commented Jul 10, 2017

@iBotPeaches strange, coverage cookies are ignored during when the snapshot is saved. See
https://github.com/Codeception/Codeception/blob/2.3/src/Codeception/Module/WebDriver.php#L2919

Could you debug, is that so? If accidentally some coverage cookies are getting saved - that's the issue.

@iBotPeaches
Copy link
Contributor

@iBotPeaches iBotPeaches commented Jul 10, 2017

@DavertMik Maybe I don't understand this enough, but at my first request (ie login), I had no session so saveSessionSnapshot was called. It only looped the following cookies

  • _identity
  • _csrf
  • XDEBUG_SESSION
  • PHPSESSID

Of course none of those were the COVERAGE cookies array so it saved them all, and thus for remainder of the test suite it loaded that session point and never had a need to save again. If additional cookies were created throughout the test suite (possibly from coverage), they were never saved since saveSessionSnapshot is only called on first test to log in.

My lines didn't match up with the above link. I see 2.3.4 came out, so I might need to report back after I update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.