Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cypress massively slows down when the test output is toggled open #8012

Closed
nphillipsworth opened this issue Jul 17, 2020 · 9 comments · Fixed by #8522
Closed

Cypress massively slows down when the test output is toggled open #8012

nphillipsworth opened this issue Jul 17, 2020 · 9 comments · Fixed by #8522
Labels

Comments

@nphillipsworth
Copy link

Current behavior:

For the clearest demonstration of what I mean see this video:

We have some fairly long running tests (1 minute long test cases)

When the test is running if the test log (left panel) is toggled open for the current test, the test speed will slow down to a crawl. The slow down is proportional to how long the given test case has been running, at the start the slow down is barely noticeable, by the end it slows to the point of barely moving.

This can be shown manually, by clicking the toggle, as in the video. Proving that this is directly related to the output being toggled and not a symptom of the test slowing down because of some other issue ( eating too much memory ).

However this does also happen when the test are run in pipeline causing some very bizarre behaviour.
There is some logic that defaults the toggle to being open on the first test case of a describe block but not the second. Meaning we can massively improve the speed of our test by introducing a dummy test case before the real one. This happens despite the tests being run in headless mode.

Desired behavior:

As far as I can see the speed of the test should never depend on the output being toggled.
If for some reason it has to, then there should be a setting that allows us to force the output to stay closed when run through our pipeline.

Test code to reproduce

Simple test case that loops round 200 times filling in an input box.

https://github.com/nphillipsworth/cypress-test-tiny/

Steps to reproduce

  1. this takes about a minute to run with the test output toggle closed.
  2. delete the first empty test case, it now will run so slow it won't complete.
    empty tests shouldn't effect run speed!

Versions

"cypress": "^4.10.0", mac, both chrome and electron

@billiegoose
Copy link

I'm a developer at stoplight.io (@stoplightio) and am seeing similar behavior:

  • In Cypress 4.8, tests run fine.
  • In Cypress 4.9 and 4.10 (with test output toggled open) we're seeing:
    • Cypress consuming as much as 80% of the CPU time and
    • operations that normally take <200ms are now taking up to 20 seconds (such as an HTTP request served from the Chrome disk cache) and
    • high memory usage, up to 750MB of memory being allocated in the span of a few seconds.

Here's some screenshots from our situation at Stoplight:

4.8.0
image

4.10.0
image

4.10.0 if I manually collapse the test output toggles in the sidebar:
image

Here's a chrome profile showing it is indeed Cypress code that is bottlenecking the CPU, with a 180ms synchronous task that's run every 200ms, leaving only 20ms for our own code:
image

(I can provide the raw profile data upon request by email (will.hilton@stoplight.io) if you'd like to open it in Chrome's debugger and zoom in and inspect the stack traces better.)

4.10.0 memory usage spike:
image

Hopefully this is enough for you to go on. If not feel free to contact me at my Stoplight email address (I don't check my GitHub notifications anymore because there's simply way too many of them). I spent about 6 hours investigating the issue yesterday before finally "solving" it by downgrading from 4.10 to 4.8 but would love to see a proper fix implemented so we can continue to upgrade to newer versions.

@cypress-bot cypress-bot bot added the stage: needs investigating Someone from Cypress needs to look at this label Jul 20, 2020
@jennifer-shehane jennifer-shehane added type: performance 🏃‍♀️ Performance related and removed stage: needs investigating Someone from Cypress needs to look at this labels Jul 20, 2020
@jvanoostveen
Copy link

Cypress 5.0 also has this issue.
I would love to see a fix for this, maybe a workaround by allowing to have the left panel with the steps collapsed or something. This issue is doubling the time for our tests. 😢

@srdjanprole
Copy link

ok, this started to be a huge problem.

@jvanoostveen
Copy link

@bahmutov As this issue has been around for some time, this has to be surfaced somewhere? I really want to make use of some of the fixes and new additions in Cypress, but this issue makes us not able to upgrade to a more recent version of Cypress.

@jvanoostveen
Copy link

@jennifer-shehane ?

@jvanoostveen
Copy link

Ah I see there is another issue, that's more alive at the moment: #8297

@cypress-bot cypress-bot bot added stage: work in progress stage: needs review The PR code is done & tested, needs review and removed stage: work in progress stage: needs review The PR code is done & tested, needs review labels Sep 8, 2020
@cypress-bot
Copy link
Contributor

cypress-bot bot commented Sep 8, 2020

The code for this is done in cypress-io/cypress#8522, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

@flotwig
Copy link
Contributor

flotwig commented Sep 8, 2020

@nphillipsworth thanks for the repro, from this I was able to identify the issue and fix it in #8522. The fix will come out with the next release of Cypress!

@cypress-bot
Copy link
Contributor

cypress-bot bot commented Sep 15, 2020

Released in 5.2.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v5.2.0, please open a new issue.

@cypress-bot cypress-bot bot locked as resolved and limited conversation to collaborators Sep 15, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants