[7.0.0] Performance regression from 6.8 (over 3x slower in our test suite)

[GC-Mark]

I have split this off from the performance regression issue related to 6.6 - 6.7 #15779

Can also confirm our tests are taking way longer in v7.0.0 than the were in v6.8.0

Also, because the tests are taking longer, we get random failing tests due to timeout issues.

Im not sure what info i can give you to help you debug this, but if there is anything you need me to do, just ask.

Our last few CI runs

v6.8.0

v7.0.0

(including a random failing test due to a timeout)

[rbayliss]

Yeah, I can confirm this as well - our tests are much slower under 7.0. In our case, it's closer to 50% slower, although we have much longer tests overall. This is only happening in CI (Github Actions) for us, local runs cypress run have no measurable difference. Rolling back to 6.9.1 fixes the slowness for us.

[csvan]

Probably related: #15827

[lassesteffen]

We also noticed a heavy increase in run time in our test suite (13mins to 28 mins = 2x slower) when upgrading from 6.8.0 to 7.0.0. Very interested in a workaround / fix for this.

[lugus]

I confirm, rollback to 6.9.

[pkyeck]

I can also confirm the increase in duration when switching from 5.6.0 to 7.0.0 on CircleCI:

before:

✔  All specs passed!                        10:10       73       73        -        -        -  

after:

✔  All specs passed!                        16:59       73       73        -        -        -  

[lukeapage]

Am trying to narrow down the commit in 7.0 that fails.. but it take a long time for me as our process is quite slow. Will continue tomorrow.

6f9e80a - slow - bad

232bd87 - ok - good

[j1000]

We are observing the same thing. On my workstation it's as fast as ever, but on our CI machine (with fewer resources) it's taking 3x as long. Maybe it's using more memory?

[JayBizzle]

CI was definitely the slowest for us, but we also saw the slow down locally too, just not as bad

[lukeapage]

I narrowed it down above #15853 (comment) (editing the comment)
but I don't see anything in the bad commit thats suspicious. I'm also in a big doubt because the cypress commit history is a absolute total nightmare to navigate - I'm used to rebasing or squash commits so you see everything flat but not only does cypress not do that, but they have a develop and a master branch and then merge between them constantly 😱

[estefafdez]

Same here, almost 1h extra running the same tests on the 6.9.1 and on the 7.0 and 7.0.1.

[lukeapage]

I can't see any obvious regression in the times it takes to run the cypress tests themselves - but I don't think that is uncommon. Cypress uses itself for testing, but all of their tests are fast because they test on tiny pages with very little js and additional resources. Where as real-world cypress usage is on pages that are resource heavy.

My own app is a very heavy SPA with 3mb of javascript and we get a 7x speed regression on low spec CI machines and no regression on fast multi-core developer machines.