Performance issues in v3.0.1

[paulpruteanu]

Current behavior:

Given each test runs (cypress run) in its own sandbox, it needs around 1.5 seconds to transpile/ prepare the code. This adds about 4-5 minutes to my overall set of 160 tests.

Eg. under v2.1.0 it takes 7 minutes 30 seconds, whereas in v3.0.1 the same tests run in about 11 minutes.

Also, when running with --headed, the window closes and opens, which at least on MacBook causes an unexpected user experience, popping a new window while I'm using another application.

Desired behavior:

To improve performance, not diminish it.

Versions

v3.0.1 vs v2.1.0, MacBook Sierra 10.12.6, Electron/ Chrome

[jennifer-shehane]

Also, when running with --headed, the window closes and opens, which at least on MacBook causes an unexpected user experience, popping a new window while I'm using another application.

We now close and reopen the test runner in between each spec file and this is expected behavior.

it needs around 1.5 seconds to transpile/ prepare the code

• Could you elaborate more on how to determined the 1.5 secs was spent transpiling/preparing code?
• Do you have your own setup for preprocessing cypress tests that differs from the default that you could share?
• Any logs while running DEBUG=cypress*?

[brian-mann]

We are actually internally aware of some performance regressions caused by 3.0 - that could be what you're experiencing.

You're saying it takes about 1.5 secs to transpile the spec code, but that code is transpiled per spec file, not per test. So suggesting you have 160 tests shouldn't matter - it should only be affected by how many spec files you have. Which by the way, how many do you have?

Also - the total transpilation time shouldn't actually be affected by the change to 3.0. Yes in 2.1.0 it would run all the specs together, but it would still have to transpile all of the specs. Now that it's split up, it still has to transpile them, but it does it only once between each spec is run. So the total aggregate of spec transpilation should not be affected since it did just as much work (albeit all up front) in 2.1.0.

[brian-mann]

Forgot to ask... why are you running all of the specs locally? Shouldn't you just run them in CI? Even at 7 1/2 minutes, its kind of crazy to try to do locally, especially with headed mode on. That task is better dedicated for an isolated worker.

[jennifer-shehane]

Unfortunately we have to close this issue as there is not enough information to reproduce the problem.

Please comment in this issue with a reproducible example and we will reopen the issue. 🙏