Cypress component testing is very slow

[MohdRafiUddin188]

Hello,

I started using Knapsack cypress pro for parallel executing component testing. I observed that the total duration is almost double due to continuous webpack compilation and project build before running the spec files.

Basically, when I specified 2 nodes for cypress component testing, each node invoked knapsack pro queue API to get the test spec for running and the API always returns only 1 spec file for executing the test spec instead it should return a list of test spec files which then get's build and executed without redundant webpack build step.

[MohdRafiUddin188]

Basically, the second request to POST /v1/queues/queue should return list of splitted test_files (Based on No. of parallel nodes running) instead of returning one test_files at a time for cypress component tetsing.

[ArturT]

Hi @MohdRafiUddin188

Please record your first CI build (it could be slower, as you noticed). You should see it in the knapsack pro dashboard. CI build should have recorded tests' execution time for all parallel nodes.

Only then, when you create a new git commit and run a fresh CI build, we could leverage the recorded tests' execution time data. We can group tests so that the Queue Mode distributes tests in batches among parallel nodes instead one by one.

Basically, the very first CI build run can be slow. The second should be faster.

[ArturT]

BTW you can contact support https://knapsackpro.com/contact and I can take a look at your recorded CI builds to verify if the first CI build is correct.

[MohdRafiUddin188]

Hey @ArturT

Thanks for suggesting this We can group tests so that the Queue Mode distributes tests in batches among parallel nodes instead one by one. For me, Sending test files in batches for execution is a much-needed thing without which I couldn't use this package.

Let me know if there is a configuration already defined for it and needs to pass OR by default it will handle it.

I was using 4 parallel nodes for 42 test files (Cypress component testing) and was expecting each node to take 25% of the total test files load once instead of one at a time which is causing the slowness.

[ArturT]

By default Queue API returns grouped tests in batches if the test file has a known execution time. You need at least 1 recorded CI build to have a known execution time for each test file.

Please note each CI node can consume a few batches in Queue Mode.

Here you can learn the difference between Queue Mode vs Regular Mode.
https://docs.knapsackpro.com/2020/how-to-speed-up-ruby-and-javascript-tests-with-ci-parallelisation

@knapsack-pro/cypress supports only Queue Mode. BTW Regular Mode is available in the knapsack_pro gem for Ruby tests.

If you would like to understand deeper how Queue API works, you can see this article:
https://docs.knapsackpro.com/2021/how-to-build-knapsack-pro-api-client-from-scratch-in-any-programming-language

Can you share a link to your CI build in the Knapsack Pro dashboard? Unfortunately, I don't know which user account yours is. I could check if you have at least one recorded CI build.

[MohdRafiUddin188]

Hey @ArturT

Here is the link to my CI build dashboard https://knapsackpro.com/dashboard/organizations/3398/projects/1997/test_suites/2904/builds

[ArturT]

I see you have a few CI builds recorded correctly. How does it work for you know?

Is there a big overhead due to continuous webpack compilation and project build? If so, then a Regular Mode would probably work better than Queue Mode for you. I'm not sure when Regular Mode will be available for the JavaScript tests. I can get back to you if we release it.

[MohdRafiUddin188]

The CI builds records work very well.

Yes, there is a significant overhead of webpack compilation and project build because of which I can't use the knapsack-cypress in my project.

[ArturT]

I don't have a quick solution for your use case. The overhead of webpack compilation is especially visible for a project with a short test suite execution time of ~10 minutes. If you would have a test suite taking 1+ hours, then tests would be grouped in larger batches.

[ArturT]

story

https://trello.com/c/KjXa29IJ/53-collect-ideas-to-improve-queue-api#comment-6488a06cbfeacae40b0a84d9

This issue is related to a high-level plan of improving Queue Mode API to behave sometimes more like Regular Mode (a static split). I added the link to Trello for our internal reference.