chore(tests): do not run unit tests in parallel

[jerolimov]

Turbo supports caching for tasks, including build(ing), lint(ing), and even test(ing).

While investigating why #1480 failed multiple times, I noticed that different tests fail our GitHub Actions and on my local machine. But when I tried to run the tests for a single workspace/plugin these tests pass...

I could reproduce this when disable the cache. The currency is 10 by default and many different workspaces fail:

yarn turbo test --force # default is --concurrency=10

...

@janus-idp/backstage-plugin-rbac#test: command (/home/christoph/git/janus-idp/backstage-plugins/plugins/rbac) yarn run test exited (1)

 Tasks:    12 successful, 47 total
Cached:    0 cached, 47 total
  Time:    1m3.393s 
Failed:    @janus-idp/backstage-plugin-rbac#test

 ERROR  run failed: command  exited (1)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

yarn turbo test --force --concurrency=47 # my PC is lagging with this option ;)

...

@janus-idp/backstage-plugin-web-terminal:test: ERROR: command finished with error: command (/home/christoph/git/janus-idp/backstage-plugins/plugins/web-terminal) yarn run test exited (1)
@janus-idp/backstage-plugin-web-terminal#test: command (/home/christoph/git/janus-idp/backstage-plugins/plugins/web-terminal) yarn run test exited (1)

 Tasks:    25 successful, 47 total
Cached:    0 cached, 47 total
  Time:    2m57.077s 
Failed:    @janus-idp/backstage-plugin-web-terminal#test

 ERROR  run failed: command  exited (1)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

But when running all workspaces in a sequence, the test runs successfully:

yarn turbo test --force --concurrency=1

...

 Tasks:    47 successful, 47 total
Cached:    0 cached, 47 total
  Time:    4m2.405s 

Done in 242.85s.

Of course, it would be nice to investigate whether we can run all or more tests in parallel. But for now, I want to ensure that the tests run sequentially / successfully.

But will this increase the test time? Yes, but only if all tests run, for example, when the package.json is changed. Tests of untouched workspaces will (still) not run! In my machine running all tests takes ~ 4 min.

I also added the yarn.lock to the globalDependencies to ensure that we rerun builds and tests if any dependency changes!

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.6% Duplication on New Code

See analysis details on SonarCloud

[jerolimov]

/cc @kadel @schultzp2020 @invincibleJai

[schultzp2020]

My only concern is how long the testing will take. I have a feeling that we can fix most of these issues by cleaning up the tests.

[jerolimov]

My only concern is how long the testing will take. I think we can fix most of these issues by cleaning up the tests.

@schultzp2020 Yep, I had similar thoughts. But I personally think it's more important to reduce the flakyness. 🤷‍♂️

I also added more information and logs to the initial comment. Running all unit tests in a sequence takes around 4 min on my machine. I can not compare it with the parallel run because it never run successfully. But I expect a something between 1-3 minutes. 😏

And the overall GHA with yarn install, etc. takes ~12 minutes (before it fails).

It will (still) run all these tests only if the package.json, yarn.lock, etc. is touched.

When only a plugin/workspace is touched, it will still run only the tests of that workspace.

[kadel]

even if this makes test runs longer I think it is worth it. Less testing failures due to flaky test will at the end save a lot more time

[kadel]

/approve
/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kadel

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [kadel]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment