chore: add worker build on CI

[SimenB]

Summary

From #7610. @rickhanlonii could you take a look? 🙂

Test plan

Will fail (for now)

[SimenB]

worker_threads is unflagged in node 11.7.0, so the extra job isn't necessary. When (if) we land #7681 I'll update this PR to remove all the disableWorkerThreads: true and see where we're at

[SimenB]

Circle has node 11.8 now, which has the umask fix. Rebased, hopefully a better result this time 🙂

[thymikee]

We test threads but we're actually running our test suite serially. Maybe it's worth to run tests by default on 2 or 4 workers and run in band e.g. on travis or somewhere?

[SimenB]

I wanna see that tests pass without being stupidly slow first. If they do, then we can test by actually spinning up workers 🙂

These have to be fixed first:

[SimenB]

haha, it just died 😛 But looking at, and fixing, multiProjectRunner might be what we're missing. If anybody wants to pick it up

[SimenB]

Test failures on node 12, that's odd...

[SimenB]

whatever that old issue was, it seems to have been fixed 👍

Will need to look into why coverage fails, but this is super promising

[SimenB]

Very interesting - tests fail on circus, but passes on GH Actions. The failing test on CI passes on my local machine. Very odd...

[rickhanlonii]

😢

[SimenB]

heh, soz. At the time it was useful to get test timings and such, now that we're down to actual test failures it should be good to move back 😉

[sibelius]

for those that want to test this, check this patch-package https://gist.github.com/sibelius/c6ef3f0890814dd4163a9ada0cd739f1

[SimenB]

@sibelius very interested in hearing numbers if you play with - does it impact memory/cpu usage?

[sibelius]

I've tried to use worker_threads because child process was failing for us, here is the issue #10144

When using worker_threads we got some

CircleCI received exit code 137 

137 means Out of Memory.

So worker threads is kind of worse than child process right now for us

mongodb-memory-server was causing OOM for us because it was spinning 1 mongod per test file, we fixed this here shelfio/jest-mongodb#209

but worker_threads still leaking on CircleCI

but it does not leak when running local, --logHeapUsage show always less than 300mb, so not sure if leakage is because of node 12.18.0 or because of docker or circleci specific

[sibelius]

we fixed most issues of our tests

and now worker_threads is working fine for us (node 12.18.0)

we didn't notice any improvement in cpu time or memory usage

[jeysal]

https://www.npmjs.com/package/piscina just leaving this here as something to maybe explore in the future, who knows, maybe it's faster than jest-workers worker threads farm impl