Enable higher parallelism in tests

[wknapik]

Tests run with --test-launcher-jobs=4 by default, including in CI. In Windows/ASAN we had to go down to 2 for unit tests, otherwise they wouldn't pass.

Relevant slack threads:

• Browser tests https://bravesoftware.slack.com/archives/CA5FPHWLF/p1619032313232000
• Unit tests in Win/ASAN https://bravesoftware.slack.com/archives/CA5FPHWLF/p1626261359325100

Failures with higher parallelism indicate real problems, like missing dependencies, race conditions, etc. And with a growing number of tests, this will keep lengthening the feedback loop (both in CI and locally), which we need to shorten.

[wknapik]

Also, retries built into the testware hide ASAN findings (https://bravesoftware.slack.com/archives/C6R461GF4/p1636537519052500). Taking out those retries would likely cause every run to fail. That applies at least to browser tests.

[rebron]

cc: @bsclifton Need sizing for this one and when we can possibly schedule it.

[wknapik]

Things have improved since this was reported

Parallelism is now increased across platforms by explicitly passing --test-launcher-jobs every time npm run test is called

Linux

• asan:
  • but: 4 -> nproc (48)
  • cut: 28 -> nproc (48)
  • bbt: 4 -> nproc (48)
  • cbt: 20 -> nproc (48)
• prs:
  • but: 4 -> nproc (48)
  • cut: 28 -> nproc (48)
  • bbt: 4 -> nproc (48)
  • cbt: 20 -> nproc (48)
• test and public builds:
  • but: 4 -> nproc (48)
  • bbt: 4 -> nproc (48)

macOS

• asan:
  • but: 4 -> nproc
  • bbt: 4 -> nproc
• prs:
  • but: 4 -> nproc
  • bbt: 4 -> nproc
• test and public builds:
  • but: 4 -> nproc
  • bbt: 4 -> nproc

Windows

• asan:
  • but: 2 -> nproc (32)
  • cut: 16 -> nproc (32)
  • bbt: 4 -> nproc/2 (16)
  • cbt: 8 -> nproc/3 (10)
• prs:
  • but: 4 -> nproc (32)
  • cut: 16 -> nproc (32)
  • bbt: 4 -> nproc/2 (16)
  • cbt: 16 -> nproc/2 (16)
• test and public builds:
  • but: 4 -> nproc (32)
  • bbt: 4 -> nproc/2 (16)

Android

• prs:
  • but: 4 -> nproc (96)
  • bbt: 4 -> nproc (96)
• test and public builds
  • but: 4 -> nproc (96)
  • bbt: 4 -> nproc (96)

iOS

• prs:
  • ? -> nproc
• test and public builds
  • ? -> nproc

[wknapik]

In the course of testing the above changes, it became clear that, at least on desktop, there's an internal limit to --test-launcher-jobs. Running with the value of 1024 doesn't produce any failures as it would be expected to (except in browser tests on Windows). The log says either "Using 1024 parallel jobs", or "Using 512 parallel jobs".

[wknapik]

Also, --test-launcher-bot-mode does not set --test-launcher-jobs even when the latter is not passed explicitly to npm run test. Might be because we set a default of 4.

[wknapik]

We now use --test-launcher-jobs=nproc across platforms and test types, except Windows ASAN builds, where it's nproc/1.5 for brave browser tests and nproc/3 for chromium browser tests (otherwise memory runs out)