test: fix CI - chunk up Jest tests #2096
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jun 30, 2022
petermetz
requested review from
takeutak,
izuru0,
jagpreetsinghsasan and
sandeepnRES
as code owners
jagpreetsinghsasan
approved these changes
Jul 4, 2022
takeutak
approved these changes
Jul 6, 2022
1. ci.yml workflows went through a massive refactoring to parallelize the test execution for two main reasons: First this makes it faster, second this also reduces the flakyness that comes from our ever growing test suite making the test runner consume more and more RAM (the back story here is that Yarn needed more than 2 GB RAM at first, and when we bumped the heap size up to 3 GB it exhausted that soon after at which point it became clear that this is not sustainable at all on the long run since our GitHub CI runners come with 7 GB RAM total) 2. tools/ci.sh can now be configured via environment variables: - DEV_BUILD_DISABLED (defaults to false) true/false: if set to true the initial configure script execution is skipped - FULL_BUILD_DISABLED (defaults to false) true/false: if set to true the final full build (webpack & prod builds) is skipped - JEST_TEST_RUNNER_DISABLED (defaults to false) true/false: if set to true, Jest tests are skipped completely. - JEST_TEST_PATTERN (default comes from jest.config.js) string pattern: if specified, it will narrow the scope of tests as specified - TAPE_TEST_RUNNER_DISABLED (defaults to false) true/false: if set to true, Tape tests are skipped completely. - TAPE_TEST_PATTERN (default comes from .taprc config file) string pattern: if specified, it will narrow the scope of tests as specified 3. sorted all the yaml keys in the ci.yml file for easier location of the jobs in the future since there is now 30+ jobs in the file which are taking up a total of 1k LoC at the moment. 4. TODO for the future is to start using job templates so that the LoC can be reduced significantly See point 3. about the details on this. Fixes hyperledger-cacti#2090 Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com>
petermetz
force-pushed
the
petermetz/issue2090
branch
from
ff805cf
to
b9706bf
Compare
johnhomantaring
added a commit
to johnhomantaring/cactus
that referenced
this pull request
Jul 20, 2022
Fixes hyperledger-cacti#1890 Depends on hyperledger-cacti#2096 Signed-off-by: john.h.o.mantaring-at-475704139995 <john.h.o.mantaring@accenture.com>
johnhomantaring
added a commit
to johnhomantaring/cactus
that referenced
this pull request
Jul 20, 2022
Fixes hyperledger-cacti#1890 Depends on hyperledger-cacti#2096 Signed-off-by: john.h.o.mantaring-at-475704139995 <john.h.o.mantaring@accenture.com>
petermetz
pushed a commit
to johnhomantaring/cactus
that referenced
this pull request
Jul 20, 2022
Fixes hyperledger-cacti#1890 Depends on hyperledger-cacti#2096 Signed-off-by: john.h.o.mantaring-at-475704139995 <john.h.o.mantaring@accenture.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ci.yml workflows went through a massive refactoring to parallelize
the test execution for two main reasons: First this makes it faster,
second this also reduces the flakyness that comes from our ever growing
test suite making the test runner consume more and more RAM (the back
story here is that Yarn needed more than 2 GB RAM at first, and
when we bumped the heap size up to 3 GB it exhausted that soon after at
which point it became clear that this is not sustainable at all on the
long run since our GitHub CI runners come with 7 GB RAM total)
tools/ci.sh can now be configured via environment variables:
DEV_BUILD_DISABLED (defaults to false)
true/false: if set to true the initial configure script execution is skipped
FULL_BUILD_DISABLED (defaults to false)
true/false: if set to true the final full build (webpack & prod builds) is skipped
JEST_TEST_RUNNER_DISABLED (defaults to false)
true/false: if set to true, Jest tests are skipped completely.
string pattern: if specified, it will narrow the scope of tests as specified
true/false: if set to true, Tape tests are skipped completely.
string pattern: if specified, it will narrow the scope of tests as specified
sorted all the yaml keys in the ci.yml file for easier location of the jobs in the future
since there is now 30+ jobs in the file which are taking up a total of 1k LoC at the moment.
TODO for the future is to start using job templates so that the LoC can be reduced significantly
See point 3. about the details on this.
Fixes #2090
Signed-off-by: Peter Somogyvari peter.somogyvari@accenture.com