🚀 🚀 🚀 🚀 ✅ 🏗 Run unit tests in parallel for 4x performance #35094
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rileyajones
force-pushed
the
test-workers
branch
4 times, most recently
from
4d93372
to
31c3d27
Compare
rileyajones
force-pushed
the
test-workers
branch
from
31c3d27
to
37eb140
Compare
|
i appreciate that there were exactly 4 🚀 s |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why Start With the Unit Tests?
While unit tests do not block job completion, they are our longest running and most credit consuming test type.
At the efforts to moment dramatically reducing the runtime of the e2e tests could actually be bottlenecked by the performance of of the
Unit Tests. For example in the below image if theEnd-to-End Testsruntime were cut in1/4theNomodule Build->End-To-End Testsflow would take approximately 14 minutes and complete prior to theUnit TestsandBundle Sizecheck completing.Performance to Be Gained
The job runs on a "Large Linux Machine" which has 4 cores, however, we do not take advantage of this fact by run tests synchronously.
By having the main test job fork itself and run a configurable amount of jobs in parallel we SHOULD be able to speed up the test runtime by approximately
4x(the number of cores is a good starting point though we may be able to gain even greater performance).2 Workers
Moving Forward
If this experiment is a success the process will be replicated to work with the e2e and integration tests.