Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Parallelize macOS tests #19446
This pull request changes our CI configuration to run tests in parallel on macOS. Specifically, it splits the macOS step into a build phase and a test phase.
During this phase, Azure Pipelines takes care of bootstrapping and building Atom. It then uploads the produced artifacts so that they can be used by the test phase described below.
Note that, perhaps counterintuitively, we will still execute main process tests as part of this phase. This is unfortunate, but there seems to be an issue when trying to execute main process tests on a container different from the one Atom was built on. These tests usually take less than 10 seconds anyways, so they should have a negligible cost on parallelism.
During this phase, Azure Pipelines takes care of downloading the artifacts published by the build phase and use them to run tests. In particular, this phase is further split into 3 jobs that are executed in parallel:
Splitting tests evenly across containers is non-trivial, especially since we don't know how long each test suite takes to execute. We could probably download that information from previous builds and schedule tests accordingly, but in this pull request I decided to go for a more low-tech approach. Empirically, I observed that the splitting strategy illustrated above achieves a good level of parallelism without introducing a ton of complexity.
With these changes, build times when the
Please, note that there is still room for improvement, but it seems like additional enhancements to CI would come at a higher cost and we may want to work on them in a separate moment.