Idea: Unify CI tests in to single strategy #10842
Conversation
|
As there still some investigation to figure out the time zone issue I have dropped that from this PoC PR . Also I notice that sometimes the bottleneck is waiting for GitHub workers, I have therefore updated this to create exactly the same number of tasks doing the same thing as the current CI workflow. The big difference is fail fast will fail all test workers across Ubuntu, Mac, and Windows. |
|
Current failure is an example of the behavior this PR brings, all test workers were stopped where as on #10855 4 Windows workers kept running for up to 26 mins. Further as this is otherwise replicating the same behavior as the current CI workflow I am going to mark this review to see if is liked or not. |
github-actions
bot
added
the
needs rebase or merge
notatallshaw
force-pushed
the
unify_ci_tests
branch
from
9097f83
to
9b70070
Compare
github-actions
bot
added
the
needs rebase or merge
|
Well this was a PoC but didn't pick up any interest so I'm closing. I think the concept is still good though. |
I had an idea based on my investigations in #10835 that all the CI Tests could be unified in to a single strategy, where as currently they are split across two "Ubuntu & Mac" and "Windows".
The main advantage of this would be the fast fail behavior, when a test fails in a strategy and "fast-fail" is set to True it cancels all other jobs in that same strategy. So all tests would be cancelled, not just "Ubuntu & Mac" or "Windows"
To simplify the logic this PR splits test groups in to 3 (where as they are currently 1 for Ubuntu and Mac and 2 for Windows): Unit, Integration 1, Integration 2. This PR also includes the additional different time zone runs from #10835
To achieve unification I set the TEMP environmental variable for Windows RAM Disk on the command line rather than the using
envparameter in the run steps. I use exclude to specify whether Windows Python 3.8 and 3.9 are not required, and I use include to specify the additional tests under different time zones: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-including-additional-values-in-combinationsLet me know if you like the look of this, fully or partially. It's just an idea I had and I'm not attached to it so feel free to reject also.