Divide spark tests on ci into multiple batches#2183
Conversation
Codecov Report
@@ Coverage Diff @@
## main #2183 +/- ##
=======================================
Coverage 99.29% 99.29%
=======================================
Files 145 145
Lines 17556 17556
=======================================
Hits 17432 17432
Misses 124 124 Continue to review full report at Codecov.
|
| spark = | ||
| pyspark >=3.2.0 | ||
| pandas >=1.3.0, <1.4.0 | ||
| woodwork[spark] >= 0.16.2 |
There was a problem hiding this comment.
Since woodwork 0.16.x and 0.17.x have incompatible spark requirements and featuretools still allows woodwork 0.16.x I tied the spark requirements to woodwork's spark requirements
|
Looks like its still expecting the aggregated spark tests to run. Other than that this looks good! |
The aggregated spark tests won't exist once this PR is merged. After the merge I'll update the branch protection rules to expect the new spark tests instead |
Splits up the spark tests into several separate CI runs to cut down on total time needed to run the unit tests