Skip to content

Divide spark tests on ci into multiple batches#2183

Merged
rwedge merged 10 commits into
mainfrom
divide-spark-tests-on-ci
Jul 19, 2022
Merged

Divide spark tests on ci into multiple batches#2183
rwedge merged 10 commits into
mainfrom
divide-spark-tests-on-ci

Conversation

@rwedge

@rwedge rwedge commented Jul 15, 2022

Copy link
Copy Markdown
Contributor

Splits up the spark tests into several separate CI runs to cut down on total time needed to run the unit tests

@codecov

codecov Bot commented Jul 15, 2022

Copy link
Copy Markdown

Codecov Report

Merging #2183 (0ea808c) into main (8057ce5) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #2183   +/-   ##
=======================================
  Coverage   99.29%   99.29%           
=======================================
  Files         145      145           
  Lines       17556    17556           
=======================================
  Hits        17432    17432           
  Misses        124      124           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8057ce5...0ea808c. Read the comment docs.

@rwedge
rwedge marked this pull request as ready for review July 18, 2022 15:02
Comment thread setup.cfg
spark =
pyspark >=3.2.0
pandas >=1.3.0, <1.4.0
woodwork[spark] >= 0.16.2

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since woodwork 0.16.x and 0.17.x have incompatible spark requirements and featuretools still allows woodwork 0.16.x I tied the spark requirements to woodwork's spark requirements

@ozzieD

ozzieD commented Jul 19, 2022

Copy link
Copy Markdown
Contributor

Looks like its still expecting the aggregated spark tests to run. Other than that this looks good!

@rwedge

rwedge commented Jul 19, 2022

Copy link
Copy Markdown
Contributor Author

Looks like its still expecting the aggregated spark tests to run. Other than that this looks good!

The aggregated spark tests won't exist once this PR is merged. After the merge I'll update the branch protection rules to expect the new spark tests instead

@rwedge
rwedge merged commit 3d9f3a2 into main Jul 19, 2022
@rwedge
rwedge deleted the divide-spark-tests-on-ci branch July 19, 2022 16:16
@dvreed77 dvreed77 mentioned this pull request Jul 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants