Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Divide spark tests on ci into multiple batches #2183

Merged
merged 10 commits into from
Jul 19, 2022
Merged

Divide spark tests on ci into multiple batches #2183

merged 10 commits into from
Jul 19, 2022

Conversation

rwedge
Copy link
Contributor

@rwedge rwedge commented Jul 15, 2022

Splits up the spark tests into several separate CI runs to cut down on total time needed to run the unit tests

@codecov
Copy link

codecov bot commented Jul 15, 2022 •
edited
Loading

Codecov Report

Merging #2183 (0ea808c) into main (8057ce5) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #2183   +/-   ##
=======================================
  Coverage   99.29%   99.29%           
=======================================
  Files         145      145           
  Lines       17556    17556           
=======================================
  Hits        17432    17432           
  Misses        124      124

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8057ce5...0ea808c. Read the comment docs.

@rwedge rwedge marked this pull request as ready for review July 18, 2022 15:02
rwedge
rwedge commented Jul 18, 2022
View reviewed changes
setup.cfg
@@ -90,8 +90,7 @@ test =
smart-open >= 5.0.0
urllib3 >= 1.26.5
spark =
pyspark >=3.2.0
pandas >=1.3.0, <1.4.0
woodwork[spark] >= 0.16.2
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since woodwork 0.16.x and 0.17.x have incompatible spark requirements and featuretools still allows woodwork 0.16.x I tied the spark requirements to woodwork's spark requirements

@ozzieD
Copy link
Contributor

ozzieD commented Jul 19, 2022

Looks like its still expecting the aggregated spark tests to run. Other than that this looks good!

@rwedge
Copy link
Contributor Author

rwedge commented Jul 19, 2022

Looks like its still expecting the aggregated spark tests to run. Other than that this looks good!

The aggregated spark tests won't exist once this PR is merged. After the merge I'll update the branch protection rules to expect the new spark tests instead

@rwedge rwedge merged commit 3d9f3a2 into main Jul 19, 2022
@rwedge rwedge deleted the divide-spark-tests-on-ci branch July 19, 2022 16:16
@dvreed77 dvreed77 mentioned this pull request Jul 19, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ozzieD ozzieD ozzieD approved these changes

@dvreed77 dvreed77 Awaiting requested review from dvreed77

@gsheni gsheni Awaiting requested review from gsheni

@tamargrey tamargrey Awaiting requested review from tamargrey

Assignees

@rwedge rwedge

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@rwedge @ozzieD