Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce CI Time by separating Prophet tests #2644

Merged
merged 23 commits into from
Aug 24, 2021
Merged

Reduce CI Time by separating Prophet tests #2644

merged 23 commits into from
Aug 24, 2021

Conversation

ParthivNaresh
Copy link
Contributor

@ParthivNaresh ParthivNaresh commented Aug 17, 2021

Fixes: #2588

This will bring git-test-other down from 19/20 min to 5/6 min. In anticipation of integrating Prophet into AutoML, running git-test-automl means that many of our tests that rely on instantiating every component/estimator will still need to instantiate Prophet, so I don't think removing it from git-test-automl is a good idea.

@codecov
Copy link

codecov bot commented Aug 17, 2021

Codecov Report

Merging #2644 (1a25066) into main (eb6af1b) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@          Coverage Diff          @@
##            main   #2644   +/-   ##
=====================================
  Coverage   99.9%   99.9%           
=====================================
  Files        298     298           
  Lines      27305   27305           
=====================================
  Hits       27261   27261           
  Misses        44      44           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eb6af1b...1a25066. Read the comment docs.

@@ -8,6 +8,7 @@ Release Notes
* Deleted ``_put_into_original_order`` helper function :pr:`2639`
* Documentation Changes
* Testing Changes
* Speed up CI time by splitting Prophet tests into a separate workflow in GitHUB :pr:`2644`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Speed up CI time by splitting Prophet tests into a separate workflow in GitHUB :pr:`2644`
* Speed up CI time by splitting Prophet tests into a separate workflow in GitHub :pr:`2644`

@ParthivNaresh ParthivNaresh changed the title Reduce CI Time Reduce CI Time by separating Prophet tests Aug 18, 2021
Makefile Outdated

.PHONY: git-test-prophet
git-test-prophet:
pytest evalml/tests/component_tests/test_prophet_regressor.py evalml/tests/component_tests/test_components.py evalml/tests/component_tests/test_utils.py evalml/tests/pipeline_tests/ evalml/tests/objective_tests/ evalml/tests/utils_tests/ -n 2 --durations 0 --timeout 300 --doctest-modules --cov=evalml --junitxml=test-reports/git-test-prophet-junit.xml --doctest-continue-on-failure
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Due to how many of our tests are constructed, we end up instantiating every estimator including Prophet, so I had to break up git-test-other to focus on the files that end up requiring Prophet. This is also the reason that I'm leaving git-test-automl as it is.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the objective_tests be in test-other? They don't depend on prophet right?

Copy link
Contributor

@chukarsten chukarsten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to merge to me

Copy link
Contributor

@bchen1116 bchen1116 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Contributor

@freddyaboulton freddyaboulton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @ParthivNaresh !

Makefile Outdated

.PHONY: git-test-prophet
git-test-prophet:
pytest evalml/tests/component_tests/test_prophet_regressor.py evalml/tests/component_tests/test_components.py evalml/tests/component_tests/test_utils.py evalml/tests/pipeline_tests/ evalml/tests/objective_tests/ evalml/tests/utils_tests/ -n 2 --durations 0 --timeout 300 --doctest-modules --cov=evalml --junitxml=test-reports/git-test-prophet-junit.xml --doctest-continue-on-failure
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the objective_tests be in test-other? They don't depend on prophet right?

@ParthivNaresh ParthivNaresh merged commit 814686f into main Aug 24, 2021
@freddyaboulton freddyaboulton deleted the reduce-CI-time branch May 13, 2022 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Prophet: speed up CI
5 participants