Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Travis test plan to get around 50 minute job limit #385

Closed
desilinguist opened this issue Nov 8, 2017 · 1 comment
Closed

Refactor Travis test plan to get around 50 minute job limit #385

desilinguist opened this issue Nov 8, 2017 · 1 comment
Assignees
Milestone

Comments

@desilinguist
Copy link
Member

Travis seems to have a 50 minute hard limit on any job in any build and that includes VM boot up time. Now that we have so many tests, hitting that 50 minute job limit seems to be happening with more recent PRs. Here's a breakdown of how long each test_X.py file takes on my MacBook Pro.

File Info
test_ablation 8 tests in 35.305s
test_classification 100 tests in 56.227s
test_cv 7 tests in 7.324s
test_custom_learner 3 tests in 1.154s
test_featureset 81 tests in 337.709s
test_input 123 tests in 0.646s
test_metrics 210 tests in 0.449s
test_output 27 tests in 239.037s
test_preprocessing 7 tests in 1.394s
test_regression 102 tests in 261.897s
test_utilities 478 tests in 21.969s

Obviously, these numbers doesn't add up to 50 minutes because (a) I am not setting up the environment each time and (b) the MacBook is way faster than the Ubuntu 14.04 VMs that Travis uses). However, we can still go by the relative ratios - it looks like test_featureset takes the most time, followed by test_regression and then test_output. So, may be we can have 3 separate jobs: one for test_featureset, one for test_regression and one for test_output + all the other files.

Thoughts?

@desilinguist desilinguist added this to the 1.5 milestone Nov 9, 2017
@desilinguist desilinguist self-assigned this Nov 9, 2017
@desilinguist
Copy link
Member Author

Addressed by #387.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant