Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TST: Speed up travis a bit #1129

Merged
merged 4 commits into from
Sep 27, 2018
Merged

TST: Speed up travis a bit #1129

merged 4 commits into from
Sep 27, 2018

Conversation

QuLogic
Copy link
Member

@QuLogic QuLogic commented Sep 25, 2018

Rationale

Currently, Travis takes approximately 250s (4m10s) to install conda, cartopy, and its dependencies, 153s (2.5m) to run tests, and 5s for other bits.

  • conda update conda does nothing since we pull the latest miniconda, so it wastes 16s
  • running tests in parallel locally gets me 120s-> 40/50s, or 2.5-3 times faster. I'm hoping we can shave a similar amount of time of the builds here.

Implications

Hopefully, this will get builds a bit closer to 5m.

Updating conda does nothing, and wastes 16 seconds of the build.
Locally, this is about 2.5-3 times faster.
We re-use the same filename a lot, so we need a lock to prevent file I/O
overlapping between processes when running tests in parallel.
@QuLogic
Copy link
Member Author

QuLogic commented Sep 26, 2018

Because we re-use filenames for tests, I added a lock using filelock (since it's used by Iris already) when doing I/O for the image tests. This prevents one process from overwriting the file a different process is trying to read.

Copy link
Contributor

@dopplershift dopplershift left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice speedup!

@dopplershift dopplershift merged commit bda63d4 into SciTools:master Sep 27, 2018
@QuLogic QuLogic deleted the xdist branch September 27, 2018 19:39
@QuLogic QuLogic added this to the 0.17 milestone Sep 27, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants