Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split up CI unit tests into two distinct shards #7867

Merged
merged 1 commit into from Jun 7, 2019

Conversation

Projects
None yet
2 participants
@Eric-Arellano
Copy link
Contributor

commented Jun 6, 2019

Problem

Now that we use the V2 test runner as of #7724, unit tests both take much longer (20 minutes -> 40-50 minutes) and have become very flaky (not exclusively thanks to V2).

Especially because the tests flake so much, it is frustrating to have to wait a whole 50 minutes to rerun the shard.

Solution

We can't use automated sharding because V2 does not support that yet, but we can introduce our own manual shards. One shard runs the V2 tests, and the other runs all blacklisted tests, the contrib tests, and the pants-plugin tests.

Result

Flakes will be slightly less painful, because when something flakes you will not have to run the entire 50 minutes of CI again, but just the subset for that specific shard.

@Eric-Arellano Eric-Arellano requested review from stuhood, jsirois and blorente Jun 6, 2019

@jsirois

jsirois approved these changes Jun 6, 2019

@Eric-Arellano Eric-Arellano merged commit aeee7cd into pantsbuild:master Jun 7, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@Eric-Arellano Eric-Arellano deleted the Eric-Arellano:shard-unit-tests branch Jun 7, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.