Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Balance pytest test sharding #8108

Merged

Conversation

@stuhood
Copy link
Member

commented Jul 25, 2019

Problem

The pytest test sharding plugin uses mod of test index over the number of shards to balance across shards. That works fine when there are a lot more tests than shards, which used to ~always be the case... until we turned on --no-fast in travis, at which point each pytest invoke would observe only the tests for a single target. This meant that in many cases there were fewer tests than shards in an invoke.

The effect was that we biased toward overloading lower shard counts (because targets with less than a dozen test methods are very common).

Solution

Balance across shards by hashing the pytest item "nodeid" (used elsewhere in this file for item display).

Result

Fewer straggling shards in travis.

@Eric-Arellano
Copy link
Contributor

left a comment

Cool! I always wondered why it was so unbalanced.

@@ -8,6 +8,7 @@

import json
import os
from zlib import crc32

This comment has been minimized.

Copy link
@Eric-Arellano

Eric-Arellano Jul 25, 2019

Contributor

Nit: In this instance, think it’s more readable to import zlib so that it’s used as zlib.crc32.

This comment has been minimized.

Copy link
@stuhood

stuhood Jul 25, 2019

Author Member

Algorithms like crc32 and sha1 are very well defined, and are probably a good counterexample to the need for absolute imports.

Show resolved Hide resolved src/python/pants/backend/python/tasks/pytest/plugin.py Outdated

stuhood added some commits Jul 25, 2019

@stuhood stuhood merged commit a996b57 into pantsbuild:master Jul 25, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@stuhood stuhood deleted the twitter:stuhood/more-balanced-test-sharding branch Jul 25, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.