Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run contrib tests with unit tests and integration tests for fewer CI delays #7709

Merged
merged 8 commits into from Jun 5, 2019

Conversation

Projects
None yet
4 participants
@Eric-Arellano
Copy link
Contributor

commented May 13, 2019

Problem

The contrib shards consistently are the blocker for CI runs because they appear near the end of the shards.

While discussing options to fix this, we realized that there is no compelling reason for the contrib folder to have its own dedicated shard(s). Instead, it would be valuable to use the split between unit tests vs. integration tests with the contrib code too.

Solution

Merge contrib tests into the unit tests and integration tests shards.

Result

The unit tests and integration tests shards will take slightly longer, but we will no longer be blocked by contrib at the end of the run.

@blorente
Copy link
Contributor

left a comment

Cool! I think the bottleneck are still the integration tests, so this change is cool :)

Show resolved Hide resolved build-support/bin/ci.sh
Exclude testprojects
The contrib folders have these, but we don't want to run them.
@stuhood
Copy link
Member

left a comment

Thanks!

@Eric-Arellano Eric-Arellano changed the title Run contrib tests with unit tests and integration tests for less CI delays WIP: Run contrib tests with unit tests and integration tests for less CI delays May 13, 2019

@Eric-Arellano

This comment has been minimized.

Copy link
Contributor Author

commented May 13, 2019

This fails due to running contrib unit tests with --chroot. I'm going to punt on this for now and maybe have a precursor PR that uses --chroot with contrib unit tests.

@Eric-Arellano Eric-Arellano changed the title WIP: Run contrib tests with unit tests and integration tests for less CI delays WIP: Run contrib tests with unit tests and integration tests for fewer CI delays Jun 3, 2019

@Eric-Arellano

This comment has been minimized.

Copy link
Contributor Author

commented Jun 4, 2019

Closing in favor of #7845. This solution is now out of sync with master as well. If we wanted to move forward with this approach, we should start by getting chroot to work for contrib.

@Eric-Arellano Eric-Arellano deleted the Eric-Arellano:merge-contrib-ci branch Jun 4, 2019

@Eric-Arellano Eric-Arellano restored the Eric-Arellano:merge-contrib-ci branch Jun 4, 2019

@Eric-Arellano Eric-Arellano reopened this Jun 4, 2019

@Eric-Arellano Eric-Arellano changed the title WIP: Run contrib tests with unit tests and integration tests for fewer CI delays Run contrib tests with unit tests and integration tests for fewer CI delays Jun 4, 2019

@Eric-Arellano
Copy link
Contributor Author

left a comment

Now ready for review.

This PR means that the unit tests will take even longer :/ @blorente and I discussed creating our own manual shards for unit tests: one to run V2 and one to run V1 + contrib + ci.sh -l. I punted on that work to do in a followup PR.

@@ -27,18 +27,9 @@ Usage: $0 (-h|-2fxbkmrjlpuneycitzsw)
-j run core jvm tests
-l run internal backends python tests
-p run core python tests
-u SHARD_NUMBER/TOTAL_SHARDS

This comment has been minimized.

Copy link
@Eric-Arellano

Eric-Arellano Jun 4, 2019

Author Contributor

This should have been removed from #7724.

@jsirois

jsirois approved these changes Jun 4, 2019

Show resolved Hide resolved build-support/bin/ci.sh Outdated
Show resolved Hide resolved build-support/bin/ci.sh Outdated
@Eric-Arellano
Copy link
Contributor Author

left a comment

Thanks for the review John!

Show resolved Hide resolved build-support/bin/ci.sh Outdated
Show resolved Hide resolved build-support/bin/ci.sh Outdated
@jsirois

jsirois approved these changes Jun 5, 2019

@Eric-Arellano Eric-Arellano merged commit 922cd76 into pantsbuild:master Jun 5, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@Eric-Arellano Eric-Arellano deleted the Eric-Arellano:merge-contrib-ci branch Jun 5, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.