Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable pantsd in Travis #7440

Merged
merged 16 commits into from Apr 15, 2019

Conversation

Projects
None yet
4 participants
@blorente
Copy link
Contributor

commented Mar 26, 2019

This is a WIP to iterate on the blockers to enable pantsd in Travis.

@blorente

This comment has been minimized.

Copy link
Contributor Author

commented Mar 26, 2019

Removing the Draft qualifier to trigger travis, but this is still not ready for review.

@blorente blorente marked this pull request as ready for review Mar 26, 2019

@blorente blorente force-pushed the blorente:blorente/pantsd-in-travis branch 4 times, most recently from ccb8c3e to 56ecae1 Mar 27, 2019

@stuhood stuhood referenced this pull request Apr 3, 2019

Closed

[Pantsd] Enable pantsd in Travis #7320

1 of 11 tasks complete

@blorente blorente force-pushed the blorente:blorente/pantsd-in-travis branch from d9818fb to b4f01ff Apr 8, 2019

stuhood and others added some commits Mar 6, 2019

Enable pantsd on "test shards" (probably with some collateral damage).
Also enable shutdown-after-use.

Add a decorator to disable pantsd in integration tests that fail under pantsd.

Blacklist a few more tests (which appear to be flaky, perhaps... gulp), and mark pantsd integration tests as hermetic.

Only apply pantsd on integration test shards.

Blacklist some other stuff

Fix test_stats_local_json_file

Add more reasoning for blacklisted tests

Skip some more tests

Add some colour to the hermetic errors

Also skip test_v2_list

Hacky way of making hermetic tests run

Import fixes

Skip some weird interactions with the hermetic hacks

Pointless Commit to trigger travis

Make stuff non-hermetic

Explicitly disable pantsd in the test coordinator

Add some color to some of the failures

@blorente blorente force-pushed the blorente:blorente/pantsd-in-travis branch from b4f01ff to c803106 Apr 9, 2019

@blorente blorente force-pushed the blorente:blorente/pantsd-in-travis branch from 0c1ae1c to 0492f1b Apr 9, 2019

@blorente

This comment has been minimized.

Copy link
Contributor Author

commented Apr 9, 2019

I'm pretty happy with the state of this, removing the WIP tag, will blacklist more tomorrow if travis is not green. Some things to note:

  • I tried to separate the concepts of "the test coordinator is running pantsd" and "the integration tests spawn pantsd". This PR implements the latter, but whenever the former is necessary (we turn pantsd on by default), it should be isolated and should not require changes to this code.
  • The way I read the env var TRAVIS_AND_PANTS_INTEGRATION_TESTS_SHOULD_HAVE_PANTSD is a bit wonky, I'm totally open to suggestions.

@blorente blorente changed the title [WIP] Enable pantsd in Travis Enable pantsd in Travis Apr 9, 2019

@stuhood

stuhood approved these changes Apr 9, 2019

Copy link
Member

left a comment

Thanks!

Show resolved Hide resolved tests/python/pants_test/pants_run_integration_test.py Outdated
@@ -284,6 +299,13 @@ def run_pants_with_workdir_without_waiting(self, command, workdir, config=None,
'--cache-bootstrap-read', '--cache-bootstrap-write'
])

if self.should_configure_pantsd():
args.append("--enable-pantsd=True")
args.append("--no-shutdown-pantsd-after-run")

This comment has been minimized.

Copy link
@stuhood

stuhood Apr 9, 2019

Member

I thought that our intent right now was to actually enable --shutdown-pantsd-after-run... but I think that you also determined that because of the temporary workdir, pantsd will shutdown after each run anyway?

This comment has been minimized.

Copy link
@blorente

blorente Apr 9, 2019

Author Contributor

So we don't want to shutdown after every run inside the tests, because some of them rely on pantsd living run after run (although they don't rely on this codepath). We do want to set it (probably) in the overpants, the test coordinator, via travis env var. But this should be done at the same time as we enable pantsd by default, and shouldn't affect these changes at all.

This comment has been minimized.

Copy link
@stuhood

stuhood Apr 9, 2019

Member

I would argue that those tests should be marked hermetic and explicitly set the relevant values. But this works too!

@illicitonion
Copy link
Contributor

left a comment

Looks great :) Thanks!

@Eric-Arellano
Copy link
Contributor

left a comment

Exciting!

@Eric-Arellano
Copy link
Contributor

left a comment

Thanks for the renames! Beyond fixing the stage entry, this lgtm.

Show resolved Hide resolved build-support/travis/travis.yml.mustache Outdated
Show resolved Hide resolved build-support/travis/travis.yml.mustache Outdated

@blorente blorente merged commit 08fe94a into pantsbuild:master Apr 15, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@stuhood

This comment has been minimized.

Copy link
Member

commented Apr 15, 2019

Hooray! Thanks @blorente .

blorente added a commit that referenced this pull request Apr 16, 2019

Bootstrap py36 in cron as well (#7579)
**Problem**
The cron job was not configured to bootstrap the 3.6 version of pants, and therefore when we added the "3.6 + pantsd" shards in #7440, these don't work.

**Solution**
To add the 3.6 bootstrapping shards to the cron job.

**Result**
The cron job should now work, maybe?

blorente added a commit that referenced this pull request Apr 17, 2019

Use ci.sh instead of travis-ci.sh (#7582)
**Problem**

`travis-ci.sh` was removed in ef7f664 (#7548), but the changes to the cron shard in  #7440  were not rebased to reflect this update.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.