Add ci.py helper script to allow parametrizing the pants_version #33
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
--pants-version unspecified would fail because of a bad config, that the contrib module would refer to pants_version even though that line was deleted.
Also modernize .travis.yml to use all the cool goodies we've learned in Pants's CI like anchors and the brew addon package!
Turns out Precise comes installed with 3.6 when requested! So we only need Pyenv for OSX, not Linux.
See if moving back to osx_setup and linux_setup works.
Path wrangling didn't work.
Think this is the style used in most Python projects and ours.
Eric-Arellano
changed the title
Good suggestion Stu!
Will lead to less of a diff on Friday when we go to 1.15.0.dev4. Plus now the pants daemon tests work on macOS for me!
It's unified in Pants repo, so no good reason to split here.
benjyw
approved these changes
Mar 13, 2019
Eric-Arellano
added a commit
that referenced
this pull request
Mar 14, 2019
…se Py36 in Linux (#34) #33 does not propagate return codes for subprocesses, meaning if the `./pants foo` commands fail, the error will print out to the screen but Travis will show everything as working. Fix this by adding `.check_returncode()`. Also we found Linux Trusty and Xenial were using Python 3.5 to execute `ci.py`, as they err when introducing `f-strings`, which are 3.6+. So, use `pyenv global` to consistently use the correct interpreters.
Eric-Arellano
added a commit
that referenced
this pull request
Mar 14, 2019
### Problem While it's good #33 stepped up our CI game by introducing a new script and allowing us to now test an unspecified `pants_version`, we were reinventing the wheel with parsing and modifying `pants.ini`, which led to confusing code. ### Solution Use Python's `ConfigParser` to greatly simplify and make more extendable `ci.py`.
Merged
Eric-Arellano
added a commit
that referenced
this pull request
Apr 1, 2019
### Problem #33 resulted in us installing Python 3.6 on every shard but Trusty and Xenial, and #37 in us installing Python 3.7 on every OSX shard. This slows down our CI unnecessarily and wastes time. ### Solution Cache the pyenv folder for every shard, even if it does not use our custom Pyenv. This is safe to do because if the folder does not exist, Travis will simply upload an empty folder to the cache. This requires adding a unique `CACHE_NAME` to every shard. Further, we must refactor `.travis.yml` to globally define the env vars needed for our custom Pyenv install and to globally define `pyenv_install_py3{6,7}` for better readability. ## Result Overall CI time reduced by 20.4 minutes and overall wall time by 6.1 minutes.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
The
./pantsscript claims to support running with both a pinnedpants_versionin thepants.iniand with thepants_versionunspecified.We currently do not test this unspecified branch, which we should be doing to guarantee that functionality always works.
Further, this is pre-work for #32, which will add
--python-versionas another parameter forci.pyto allow an unspecified Python version vs.pants.inispecifying 2.7 or 3.6 (FYI resulting.travis.yml).Solution
Introduce a new
ci.pyhelper script that uses argparse to have the parameter--pants-version {unspecified,config}.Requires adding Python 3 to CI shards
Because this script uses modern Python 3—i.e. enums, f-strings, type hints, keyword-only args, and
subprocess.run()—it requires Python 3.5+ to run.Even though we can't yet run
./pantswith Python 3 until #32 lands, this PR does some of the pre-work in adding Python 3 to each of the shards so that we can run this script. This is valid to do because every single shard is going to run the tests with both Py27 and Py36—unlike Pants where we have dedicated shards for each interpreter—so there is only benefit in ensuring each shard has Py27 and Py36.Also updates test's pinned
pants_version