New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unblacklist JVM check_style Python 3 integration test thanks to using Python 3.5+ #6882

Merged
merged 3 commits into from Dec 12, 2018

Conversation

Projects
None yet
2 participants
@Eric-Arellano
Copy link
Contributor

Eric-Arellano commented Dec 7, 2018

Problem

shutil.copytree was failing in CI for the backend/jvm/tasks:checkstyle_integration target due to a bug with Python 3.4.

shutil.Error: [('/home/travis/build/pantsbuild/pants/build-support/bin/native/src', '/tmp/tmpjdehtsq4/build-support/bin/native/src', "[Errno 21] Is a directory: '/home/travis/build/pantsbuild/pants/build-support/bin/native/src'"), ('/home/travis/build/pantsbuild/pants/build-support/pants_dev_deps.venv/include/python2.7', '/tmp/tmpjdehtsq4/build-support/pants_dev_deps.venv/include/python2.7', "[Errno 21] Is a directory: '/home/travis/build/pantsbuild/pants/build-support/pants_dev_deps.venv/include/python2.7'"), ('/home/travis/build/pantsbuild/pants/build-support/pants_dev_deps.venv/lib/python2.7/config', '/tmp/tmpjdehtsq4/build-support/pants_dev_deps.venv/lib/python2.7/config', "[Errno 21] Is a directory: '/home/travis/build/pantsbuild/pants/build-support/pants_dev_deps.venv/lib/python2.7/config'"), ('/home/travis/build/pantsbuild/pants/build-support/pants_dev_deps.venv/lib/python2.7/encodings', '/tmp/tmpjdehtsq4/build-support/pants_dev_deps.venv/lib/python2.7/encodings', "[Errno 21] Is a directory: '/home/travis/build/pantsbuild/pants/build-support/pants_dev_deps.venv/lib/python2.7/encodings'"), ('/home/travis/build/pantsbuild/pants/build-support/pants_dev_deps.venv/lib/python2.7/lib-dynload', '/tmp/tmpjdehtsq4/build-support/pants_dev_deps.venv/lib/python2.7/lib-dynload', "[Errno 21] Is a directory: '/home/travis/build/pantsbuild/pants/build-support/pants_dev_deps.venv/lib/python2.7/lib-dynload'")]

This error is confusing, because those are intentionally directories. Also, it doesn't reproduce on Python 2.7 and other Python 3 versions.

The bug is from the std lib's handling of symbolic directories. shutil.copytree treated all symbolic folders as files, which leads to the error is a directory. This was fixed in Python 3.5+.

Solution

We upgraded Travis to Python 3.5 with #6885. This allows us to raise the floor of minimum Python 3 to 3.5, fixing this bug.

@Eric-Arellano Eric-Arellano force-pushed the Eric-Arellano:py3-jvm-checkstyle branch from 1df69dc to 0c847ee Dec 7, 2018

@Eric-Arellano Eric-Arellano changed the title WIP - fix JVM check_style Python 3 integration test Blocked - fix JVM check_style Python 3 integration test Dec 7, 2018

Eric-Arellano added a commit that referenced this pull request Dec 12, 2018

Upgrade Travis to Xenial (Ubuntu 16.04) (#6885)
Travis released Xenial (Ubuntu 16.04) (see https://changelog.travis-ci.com/ubuntu-xenial-16-04-build-environment-is-here!-79690). This PR moves us to this new image. 

The two major changes are:
- Switch to Python 3.5 in CI. This coincides with prior discussions in Slack that we want to target Python 3.5 as the minimum Python 3 version. This is to give us access to native type hints and async/await syntax, along with general improvements such as fixing the `shutil` bug identified in #6882.
- Switch to openjdk over oraclejdk. This is mostly because travis only bakes version 10 and 11 of the oracle jdk and has removed all the tooling to get other versions of that jdk. While image comes with version 8 of the openjdk.

@Eric-Arellano Eric-Arellano changed the title Blocked - fix JVM check_style Python 3 integration test Unblacklist JVM check_style Python 3 integration test thanks to using Python 3.5+ Dec 12, 2018

@Eric-Arellano Eric-Arellano requested review from stuhood and wisechengyi Dec 12, 2018

@stuhood
Copy link
Member

stuhood left a comment

Yay!

@Eric-Arellano Eric-Arellano merged commit cf40141 into pantsbuild:master Dec 12, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@Eric-Arellano Eric-Arellano deleted the Eric-Arellano:py3-jvm-checkstyle branch Dec 12, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment