Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to Pex 1.6.10. #8187

Merged
merged 2 commits into from Aug 20, 2019

Conversation

@jsirois
Copy link
Member

commented Aug 20, 2019

This pulls in a fix for the pexes Pants generates when run under CPython
3.7+ and namespace packages are involved.

See: pantsbuild/pex#756

Before the fix both our own tool pexes and pexes we generated for end
users could fail like so:

$ ./pants --python-setup-interpreter-constraints="['CPython==3.7.*']" \
  test.pytest \
  --cache-ignore \
  tests/python/pants_test/backend/graph_info/tasks:cloc
...
17:16:00 00:26   [test]
...
17:16:00 00:26     [pytest]
                   Invalidated 1 target.
                   scrubbed PYTHONPATH=/home/jsirois/dev/pantsbuild/pants/src/python: from pytest environment
17:16:00 00:26       [run]
                     Traceback (most recent call last):
...
                       File "/home/jsirois/dev/pantsbuild/pants/.pants.d/test/pytest-prep/CPython-3.7.4/e307788111116c190fe04c233d334b12cdc63f77/.bootstrap/pex/pex.py", line 179, in minimum_sys_modules
                         if cls._tainted_path(module_file, site_libs):
                       File "/home/jsirois/dev/pantsbuild/pants/.pants.d/test/pytest-prep/CPython-3.7.4/e307788111116c190fe04c233d334b12cdc63f77/.bootstrap/pex/pex.py", line 157, in _tainted_path
                         paths = frozenset([path, os.path.realpath(path)])
                       File "/home/jsirois/dev/pantsbuild/jsirois-pants/build-support/pants_dev_deps.py37.venv/lib/python3.7/posixpath.py", line 394, in realpath
                         filename = os.fspath(filename)
                     TypeError: expected str, bytes or os.PathLike object, not NoneType
...
Upgrade to Pex 1.6.10.
This pulls in a fix for the pexes Pants generates when run under CPython
3.7+ and namespace packages are involved.

See: pantsbuild/pex#756

Before the fix both our own tool pexes and pexes we generated for end
users could fail like so:
```
$ ./pants --python-setup-interpreter-constraints="['CPython==3.7.*']" \
  test.pytest \
  --cache-ignore \
  tests/python/pants_test/backend/graph_info/tasks:cloc
...
17:16:00 00:26   [test]
...
17:16:00 00:26     [pytest]
                   Invalidated 1 target.
                   scrubbed PYTHONPATH=/home/jsirois/dev/pantsbuild/pants/src/python: from pytest environment
17:16:00 00:26       [run]
                     Traceback (most recent call last):
...
                       File "/home/jsirois/dev/pantsbuild/pants/.pants.d/test/pytest-prep/CPython-3.7.4/e307788111116c190fe04c233d334b12cdc63f77/.bootstrap/pex/pex.py", line 179, in minimum_sys_modules
                         if cls._tainted_path(module_file, site_libs):
                       File "/home/jsirois/dev/pantsbuild/pants/.pants.d/test/pytest-prep/CPython-3.7.4/e307788111116c190fe04c233d334b12cdc63f77/.bootstrap/pex/pex.py", line 157, in _tainted_path
                         paths = frozenset([path, os.path.realpath(path)])
                       File "/home/jsirois/dev/pantsbuild/jsirois-pants/build-support/pants_dev_deps.py37.venv/lib/python3.7/posixpath.py", line 394, in realpath
                         filename = os.fspath(filename)
                     TypeError: expected str, bytes or os.PathLike object, not NoneType
...
```

@jsirois jsirois requested review from Eric-Arellano, gshuflin and stuhood Aug 20, 2019

@Eric-Arellano
Copy link
Contributor

left a comment

Think you need to update here too:

@rule(DownloadedPexBin, [])
def download_pex_bin():
# TODO: Inject versions and digests here through some option, rather than hard-coding it.
url = 'https://github.com/pantsbuild/pex/releases/download/v1.6.8/pex'
digest = Digest('2ca320aede7e7bbcb907af54c9de832707a1df965fb5a0d560f2df29ba8a2f3d', 1866441)

@jsirois

This comment has been minimized.

Copy link
Member Author

commented Aug 20, 2019

Think you need to update here too:

@rule(DownloadedPexBin, [])
def download_pex_bin():
# TODO: Inject versions and digests here through some option, rather than hard-coding it.
url = 'https://github.com/pantsbuild/pex/releases/download/v1.6.8/pex'
digest = Digest('2ca320aede7e7bbcb907af54c9de832707a1df965fb5a0d560f2df29ba8a2f3d', 1866441)

Hrm, yup. Good catch - fixed.

@stuhood
Copy link
Member

left a comment

Thanks!

You had a test timeout that I also saw in another PR: have restarted it here, will bump it there.

@jsirois jsirois merged commit 3883b56 into pantsbuild:master Aug 20, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@jsirois jsirois deleted the jsirois:pex/upgrade branch Aug 20, 2019

@jsirois

This comment has been minimized.

Copy link
Member Author

commented Aug 20, 2019

You had a test timeout that I also saw in another PR: have restarted it here, will bump it there.

Thanks Stu!

patliu85 added a commit to twitter/pants that referenced this pull request Aug 20, 2019
Upgrade to Pex 1.6.10. (pantsbuild#8187)
This pulls in a fix for the pexes Pants generates when run under CPython
3.7+ and namespace packages are involved.

See: pantsbuild/pex#756

Before the fix both our own tool pexes and pexes we generated for end
users could fail like so:
```
$ ./pants --python-setup-interpreter-constraints="['CPython==3.7.*']" \
  test.pytest \
  --cache-ignore \
  tests/python/pants_test/backend/graph_info/tasks:cloc
...
17:16:00 00:26   [test]
...
17:16:00 00:26     [pytest]
                   Invalidated 1 target.
                   scrubbed PYTHONPATH=/home/jsirois/dev/pantsbuild/pants/src/python: from pytest environment
17:16:00 00:26       [run]
                     Traceback (most recent call last):
...
                       File "/home/jsirois/dev/pantsbuild/pants/.pants.d/test/pytest-prep/CPython-3.7.4/e307788111116c190fe04c233d334b12cdc63f77/.bootstrap/pex/pex.py", line 179, in minimum_sys_modules
                         if cls._tainted_path(module_file, site_libs):
                       File "/home/jsirois/dev/pantsbuild/pants/.pants.d/test/pytest-prep/CPython-3.7.4/e307788111116c190fe04c233d334b12cdc63f77/.bootstrap/pex/pex.py", line 157, in _tainted_path
                         paths = frozenset([path, os.path.realpath(path)])
                       File "/home/jsirois/dev/pantsbuild/jsirois-pants/build-support/pants_dev_deps.py37.venv/lib/python3.7/posixpath.py", line 394, in realpath
                         filename = os.fspath(filename)
                     TypeError: expected str, bytes or os.PathLike object, not NoneType
...
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.