New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to pex 1.4.3. #5910

Merged
merged 1 commit into from Jun 7, 2018

Conversation

Projects
None yet
3 participants
@jsirois
Copy link
Member

jsirois commented Jun 4, 2018

This hacks around a few issues with the 1.4.x pex API. We concoct a
minimal local Platform to pass to resolve where a local interpreter
is passed to work around pantsbuild/pex#511.
We also now consolidate PythonInterpreter construction in production
code to helper that ensures the interpreters we create are bare
(isolated) except for the specific extras we require to work around
pantsbuild/pex#510.

Upgrading pex to take advantage of the worked around issues noted above
is tracked by #5922.

Fixes #5906

Upgrade to pex 1.4.3.
This hacks around a few issues with the 1.4.x pex API. We concoct a
minimal local `Platform` to pass to `resolve` where a local interpreter
is passed to work around pantsbuild/pex#511.
We also now consolidate `PythonInterpreter` construction in production
code to helper that ensures the interpreters we create are bare
(isolated) except for the specific extras we require to work around
pantsbuild/pex#510.

Upgrading pex to take advantage of the worked around issues noted above
is tracked by #5922.

Fixes #5906

@jsirois jsirois force-pushed the jsirois:issues/5906 branch from 41b8a73 to d9f7c2c Jun 7, 2018

@kwlzn

kwlzn approved these changes Jun 7, 2018

Copy link
Member

kwlzn left a comment

thanks John! lgtm!

@jsirois jsirois merged commit adffe37 into pantsbuild:master Jun 7, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@jsirois jsirois deleted the jsirois:issues/5906 branch Jun 7, 2018

@illicitonion

This comment has been minimized.

Copy link
Contributor

illicitonion commented Jun 7, 2018

This broke my pants at HEAD:

pants$ ./pants -q --changed-parent=master lint

FAILURE
Exception caught: (<class 'pex.executor.ExecutableNotFound'>)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/pants_loader.py", line 73, in <module>
    main()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/pants_loader.py", line 69, in main
    PantsLoader.run()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/pants_loader.py", line 65, in run
    cls.load_and_execute(entrypoint)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/pants_loader.py", line 58, in load_and_execute
    entrypoint_main()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/pants_exe.py", line 39, in main
    PantsRunner(exiter, start_time=start_time).run()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/pants_runner.py", line 53, in run
    return runner.run()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/local_pants_runner.py", line 45, in run
    self._run()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/local_pants_runner.py", line 90, in _run
    goal_runner_result = goal_runner.run()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/goal_runner.py", line 244, in run
    result = self._execute_engine()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/goal_runner.py", line 232, in _execute_engine
    result = engine.execute(self._context, self._goals)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/engine/legacy_engine.py", line 26, in execute
    self.attempt(context, goals)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/engine/round_engine.py", line 233, in attempt
    goal_executor.attempt(explain)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/engine/round_engine.py", line 49, in attempt
    task.execute()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/backend/python/tasks/select_interpreter.py", line 75, in execute
    interpreter = self._get_interpreter(interpreter_path_file)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/backend/python/tasks/select_interpreter.py", line 98, in _get_interpreter
    interpreter = create_bare_interpreter(binary)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/backend/python/pex_util.py", line 22, in create_bare_interpreter
    interpreter_with_extras = PythonInterpreter.from_binary(binary_path)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/interpreter.py", line 375, in from_binary
    cls.CACHE[binary] = cls._from_binary_external(binary, path_extras)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/interpreter.py", line 337, in _from_binary_external
    stdout, _ = Executor.execute([binary], env=environ, stdin_payload=_generate_identity_source())
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/executor.py", line 99, in execute
    process = cls.open_process(cmd=cmd, env=env, cwd=cwd, **kwargs)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/executor.py", line 83, in open_process
    raise cls.ExecutableNotFound(cmd, e)

Exception message: caught OSError(2, "No such file or directory: '/opt/twitter_mde/package/eepython27/2b9f44c3673ea5b18e3c12bf36c35ef93e3d03a547f10d19f45d7097b09a85a1/bin/python2.7\\tCPython-2.7.14'") while trying to execute `['/opt/twitter_mde/package/eepython27/2b9f44c3673ea5b18e3c12bf36c35ef93e3d03a547f10d19f45d7097b09a85a1/bin/python2.7\tCPython-2.7.14']` while trying to execute `['/opt/twitter_mde/package/eepython27/2b9f44c3673ea5b18e3c12bf36c35ef93e3d03a547f10d19f45d7097b09a85a1/bin/python2.7\tCPython-2.7.14']`
@jsirois

This comment has been minimized.

Copy link
Member

jsirois commented Jun 7, 2018

Hrm. The version bump in the task was meant to cover the interpreter file format change. I'll followup here presently. Sorry for the breakage!

@jsirois

This comment has been minimized.

Copy link
Member

jsirois commented Jun 7, 2018

@illicitonion you probably figured this out but the workaround is a ./pants clean-all :(

@jsirois

This comment has been minimized.

Copy link
Member

jsirois commented Jun 7, 2018

Fix in #5925

@illicitonion

This comment has been minimized.

Copy link
Contributor

illicitonion commented Jun 7, 2018

Thanks for the quick fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment