Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to pex 1.4.3. #5910

Merged
merged 1 commit into from
Jun 7, 2018
Merged

Upgrade to pex 1.4.3. #5910

merged 1 commit into from
Jun 7, 2018

Conversation

jsirois
Copy link
Contributor

@jsirois jsirois commented Jun 4, 2018

This hacks around a few issues with the 1.4.x pex API. We concoct a
minimal local Platform to pass to resolve where a local interpreter
is passed to work around pex-tool/pex#511.
We also now consolidate PythonInterpreter construction in production
code to helper that ensures the interpreters we create are bare
(isolated) except for the specific extras we require to work around
pex-tool/pex#510.

Upgrading pex to take advantage of the worked around issues noted above
is tracked by #5922.

Fixes #5906

This hacks around a few issues with the 1.4.x pex API. We concoct a
minimal local `Platform` to pass to `resolve` where a local interpreter
is passed to work around pex-tool/pex#511.
We also now consolidate `PythonInterpreter` construction in production
code to helper that ensures the interpreters we create are bare
(isolated) except for the specific extras we require to work around
pex-tool/pex#510.

Upgrading pex to take advantage of the worked around issues noted above
is tracked by pantsbuild#5922.

Fixes pantsbuild#5906
Copy link
Member

@kwlzn kwlzn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks John! lgtm!

@jsirois jsirois merged commit adffe37 into pantsbuild:master Jun 7, 2018
@jsirois jsirois deleted the issues/5906 branch June 7, 2018 09:12
@illicitonion
Copy link
Contributor

This broke my pants at HEAD:

pants$ ./pants -q --changed-parent=master lint

FAILURE
Exception caught: (<class 'pex.executor.ExecutableNotFound'>)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/pants_loader.py", line 73, in <module>
    main()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/pants_loader.py", line 69, in main
    PantsLoader.run()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/pants_loader.py", line 65, in run
    cls.load_and_execute(entrypoint)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/pants_loader.py", line 58, in load_and_execute
    entrypoint_main()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/pants_exe.py", line 39, in main
    PantsRunner(exiter, start_time=start_time).run()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/pants_runner.py", line 53, in run
    return runner.run()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/local_pants_runner.py", line 45, in run
    self._run()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/local_pants_runner.py", line 90, in _run
    goal_runner_result = goal_runner.run()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/goal_runner.py", line 244, in run
    result = self._execute_engine()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/bin/goal_runner.py", line 232, in _execute_engine
    result = engine.execute(self._context, self._goals)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/engine/legacy_engine.py", line 26, in execute
    self.attempt(context, goals)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/engine/round_engine.py", line 233, in attempt
    goal_executor.attempt(explain)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/engine/round_engine.py", line 49, in attempt
    task.execute()
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/backend/python/tasks/select_interpreter.py", line 75, in execute
    interpreter = self._get_interpreter(interpreter_path_file)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/backend/python/tasks/select_interpreter.py", line 98, in _get_interpreter
    interpreter = create_bare_interpreter(binary)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/src/python/pants/backend/python/pex_util.py", line 22, in create_bare_interpreter
    interpreter_with_extras = PythonInterpreter.from_binary(binary_path)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/interpreter.py", line 375, in from_binary
    cls.CACHE[binary] = cls._from_binary_external(binary, path_extras)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/interpreter.py", line 337, in _from_binary_external
    stdout, _ = Executor.execute([binary], env=environ, stdin_payload=_generate_identity_source())
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/executor.py", line 99, in execute
    process = cls.open_process(cmd=cmd, env=env, cwd=cwd, **kwargs)
  File "/Users/dwagnerhall/src/github.com/pantsbuild/pants/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/executor.py", line 83, in open_process
    raise cls.ExecutableNotFound(cmd, e)

Exception message: caught OSError(2, "No such file or directory: '/opt/twitter_mde/package/eepython27/2b9f44c3673ea5b18e3c12bf36c35ef93e3d03a547f10d19f45d7097b09a85a1/bin/python2.7\\tCPython-2.7.14'") while trying to execute `['/opt/twitter_mde/package/eepython27/2b9f44c3673ea5b18e3c12bf36c35ef93e3d03a547f10d19f45d7097b09a85a1/bin/python2.7\tCPython-2.7.14']` while trying to execute `['/opt/twitter_mde/package/eepython27/2b9f44c3673ea5b18e3c12bf36c35ef93e3d03a547f10d19f45d7097b09a85a1/bin/python2.7\tCPython-2.7.14']`

@jsirois
Copy link
Contributor Author

jsirois commented Jun 7, 2018

Hrm. The version bump in the task was meant to cover the interpreter file format change. I'll followup here presently. Sorry for the breakage!

@jsirois
Copy link
Contributor Author

jsirois commented Jun 7, 2018

@illicitonion you probably figured this out but the workaround is a ./pants clean-all :(

@jsirois
Copy link
Contributor Author

jsirois commented Jun 7, 2018

Fix in #5925

@illicitonion
Copy link
Contributor

Thanks for the quick fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade pants to pex 1.4.x
3 participants