Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Distutils fail if sys.executable is None #72738

Closed
iamale mannequin opened this issue Oct 28, 2016 · 10 comments
Closed

Distutils fail if sys.executable is None #72738

iamale mannequin opened this issue Oct 28, 2016 · 10 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@iamale
Copy link
Mannequin

iamale mannequin commented Oct 28, 2016

BPO 28552
Nosy @vstinner, @merwok, @bitdancer, @dstufft, @iamale
PRs
  • bpo-28552: Fix distutils.sysconfig for empty sys.executable #12875
  • [3.7] bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875) #12948
  • [2.7] bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875) #12949
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-04-25.11:16:46.479>
    created_at = <Date 2016-10-28.22:13:05.332>
    labels = ['type-bug', 'library']
    title = 'Distutils fail if sys.executable is None'
    updated_at = <Date 2019-04-25.11:16:46.479>
    user = 'https://github.com/iamale'

    bugs.python.org fields:

    activity = <Date 2019-04-25.11:16:46.479>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-04-25.11:16:46.479>
    closer = 'vstinner'
    components = ['Distutils']
    creation = <Date 2016-10-28.22:13:05.332>
    creator = 'iamale'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 28552
    keywords = ['patch']
    message_count = 10.0
    messages = ['279633', '279653', '279665', '279692', '279694', '340489', '340829', '340833', '340835', '340836']
    nosy_count = 5.0
    nosy_names = ['vstinner', 'eric.araujo', 'r.david.murray', 'dstufft', 'iamale']
    pr_nums = ['12875', '12948', '12949']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue28552'
    versions = []

    @iamale
    Copy link
    Mannequin Author

    iamale mannequin commented Oct 28, 2016

    For example, Jython 2.7. When we try to execute jython-standalone -m pip, distutils.sysconfig tries to parse sys.executable as a path and obviously fails:

        Traceback (most recent call last):
          File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/runpy.py", line 161, in _run_module_as_main
          File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/runpy.py", line 72, in _run_code
          File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/ensurepip/__main__.py", line 4, in <module>
          File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/ensurepip/__init__.py", line 220, in _main
          File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/ensurepip/__init__.py", line 123, in bootstrap
          File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/ensurepip/__init__.py", line 45, in _run_pip
          File "/tmp/tmp9QnVEm/pip-1.6-py2.py3-none-any.whl/pip/__init__.py", line 10, in <module>
          File "/tmp/tmp9QnVEm/pip-1.6-py2.py3-none-any.whl/pip/util.py", line 13, in <module>
          File "/tmp/tmp9QnVEm/pip-1.6-py2.py3-none-any.whl/pip/backwardcompat/__init__.py", line 115, in <module>
          File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/distutils/sysconfig.py", line 28, in <module>
          File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/posixpath.py", line 367, in realpath
          File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/posixpath.py", line 373, in _joinrealpath
          File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/posixpath.py", line 61, in isabs
        AttributeError: 'NoneType' object has no attribute 'startswith'

    @iamale iamale mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Oct 28, 2016
    @bitdancer
    Copy link
    Member

    I'm not sure it is reasonable to expect distutils to work if sys.executable is not valid. What do you think distutils could do instead?

    @iamale
    Copy link
    Mannequin Author

    iamale mannequin commented Oct 29, 2016

    Well, project_base (which is what sys.executable is used for) is used only during build (i. e. python_build is True), if I understand it correctly. Normally, sys.prefix and sys.exec_prefix are used for all the paths.

    Also, project_base can be explicitly set using _PYTHON_PROJECT_BASE environ variable, in which case we still don't need sys.executable (but it still would fail without one right now).

    @bitdancer
    Copy link
    Member

    Yes, you are right; I was thinking that distutils and/or pip re-executed python for certain tasks, but upon reflection I don't think they do.

    @dstufft
    Copy link
    Member

    dstufft commented Oct 29, 2016

    We re-execute Python to run setup.py. Even from wheels we do it to compile pyc files.

    Sent from my iPhone

    On Oct 29, 2016, at 2:38 PM, R. David Murray <report@bugs.python.org> wrote:

    R. David Murray added the comment:

    Yes, you are right; I was thinking that distutils and/or pip re-executed python for certain tasks, but upon reflection I don't think they do.

    ----------
    stage: -> needs patch


    Python tracker <report@bugs.python.org>
    <http://bugs.python.org/issue28552\>


    @vstinner
    Copy link
    Member

    This issue can be reproduced with:

    diff --git a/Lib/site.py b/Lib/site.py
    index ad1146332b..c850109c19 100644
    --- a/Lib/site.py
    +++ b/Lib/site.py
    @@ -638,3 +638,5 @@ def _script():
     
     if __name__ == '__main__':
         _script()
    +
    +sys.executable = None

    Attached PR 12875 fix distutils.sysconfig and the distutils build command if sys.executable is None or an empty string. I don't expect that everything works magically, but at least, it's possible to run "./python -m distutils.sysconfig" and use "make" in Python which runs "./python -E ./setup.py build". I'm surprised, but setup.py is able to build C extensions using sys.executable = None :-)

    I made a similar fix for sysconfig in bpo-7774: commit 171ba05.

    @vstinner
    Copy link
    Member

    New changeset 0ef8c15 by Victor Stinner in branch 'master':
    bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875)
    0ef8c15

    @vstinner
    Copy link
    Member

    New changeset 3076a3e by Victor Stinner (Miss Islington (bot)) in branch '3.7':
    bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875) (GH-12948)
    3076a3e

    @vstinner
    Copy link
    Member

    New changeset f4edd39 by Victor Stinner in branch '2.7':
    bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875) (GH-12949)
    f4edd39

    @vstinner
    Copy link
    Member

    I fixed the bug in Python 2.7, 3.7 and master branch (future 3.8).

    Thanks Alexander P for the bug report ;-)

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants