sysconfg.is_python_build() is buggy

[pythonhacker]

BPO	23746
Nosy	@vsajip, @vstinner, @matrixise
Files	sysconfig_test.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-08-25.14:14:27.441>
created_at = <Date 2015-03-23.09:13:02.710>
labels = ['easy', 'type-feature', 'docs']
title = 'sysconfg.is_python_build() is buggy'
updated_at = <Date 2016-08-25.14:14:27.439>
user = 'https://bugs.python.org/pythonhacker'

bugs.python.org fields:

activity = <Date 2016-08-25.14:14:27.439>
actor = 'python-dev'
assignee = 'docs@python'
closed = True
closed_date = <Date 2016-08-25.14:14:27.441>
closer = 'python-dev'
components = ['Documentation']
creation = <Date 2015-03-23.09:13:02.710>
creator = 'pythonhacker'
dependencies = []
files = ['38649']
hgrepos = []
issue_num = 23746
keywords = ['easy']
message_count = 8.0
messages = ['238993', '238994', '238998', '239000', '239002', '239003', '273649', '273651']
nosy_count = 6.0
nosy_names = ['vinay.sajip', 'vstinner', 'pythonhacker', 'docs@python', 'python-dev', 'matrixise']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue23746'
versions = ['Python 3.5', 'Python 3.6']

[pythonhacker]

On Python 3.5.0a1+ built from source,

>>> import sysconfig
>>> sysconfig.is_python_build()
False
>>> sysconfig.is_python_build(True)
False
>>> sysconfig._PROJECT_BASE
'/opt/bin'
>>> import sys
>>> sys._home
>>>

The problem is, when sys._home is None, this function uses _is_python_source_dir(_PROJECT_BASE) . In this case the _PROJECT_BASE is clearly passed wrongly as '/opt/bin'. That is the INSTALL_PREFIX, not _PROJECT_BASE .

Let us do a small hack and set _PROJECT_BASE to the folder where I build this Python version.

# Of course this can't be reproduced but you get the idea.

>>> sysconfig._PROJECT_BASE='/home/anand/code/cpython/'
>>> sysconfig.is_python_build()
True

The documentation says,

"

sysconfig.is_python_build()

Return True if the current Python installation was built from source.

"

which is clearly in conflict with what it is doing.

From a quick look at sysconfig.py it looks like it is calculating _PROJECT_BASE wrongly.

I can give a patch for this, but first I am more interested in finding out what this function is supposed to do - why have this function if you are not able to get the details of the build environment from the built interpreter ? Clearly it is not doing that here.

The conclusions are part of the attached file in comments.

[vstinner]

IMO it's more a documentation issue than a bug.

sysconfig.is_python_build() returns True when Python is run from its source code directory. For example, I compiled Python in ~/prog/python/default.

~/prog/python/default/python returns True.

I installed python to /opt/py35/bin/python: it returns False.

[pythonhacker]

Thanks. From current documentation it isn't clear. Agree to make this a doc bug then.

It should also be made clear this would work only for the executable built in the source directory. As in,

(Running from the source folder using source Python executable).

anand@toshiba-laptop:~/code/cpython$ ./python 
Python 3.5.0a1+ (default:656543a2ad75, Mar  3 2015, 22:56:27) 
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.is_python_build()
True

[pythonhacker]

On second thoughts, why have such a function if it works correctly only when executed from source Cpython folder using source CPython executable and not otherwise ?

Cuz if I am calling it like that I already KNOW that I am in such an environment.

Possibly this is an internal function to be used by developers to find out if they are in such an environment.

IMHO Then -> make it private (not callable outside) or drop it altogether.

[vstinner]

On second thoughts, why have such a function if it works correctly only when executed from source Cpython folder using source CPython executable and not otherwise ?

It's useful to know such information. Python behaves a little bit
differently when run from the source code (not installed).

Example of usage of this function:

Lib/test/test_asdl_parser.py:12:if not sysconfig.is_python_build():
Lib/test/test_gdb.py:38:if not sysconfig.is_python_build():
Lib/test/test_pyexpat.py:448: if sysconfig.is_python_build():
Lib/test/test_subprocess.py:415:
@unittest.skipIf(sysconfig.is_python_build(),

[pythonhacker]

Lib/test/test_asdl_parser.py:12:if not sysconfig.is_python_build():

As I guessed - it is used internally by unit tests and possibly has no other significan audience.

It is best to document this clearly.

[vsajip]

It's not *only* used in test - it's also used in third party libraries. For example, it's used in distlib (which is vendored by pip) to determine whether it's being run from a source build.

[python-dev]

New changeset bf51f1f3569a by Vinay Sajip in branch '3.5':
Issue bpo-23746: Clarified is_python_build() documentation.
https://hg.python.org/cpython/rev/bf51f1f3569a

New changeset a0a7c5b1af97 by Vinay Sajip in branch 'default':
Closes bpo-23746: merged update from 3.5.
https://hg.python.org/cpython/rev/a0a7c5b1af97