expose the Android API level in sysconfig.get_config_vars()

[xdegaye]

BPO	27442
Nosy	@vstinner, @ned-deily, @skrah, @xdegaye, @berkerpeksag, @yan12125
Files	android_api.patch android_api_2.patch android_api_3.patch issue27442-3.patch: regenerated android_api_3.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/xdegaye'
closed_at = <Date 2016-07-09.09:09:29.464>
created_at = <Date 2016-07-02.14:52:55.319>
labels = ['type-feature', 'library']
title = 'expose the Android API level in sysconfig.get_config_vars()'
updated_at = <Date 2016-07-09.09:09:29.463>
user = 'https://github.com/xdegaye'

bugs.python.org fields:

activity = <Date 2016-07-09.09:09:29.463>
actor = 'xdegaye'
assignee = 'xdegaye'
closed = True
closed_date = <Date 2016-07-09.09:09:29.464>
closer = 'xdegaye'
components = ['Library (Lib)']
creation = <Date 2016-07-02.14:52:55.319>
creator = 'xdegaye'
dependencies = []
files = ['43610', '43622', '43636', '43660']
hgrepos = []
issue_num = 27442
keywords = ['patch']
message_count = 29.0
messages = ['269716', '269717', '269719', '269720', '269721', '269724', '269726', '269748', '269756', '269757', '269807', '269828', '269829', '269832', '269834', '269878', '269902', '269937', '269938', '269954', '269958', '269962', '269963', '269980', '269981', '269982', '270008', '270013', '270031']
nosy_count = 7.0
nosy_names = ['vstinner', 'ned.deily', 'skrah', 'xdegaye', 'python-dev', 'berker.peksag', 'yan12125']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue27442'
versions = ['Python 3.6']

[xdegaye]

Expose the Android API level that python was built against, in sys.implementation as _android_api.
Purposes:
* Identify the platform as Android.
* Allow detecting a mismatch with the runtime sdk version returned by platform.android_ver() (bpo-26855), for example when the runtime sdk is lower than the built API level.

[xdegaye]

Patch added.

[yan12125]

I don't think sys.implementation is a good place to contain the information of the underlying platform. By Doc/sys.rst:

An object containing information about the implementation of the currently running Python interpreter.

[yan12125]

Sorry, it's Doc/library/sys.rst

[xdegaye]

I don't think sys.implementation is a good place to contain the information of the underlying platform.

Quite the opposite, the ndk API level gives an information about the implementation of the currently running Python interpreter saying that this Python has been built against this version of Android libc identified by this API level.

[yan12125]

OK I see the rationale.

[vstinner]

I woupd prefer to have a public attribute.

[ned-deily]

Typically, for other sorts of build configuration data, we have relied on extracting that from the ./configure-produced Makefile and making it available via sysconfig.get_config_var(). I think we should be cautious about bloating sys.implementation with platform-specific data unless there is an overriding need for it, for example, if it is needed during interpreter initialization before sysconfig can be initialized. If not, I'd look at adding the needed values as configuration variables in configure.ac.

[xdegaye]

@ned
The information that the interpreter is an Android built, is needed in bpo-16353 [1][2]. In msg266135, the corresponding patch adds the 'is_android' attribute to the already bloated sys module. I had thought then about using sysconfig.get_config_var() as you are suggesting, but this means that the os module would need now to import sysconfig on all linux platforms and build this big dictionary (about 650 entries on linux).

So it seems that the choice is between:
(1) adding an attribute (is_android or android_api) to the bloated sys module
(2) bloating sys.implementation with platform-specific data

I think that (2) would be better since _android_api is an information about the implementation, it tells that in these binaries, such and such features are available or not (bpo-26857 for example). The drawback is that it would not be documented except in Misc/NEWS [2].

@victor
Do you mean a public attribute of sys.implementation ?
Adding a new sys.implementation required attribute is described in PEP-421. In that case the attribute name could be 'libc_version' or 'libc'.

[1] msg175006 suggested another approach though, but this seems to be abandoned.
[2] For completeness, Stefan has submitted a feature request to google in msg266089.

[xdegaye]

Well, bpo-16353 has been entered upon attempting to fix bpo-16255 (see msg173477).

So an alternative exists. bpo-16353 could be closed as 'wont't fix'. The list of locations where '/bin/sh' is hard coded in the standard library in msg266084 shows that only the subprocess module and the test suite need to know the location of the system shell. So the subprocess module and bpo-27027 could deduct the location of this shell via sysconfig.get_config_var('android_api').

[xdegaye]

A patch to expose the Android API level in sysconfig.get_config_vars().

[xdegaye]

According to bpo-27453, do this minor change in the patch: s/$CC -E/$CPP $CPPFLAGS.

[yan12125]

Here's an issue - there's already a macro called ANDROID_API defined in libcutils [1] If someone is going to integrate Python into AOSP, redefining macros may cause a problem.

[1] https://android.googlesource.com/platform/system/core/+/master/include/cutils/compiler.h#42

[xdegaye]

Integrating Python into AOSP does not make sense. The patch can be changed with s/ANDROID_API/ANDROID_API_LEVEL.

[yan12125]

Yep adding Python to Android's build system is a rare case. Just to mention there's already an macro and avoiding possible redefined macros is always good.

[xdegaye]

New patch incorporating the substitutions to '$CPP $CPPFLAGS' and ANDROID_API_LEVEL.

[vstinner]

+ @unittest.skipIf(sysconfig.get_config_var('ANDROID_API_LEVEL') == 0,
+ "not an android platform")

Hum, sysconfig.get_config_var() returns None for unknown variable. Why checking ==0?

@xavier: Are you generating the patch using "hg diff"? I don't see the base revision in your patch, and so there is no [Review] link on your patch.

[xdegaye]

In sysconfig.parse_config_h(), the variables in pyconfig.h that have a commented-out '#undef' line are set to 0. Fortunately, there is no Android API level 0.

Checking '== 0' ensures that autoreconf has been run to add '#undef ANDROID_API_LEVEL' to pyconfig.h.in. If this autoreconf step were to be missed, the test would (correctly) fail on the buildbots that are not Android as get_config_var() would return None then and the test would not be skipped and fail.

Most of the tests in the Python test suite do check 'not sysconfig.get_config_var()' instead, except:
Lib/test/test_cmath.py|543 col 22| @unittest.skipIf(sysconfig.get_config_var('TANH_PRESERVES_ZERO_SIGN') == 0, "system tanh() function doesn't copy the sign")
Lib/test/test_math.py|978 col 22| @unittest.skipIf(sysconfig.get_config_var('TANH_PRESERVES_ZERO_SIGN') == 0, "system tanh() function doesn't copy the sign")

[xdegaye]

Are you generating the patch using "hg diff"? I don't see the base revision in your patch, and so there is no [Review] link on your patch.

I am using 'hg diff' with ~/.hgrc set to 'git = on' and this time, naively removed the '--git' from the output of 'hg diff' instead of commenting out 'git = on' in the config file :(
(not finding any reference explaining why this setting is wrong for Rietveld). I will change that to have 'git = on' always commented out.

The Python Developer’s Guide in section '16.1. Minimal Configuration' recommends setting 'git = on'. Since we are switching to git shortly, it is probably not very useful to enter a new issue to update the Guide explaining that this setting must not be used when producing patches to be reviewed by Rietveld because the base revision is missing in this case.

[berkerpeksag]

I am using 'hg diff' with ~/.hgrc set to 'git = on' and this time, naively removed the '--git' from the output of 'hg diff' instead of commenting out 'git = on' in the config file :(

You don't need to do that. I've been using the same setting [1] for 5 years without having a single problem. If you have a fresh clone of https://hg.python.org/cpython/ and using the 'default' branch, you're good.

[1]

[diff]
git = True
showfunc = True
unified = 8

[vstinner]

+ @unittest.skipIf(sysconfig.get_config_var('ANDROID_API_LEVEL') == 0,
+ "not an android platform")
+ def test_is_android(self):
+ # Use an heuristic, the shell on Android is at /system/bin/sh.
+ proc = subprocess.run(['/system/bin/sh', '-c', 'echo OK'],
+ stdout=subprocess.PIPE)
+ self.assertIn(b'OK', proc.stdout)

I dislike such functional test, it might fail and might need maintaince.

I suggest to push the change without the unit test.

[xdegaye]

@berker
This does not work for me.

I found an old discussion [1] on the git format problem with Rietveld. The thread diverges rapidly toward another subject, "submitting a branch instead of a patch to the issue tracker" but comes back to the original subject at the end. It seems that "not including the base changeset id in the --git diff is an oversight" of mercurial that has never been fixed.

[1] http://grokbase.com/t/python/python-dev/1135q4xxa8/hg-diff

[xdegaye]

I suggest to push the change without the unit test.
Agreed.

[skrah]

I did "hg pull -u" before re-generating the diff. Other than that, sometimes newer mercurial versions behave better (I have 2.8.2).

[vstinner]

bpo-27442-3.patch: be careful, the new item in Misc/NEWS is not at the top of its section. Moreover, I still consider that the unit test is risky and not really needed (just remove it).

Good job, there is now a [review] button :-)

[skrah]

Good point about Misc/NEWS: I just regenerated the patch mindlessly to check the "diff --git" situation :)

Actually, Xavier, it's often better to leave out the NEWS section in the bug tracker patches and just add it before pushing. Perhaps Rietveld choked on the NEWS file before because it's constantly changing?

[xdegaye]

Thanks Berker and Stefan for your help on 'hg diff --git'. Yes Stefan, maybe your patch has a [review] button because Rietveld found that your patch applied cleanly against the head of the default branch at the time you have submitted it, so it could guess the base revision while mines were submitted late, or rather without a 'hg pull -u' right before the submission ?

[skrah]

I guess so. Our Rietveld setup apparently has some heuristics that work best when you're on the default branch and completely synced with the main repo.

[python-dev]

New changeset 46567fda0b29 by Xavier de Gaye in branch 'default':
Issue bpo-27442: Expose the Android API level in sysconfig.get_config_vars()
https://hg.python.org/cpython/rev/46567fda0b29