Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

expose the Android API level in sysconfig.get_config_vars() #71629

Closed
xdegaye mannequin opened this issue Jul 2, 2016 · 29 comments
Closed

expose the Android API level in sysconfig.get_config_vars() #71629

xdegaye mannequin opened this issue Jul 2, 2016 · 29 comments
Assignees
Labels
stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@xdegaye
Copy link
Mannequin

xdegaye mannequin commented Jul 2, 2016

BPO 27442
Nosy @vstinner, @ned-deily, @skrah, @xdegaye, @berkerpeksag, @yan12125
Files
  • android_api.patch
  • android_api_2.patch
  • android_api_3.patch
  • issue27442-3.patch: regenerated android_api_3.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/xdegaye'
    closed_at = <Date 2016-07-09.09:09:29.464>
    created_at = <Date 2016-07-02.14:52:55.319>
    labels = ['type-feature', 'library']
    title = 'expose the Android API level in sysconfig.get_config_vars()'
    updated_at = <Date 2016-07-09.09:09:29.463>
    user = 'https://github.com/xdegaye'

    bugs.python.org fields:

    activity = <Date 2016-07-09.09:09:29.463>
    actor = 'xdegaye'
    assignee = 'xdegaye'
    closed = True
    closed_date = <Date 2016-07-09.09:09:29.464>
    closer = 'xdegaye'
    components = ['Library (Lib)']
    creation = <Date 2016-07-02.14:52:55.319>
    creator = 'xdegaye'
    dependencies = []
    files = ['43610', '43622', '43636', '43660']
    hgrepos = []
    issue_num = 27442
    keywords = ['patch']
    message_count = 29.0
    messages = ['269716', '269717', '269719', '269720', '269721', '269724', '269726', '269748', '269756', '269757', '269807', '269828', '269829', '269832', '269834', '269878', '269902', '269937', '269938', '269954', '269958', '269962', '269963', '269980', '269981', '269982', '270008', '270013', '270031']
    nosy_count = 7.0
    nosy_names = ['vstinner', 'ned.deily', 'skrah', 'xdegaye', 'python-dev', 'berker.peksag', 'yan12125']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue27442'
    versions = ['Python 3.6']

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 2, 2016

    Expose the Android API level that python was built against, in sys.implementation as _android_api.
    Purposes:
    * Identify the platform as Android.
    * Allow detecting a mismatch with the runtime sdk version returned by platform.android_ver() (bpo-26855), for example when the runtime sdk is lower than the built API level.

    @xdegaye xdegaye mannequin added the interpreter-core Interpreter core (Objects, Python, Grammar, and Parser dirs) label Jul 2, 2016
    @xdegaye xdegaye mannequin self-assigned this Jul 2, 2016
    @xdegaye xdegaye mannequin added the type-feature A feature request or enhancement label Jul 2, 2016
    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 2, 2016

    Patch added.

    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented Jul 2, 2016

    I don't think sys.implementation is a good place to contain the information of the underlying platform. By Doc/sys.rst:

    An object containing information about the implementation of the currently running Python interpreter.

    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented Jul 2, 2016

    Sorry, it's Doc/library/sys.rst

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 2, 2016

    I don't think sys.implementation is a good place to contain the information of the underlying platform.

    Quite the opposite, the ndk API level gives an information about the implementation of the currently running Python interpreter saying that this Python has been built against this version of Android libc identified by this API level.

    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented Jul 2, 2016

    OK I see the rationale.

    @vstinner
    Copy link
    Member

    vstinner commented Jul 2, 2016

    I woupd prefer to have a public attribute.

    @ned-deily
    Copy link
    Member

    ned-deily commented Jul 3, 2016

    Typically, for other sorts of build configuration data, we have relied on extracting that from the ./configure-produced Makefile and making it available via sysconfig.get_config_var(). I think we should be cautious about bloating sys.implementation with platform-specific data unless there is an overriding need for it, for example, if it is needed during interpreter initialization before sysconfig can be initialized. If not, I'd look at adding the needed values as configuration variables in configure.ac.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 3, 2016

    @ned
    The information that the interpreter is an Android built, is needed in bpo-16353 [1][2]. In msg266135, the corresponding patch adds the 'is_android' attribute to the already bloated sys module. I had thought then about using sysconfig.get_config_var() as you are suggesting, but this means that the os module would need now to import sysconfig on all linux platforms and build this big dictionary (about 650 entries on linux).

    So it seems that the choice is between:
    (1) adding an attribute (is_android or android_api) to the bloated sys module
    (2) bloating sys.implementation with platform-specific data

    I think that (2) would be better since _android_api is an information about the implementation, it tells that in these binaries, such and such features are available or not (bpo-26857 for example). The drawback is that it would not be documented except in Misc/NEWS [2].

    @victor
    Do you mean a public attribute of sys.implementation ?
    Adding a new sys.implementation required attribute is described in PEP-421. In that case the attribute name could be 'libc_version' or 'libc'.

    [1] msg175006 suggested another approach though, but this seems to be abandoned.
    [2] For completeness, Stefan has submitted a feature request to google in msg266089.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 3, 2016

    Well, bpo-16353 has been entered upon attempting to fix bpo-16255 (see msg173477).

    So an alternative exists. bpo-16353 could be closed as 'wont't fix'. The list of locations where '/bin/sh' is hard coded in the standard library in msg266084 shows that only the subprocess module and the test suite need to know the location of the system shell. So the subprocess module and bpo-27027 could deduct the location of this shell via sysconfig.get_config_var('android_api').

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 4, 2016

    A patch to expose the Android API level in sysconfig.get_config_vars().

    @xdegaye xdegaye mannequin added stdlib Python modules in the Lib dir and removed interpreter-core Interpreter core (Objects, Python, Grammar, and Parser dirs) labels Jul 4, 2016
    @xdegaye xdegaye mannequin changed the title expose Android API level in sys.implementation expose the Android API level in sysconfig.get_config_vars() Jul 4, 2016
    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 5, 2016

    According to bpo-27453, do this minor change in the patch: s/$CC -E/$CPP $CPPFLAGS.

    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented Jul 5, 2016

    Here's an issue - there's already a macro called ANDROID_API defined in libcutils [1] If someone is going to integrate Python into AOSP, redefining macros may cause a problem.

    [1] https://android.googlesource.com/platform/system/core/+/master/include/cutils/compiler.h#42

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 5, 2016

    Integrating Python into AOSP does not make sense. The patch can be changed with s/ANDROID_API/ANDROID_API_LEVEL.

    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented Jul 5, 2016

    Yep adding Python to Android's build system is a rare case. Just to mention there's already an macro and avoiding possible redefined macros is always good.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 6, 2016

    New patch incorporating the substitutions to '$CPP $CPPFLAGS' and ANDROID_API_LEVEL.

    @vstinner
    Copy link
    Member

    vstinner commented Jul 6, 2016

    + @unittest.skipIf(sysconfig.get_config_var('ANDROID_API_LEVEL') == 0,
    + "not an android platform")

    Hum, sysconfig.get_config_var() returns None for unknown variable. Why checking ==0?

    @xavier: Are you generating the patch using "hg diff"? I don't see the base revision in your patch, and so there is no [Review] link on your patch.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 7, 2016

    In sysconfig.parse_config_h(), the variables in pyconfig.h that have a commented-out '#undef' line are set to 0. Fortunately, there is no Android API level 0.

    Checking '== 0' ensures that autoreconf has been run to add '#undef ANDROID_API_LEVEL' to pyconfig.h.in. If this autoreconf step were to be missed, the test would (correctly) fail on the buildbots that are not Android as get_config_var() would return None then and the test would not be skipped and fail.

    Most of the tests in the Python test suite do check 'not sysconfig.get_config_var()' instead, except:
    Lib/test/test_cmath.py|543 col 22| @unittest.skipIf(sysconfig.get_config_var('TANH_PRESERVES_ZERO_SIGN') == 0, "system tanh() function doesn't copy the sign")
    Lib/test/test_math.py|978 col 22| @unittest.skipIf(sysconfig.get_config_var('TANH_PRESERVES_ZERO_SIGN') == 0, "system tanh() function doesn't copy the sign")

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 7, 2016

    Are you generating the patch using "hg diff"? I don't see the base revision in your patch, and so there is no [Review] link on your patch.

    I am using 'hg diff' with ~/.hgrc set to 'git = on' and this time, naively removed the '--git' from the output of 'hg diff' instead of commenting out 'git = on' in the config file :(
    (not finding any reference explaining why this setting is wrong for Rietveld). I will change that to have 'git = on' always commented out.

    The Python Developer’s Guide in section '16.1. Minimal Configuration' recommends setting 'git = on'. Since we are switching to git shortly, it is probably not very useful to enter a new issue to update the Guide explaining that this setting must not be used when producing patches to be reviewed by Rietveld because the base revision is missing in this case.

    @berkerpeksag
    Copy link
    Member

    berkerpeksag commented Jul 7, 2016

    I am using 'hg diff' with ~/.hgrc set to 'git = on' and this time, naively removed the '--git' from the output of 'hg diff' instead of commenting out 'git = on' in the config file :(

    You don't need to do that. I've been using the same setting [1] for 5 years without having a single problem. If you have a fresh clone of https://hg.python.org/cpython/ and using the 'default' branch, you're good.

    [1]

    [diff]
    git = True
    showfunc = True
    unified = 8

    @vstinner
    Copy link
    Member

    vstinner commented Jul 7, 2016

    + @unittest.skipIf(sysconfig.get_config_var('ANDROID_API_LEVEL') == 0,
    + "not an android platform")
    + def test_is_android(self):
    + # Use an heuristic, the shell on Android is at /system/bin/sh.
    + proc = subprocess.run(['/system/bin/sh', '-c', 'echo OK'],
    + stdout=subprocess.PIPE)
    + self.assertIn(b'OK', proc.stdout)

    I dislike such functional test, it might fail and might need maintaince.

    I suggest to push the change without the unit test.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 7, 2016

    @berker
    This does not work for me.

    I found an old discussion [1] on the git format problem with Rietveld. The thread diverges rapidly toward another subject, "submitting a branch instead of a patch to the issue tracker" but comes back to the original subject at the end. It seems that "not including the base changeset id in the --git diff is an oversight" of mercurial that has never been fixed.

    [1] http://grokbase.com/t/python/python-dev/1135q4xxa8/hg-diff

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 7, 2016

    I suggest to push the change without the unit test.
    Agreed.

    @skrah
    Copy link
    Mannequin

    skrah mannequin commented Jul 8, 2016

    I did "hg pull -u" before re-generating the diff. Other than that, sometimes newer mercurial versions behave better (I have 2.8.2).

    @vstinner
    Copy link
    Member

    vstinner commented Jul 8, 2016

    bpo-27442-3.patch: be careful, the new item in Misc/NEWS is not at the top of its section. Moreover, I still consider that the unit test is risky and not really needed (just remove it).

    Good job, there is now a [review] button :-)

    @skrah
    Copy link
    Mannequin

    skrah mannequin commented Jul 8, 2016

    Good point about Misc/NEWS: I just regenerated the patch mindlessly to check the "diff --git" situation :)

    Actually, Xavier, it's often better to leave out the NEWS section in the bug tracker patches and just add it before pushing. Perhaps Rietveld choked on the NEWS file before because it's constantly changing?

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 8, 2016

    Thanks Berker and Stefan for your help on 'hg diff --git'. Yes Stefan, maybe your patch has a [review] button because Rietveld found that your patch applied cleanly against the head of the default branch at the time you have submitted it, so it could guess the base revision while mines were submitted late, or rather without a 'hg pull -u' right before the submission ?

    @skrah
    Copy link
    Mannequin

    skrah mannequin commented Jul 8, 2016

    I guess so. Our Rietveld setup apparently has some heuristics that work best when you're on the default branch and completely synced with the main repo.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jul 9, 2016

    New changeset 46567fda0b29 by Xavier de Gaye in branch 'default':
    Issue bpo-27442: Expose the Android API level in sysconfig.get_config_vars()
    https://hg.python.org/cpython/rev/46567fda0b29

    @xdegaye xdegaye mannequin closed this as completed Jul 9, 2016
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants