Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[C API] Clean-up the Python.h header file #89597

Closed
vstinner opened this issue Oct 11, 2021 · 19 comments
Closed

[C API] Clean-up the Python.h header file #89597

vstinner opened this issue Oct 11, 2021 · 19 comments

Comments

@vstinner
Copy link
Member

vstinner commented Oct 11, 2021

BPO 45434
Nosy @Yhg1s, @vstinner, @corona10
PRs
  • bpo-45434: Clean-up Python.h header file #28883
  • bpo-45434: Python.h no longer includes <stdlib.h> #28888
  • bpo-45434: Convert Py_GETENV() macro to a function #28912
  • bpo-45434: bytearrayobject.h no longer includes <stdarg.h> #28913
  • bpo-45434: pyport.h no longer includes <stdlib.h> #28914
  • bpo-45434: Move _Py_BEGIN_SUPPRESS_IPH to pycore_fileutils.h #28922
  • bpo-45434: Remove pystrhex.h header file #28923
  • bpo-45434: Mark the PyTokenizer C API as private #28924
  • bpo-45434: Limited Python.h no longer includes stdio.h #28960
  • bpo-45434: Remove useless space in includes #28963
  • bpo-45434: Remove Include/eval.h header file #28973
  • bpo-45434: Include stdlib.h for specialize stat #29015
  • bpo-45434: Only exclude <stdlib.h> in Python 3.11 limited C API #29027
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-10-19.15:01:04.891>
    created_at = <Date 2021-10-11.20:08:05.746>
    labels = ['expert-C-API', '3.11']
    title = '[C API] Clean-up the Python.h header file'
    updated_at = <Date 2021-10-19.15:01:04.890>
    user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2021-10-19.15:01:04.890>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-10-19.15:01:04.891>
    closer = 'vstinner'
    components = ['C API']
    creation = <Date 2021-10-11.20:08:05.746>
    creator = 'vstinner'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 45434
    keywords = ['patch']
    message_count = 19.0
    messages = ['403675', '403684', '403689', '403775', '403785', '403829', '403831', '403841', '403842', '403843', '403859', '403940', '403959', '403963', '404002', '404162', '404170', '404172', '404283']
    nosy_count = 3.0
    nosy_names = ['twouters', 'vstinner', 'corona10']
    pr_nums = ['28883', '28888', '28912', '28913', '28914', '28922', '28923', '28924', '28960', '28963', '28973', '29015', '29027']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue45434'
    versions = ['Python 3.11']

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 11, 2021

    I would like to remove #include <stdlib.h> from Python.h, and make Python.h smaller.

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 11, 2021

    New changeset 47717d1 by Victor Stinner in branch 'main':
    bpo-45434: Cleanup Python.h header file (GH-28883)
    47717d1

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 11, 2021

    New changeset 03ea862 by Victor Stinner in branch 'main':
    bpo-45434: Python.h no longer includes <stdlib.h> (GH-28888)
    03ea862

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 13, 2021

    New changeset 489176e by Victor Stinner in branch 'main':
    bpo-45434: Convert Py_GETENV() macro to a function (GH-28912)
    489176e

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 13, 2021

    New changeset c63623a by Victor Stinner in branch 'main':
    bpo-45434: bytearrayobject.h no longer includes <stdarg.h> (GH-28913)
    c63623a

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 13, 2021

    New changeset 97308df by Victor Stinner in branch 'main':
    bpo-45434: Move _Py_BEGIN_SUPPRESS_IPH to pycore_fileutils.h (GH-28922)
    97308df

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 13, 2021

    New changeset bbe7497 by Victor Stinner in branch 'main':
    bpo-45434: Remove pystrhex.h header file (GH-28923)
    bbe7497

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 13, 2021

    Oh, Include/errcode.h *is* used by PyRun_InteractiveOneFlags() and its variants. It's mentioned in its documentation:

    Returns ``0`` when the input was
    executed successfully, ``-1`` if there was an exception, or an error code
    from the :file:`errcode.h` include file distributed as part of Python if
    there was a parse error. (Note that :file:`errcode.h` is not included by
    :file:`Python.h`, so must be included specifically if needed.)

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 13, 2021

    New changeset 713bb19 by Victor Stinner in branch 'main':
    bpo-45434: Mark the PyTokenizer C API as private (GH-28924)
    713bb19

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 13, 2021

    PyRun_InteractiveOneObjectEx() returns a single error code of errcode.h: E_EOF (11):

        if (mod == NULL) {
            _PyArena_Free(arena);
            if (errcode == E_EOF) {
                PyErr_Clear();
                return E_EOF;
            }
            return -1;
        }

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 13, 2021

    New changeset aac29af by Victor Stinner in branch 'main':
    bpo-45434: pyport.h no longer includes <stdlib.h> (GH-28914)
    aac29af

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 14, 2021

    I tried to remove the <string.h> include from Python.h:

    -#include <string.h> // memcpy()

    ... but it required me to modify 100+ files. I don't know the exact number, I gave up after 100 files. I was near half of C extensions built by setup.py, so the number total may be around 150 files which should include explicitly <string.h>.

    "#include <errno.h>" can now be removed from Python.h, but I'm not sure if it's a good idea. I started by removing <stdlib.h> and my plan is to see if it breaks third party C extensions. Right now, Cython is broken which prevents me to use my https://github.com/vstinner/pythonci tool :-(

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 14, 2021

    New changeset 2849947 by Victor Stinner in branch 'main':
    bpo-45434: Limited Python.h no longer includes stdio.h (GH-28960)
    2849947

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 14, 2021

    New changeset 7076bef by Victor Stinner in branch 'main':
    bpo-45434: Remove useless space in includes (GH-28963)
    7076bef

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 15, 2021

    New changeset 105582e by Victor Stinner in branch 'main':
    bpo-45434: Remove Include/eval.h header file (GH-28973)
    105582e

    @corona10
    Copy link
    Member

    corona10 commented Oct 18, 2021

    New changeset fd03917 by Dong-hee Na in branch 'main':
    bpo-45434: Include stdlib.h for specialize stat (GH-29015)
    fd03917

    @Yhg1s
    Copy link
    Member

    Yhg1s commented Oct 18, 2021

    Victor, what's the benefit of doing this work? Are there real problems this fixes? I'm worried about the churn in third-party extensions, examples, tutorials, etc, especially for audiences that upon seeing a compiler error won't immediately realise they need to include stdlib.h themselves. (Also, since Python.h sets things like _POSIX_C_SOURCE and _XOPEN_SOURCE, including them in the wrong order can produce even more confusing errors, or errors that only appear on some platforms.)

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 18, 2021

    I reopen the issue.

    @vstinner vstinner reopened this Oct 18, 2021
    @vstinner vstinner reopened this Oct 18, 2021
    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 19, 2021

    New changeset 52af075 by Victor Stinner in branch 'main':
    bpo-45434: Only exclude <stdlib.h> in Python 3.11 limited C API (GH-29027)
    52af075

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants