[C API] Clean-up the Python.h header file

[vstinner]

BPO	45434
Nosy	@Yhg1s, @vstinner, @corona10
PRs	bpo-45434: Clean-up Python.h header file #28883 bpo-45434: Python.h no longer includes <stdlib.h> #28888 bpo-45434: Convert Py_GETENV() macro to a function #28912 bpo-45434: bytearrayobject.h no longer includes <stdarg.h> #28913 bpo-45434: pyport.h no longer includes <stdlib.h> #28914 bpo-45434: Move _Py_BEGIN_SUPPRESS_IPH to pycore_fileutils.h #28922 bpo-45434: Remove pystrhex.h header file #28923 bpo-45434: Mark the PyTokenizer C API as private #28924 bpo-45434: Limited Python.h no longer includes stdio.h #28960 bpo-45434: Remove useless space in includes #28963 bpo-45434: Remove Include/eval.h header file #28973 bpo-45434: Include stdlib.h for specialize stat #29015 bpo-45434: Only exclude <stdlib.h> in Python 3.11 limited C API #29027

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-10-19.15:01:04.891>
created_at = <Date 2021-10-11.20:08:05.746>
labels = ['expert-C-API', '3.11']
title = '[C API] Clean-up the Python.h header file'
updated_at = <Date 2021-10-19.15:01:04.890>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2021-10-19.15:01:04.890>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2021-10-19.15:01:04.891>
closer = 'vstinner'
components = ['C API']
creation = <Date 2021-10-11.20:08:05.746>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 45434
keywords = ['patch']
message_count = 19.0
messages = ['403675', '403684', '403689', '403775', '403785', '403829', '403831', '403841', '403842', '403843', '403859', '403940', '403959', '403963', '404002', '404162', '404170', '404172', '404283']
nosy_count = 3.0
nosy_names = ['twouters', 'vstinner', 'corona10']
pr_nums = ['28883', '28888', '28912', '28913', '28914', '28922', '28923', '28924', '28960', '28963', '28973', '29015', '29027']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue45434'
versions = ['Python 3.11']

[vstinner]

I would like to remove #include <stdlib.h> from Python.h, and make Python.h smaller.

[vstinner]

New changeset 47717d1 by Victor Stinner in branch 'main':
bpo-45434: Cleanup Python.h header file (GH-28883)
47717d1

[vstinner]

New changeset 03ea862 by Victor Stinner in branch 'main':
bpo-45434: Python.h no longer includes <stdlib.h> (GH-28888)
03ea862

[vstinner]

New changeset 489176e by Victor Stinner in branch 'main':
bpo-45434: Convert Py_GETENV() macro to a function (GH-28912)
489176e

[vstinner]

New changeset c63623a by Victor Stinner in branch 'main':
bpo-45434: bytearrayobject.h no longer includes <stdarg.h> (GH-28913)
c63623a

[vstinner]

New changeset 97308df by Victor Stinner in branch 'main':
bpo-45434: Move _Py_BEGIN_SUPPRESS_IPH to pycore_fileutils.h (GH-28922)
97308df

[vstinner]

New changeset bbe7497 by Victor Stinner in branch 'main':
bpo-45434: Remove pystrhex.h header file (GH-28923)
bbe7497

[vstinner]

Oh, Include/errcode.h *is* used by PyRun_InteractiveOneFlags() and its variants. It's mentioned in its documentation:

Returns ``0`` when the input was
executed successfully, ``-1`` if there was an exception, or an error code
from the :file:`errcode.h` include file distributed as part of Python if
there was a parse error. (Note that :file:`errcode.h` is not included by
:file:`Python.h`, so must be included specifically if needed.)

[vstinner]

New changeset 713bb19 by Victor Stinner in branch 'main':
bpo-45434: Mark the PyTokenizer C API as private (GH-28924)
713bb19

[vstinner]

PyRun_InteractiveOneObjectEx() returns a single error code of errcode.h: E_EOF (11):

    if (mod == NULL) {
        _PyArena_Free(arena);
        if (errcode == E_EOF) {
            PyErr_Clear();
            return E_EOF;
        }
        return -1;
    }

[vstinner]

New changeset aac29af by Victor Stinner in branch 'main':
bpo-45434: pyport.h no longer includes <stdlib.h> (GH-28914)
aac29af

[vstinner]

I tried to remove the <string.h> include from Python.h:

-#include <string.h> // memcpy()

... but it required me to modify 100+ files. I don't know the exact number, I gave up after 100 files. I was near half of C extensions built by setup.py, so the number total may be around 150 files which should include explicitly <string.h>.

"#include <errno.h>" can now be removed from Python.h, but I'm not sure if it's a good idea. I started by removing <stdlib.h> and my plan is to see if it breaks third party C extensions. Right now, Cython is broken which prevents me to use my https://github.com/vstinner/pythonci tool :-(

[vstinner]

New changeset 2849947 by Victor Stinner in branch 'main':
bpo-45434: Limited Python.h no longer includes stdio.h (GH-28960)
2849947

[vstinner]

New changeset 7076bef by Victor Stinner in branch 'main':
bpo-45434: Remove useless space in includes (GH-28963)
7076bef

[vstinner]

New changeset 105582e by Victor Stinner in branch 'main':
bpo-45434: Remove Include/eval.h header file (GH-28973)
105582e

[corona10]

New changeset fd03917 by Dong-hee Na in branch 'main':
bpo-45434: Include stdlib.h for specialize stat (GH-29015)
fd03917

[Yhg1s]

Victor, what's the benefit of doing this work? Are there real problems this fixes? I'm worried about the churn in third-party extensions, examples, tutorials, etc, especially for audiences that upon seeing a compiler error won't immediately realise they need to include stdlib.h themselves. (Also, since Python.h sets things like _POSIX_C_SOURCE and _XOPEN_SOURCE, including them in the wrong order can produce even more confusing errors, or errors that only appear on some platforms.)

[vstinner]

I reopen the issue.

[vstinner]

New changeset 52af075 by Victor Stinner in branch 'main':
bpo-45434: Only exclude <stdlib.h> in Python 3.11 limited C API (GH-29027)
52af075