Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move PyArena C API to the internal C API #87410

Closed
vstinner opened this issue Feb 17, 2021 · 19 comments
Closed

Move PyArena C API to the internal C API #87410

vstinner opened this issue Feb 17, 2021 · 19 comments

Comments

@vstinner
Copy link
Member

@vstinner vstinner commented Feb 17, 2021

BPO 43244
Nosy @vstinner, @shihai1991
PRs
  • #24688
  • #24907
  • #24908
  • #24910
  • #24911
  • #24912
  • #24913
  • #24914
  • #24933
  • #25000
  • #25001
  • #25007
  • #25242
  • #25243
  • #25252
  • #25256
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-03-24.01:26:54.289>
    created_at = <Date 2021-02-17.18:31:30.182>
    labels = ['expert-C-API', '3.10']
    title = 'Move PyArena C API to the internal C API'
    updated_at = <Date 2021-05-26.22:44:01.883>
    user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2021-05-26.22:44:01.883>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-03-24.01:26:54.289>
    closer = 'vstinner'
    components = ['C API']
    creation = <Date 2021-02-17.18:31:30.182>
    creator = 'vstinner'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 43244
    keywords = ['patch']
    message_count = 19.0
    messages = ['387180', '388973', '388976', '388981', '389001', '389016', '389018', '389075', '389408', '389416', '389419', '389423', '389424', '389501', '390416', '390444', '390472', '390473', '394489']
    nosy_count = 2.0
    nosy_names = ['vstinner', 'shihai1991']
    pr_nums = ['24688', '24907', '24908', '24910', '24911', '24912', '24913', '24914', '24933', '25000', '25001', '25007', '25242', '25243', '25252', '25256']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue43244'
    versions = ['Python 3.10']

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Feb 17, 2021

    Include/cpython/pyarena.h declares 4 functions which are implemented in Python/pyarena.c. The header file exists to access these functions from the compiler. IMO there was no intent to expose these functions to the public Python C API.

    Moreover, the PEP-384 explicitly excludes this API from the stable ABI (from the limited C API).

    I propose to move these functions to the internal C API.

    Since these functions are no even documented, I don't think that a deprecation period is needed.

    Note: The PR 24550 moved Include/pyarena.h to Include/cpython/pyarena.h.

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Mar 17, 2021

    New changeset b4536e1 by Victor Stinner in branch 'master':
    bpo-43244: Rename pycore_ast.h to pycore_ast_state.h (GH-24907)
    b4536e1

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Mar 17, 2021

    New changeset 526fdeb by Victor Stinner in branch 'master':
    bpo-43244: Add pycore_ast.h header file (GH-24908)
    526fdeb

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Mar 18, 2021

    New changeset e0bf70d by Victor Stinner in branch 'master':
    bpo-43244: Fix test_peg_generator for PyAST_Validate() (GH-24912)
    e0bf70d

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Mar 18, 2021

    New changeset 6af528b by Victor Stinner in branch 'master':
    bpo-43244: Fix test_peg_generators on Windows (GH-24913)
    6af528b

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Mar 18, 2021

    New changeset eec8e61 by Victor Stinner in branch 'master':
    bpo-43244: Remove the PyAST_Validate() function (GH-24911)
    eec8e61

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Mar 18, 2021

    The work on this issue started in Python 3.9 with bpo-21120 which excluded Python-ast.h, ast.h and asdl.h from the limited C API: commit 421a72a.

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Mar 19, 2021

    New changeset 28ad12f by Victor Stinner in branch 'master':
    bpo-43244: Remove symtable.h header file (GH-24910)
    28ad12f

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Mar 23, 2021

    New changeset 94faa07 by Victor Stinner in branch 'master':
    bpo-43244: Remove ast.h, asdl.h, Python-ast.h headers (GH-24933)
    94faa07

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Mar 23, 2021

    New changeset a81fca6 by Victor Stinner in branch 'master':
    bpo-43244: Add pycore_compile.h header file (GH-25000)
    a81fca6

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Mar 24, 2021

    New changeset 57364ce by Victor Stinner in branch 'master':
    bpo-43244: Remove parser_interface.h header file (GH-25001)
    57364ce

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Mar 24, 2021

    New changeset 8370e07 by Victor Stinner in branch 'master':
    bpo-43244: Remove the pyarena.h header (GH-25007)
    8370e07

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Mar 24, 2021

    Ok, the whole C API related to parser, compiler, AST, ASDL and PyArena moved to the internal C API. So we will have more freedom to evolve it!

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Mar 25, 2021

    In short, this issue is a follow-up of bpo-40939.

    These C API removal are related to the final step of the PEP-617 "New PEG parser for CPython": removal of the PyParser C API.

    • The PyParser C API produced "node*" objects, like PyParser_SimpleParseFile(FILE*) => node*. This C API was removed in Python 3.10: bpo-40939 "Remove the old parser".

    • The AST C API takes "node*" as input: PyAST_FromNode(node*) => mod_ty. I removed these APIs.

    • symtable takes "mod_ty" as input.

    • ASDL and PyArena API were only useful for the removed AST C APIs.

    I searched for removed functions in the top PyPI 4000 projects, there is a single project on 4000: typed_ast. This typed_ast project is special, it copies directly code from CPython. Even after the C API removed, typed_ast can continue to use the internal C API, it only has to define the Py_BUILD_CORE_MODULE macro.

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Apr 7, 2021

    New changeset d36d6a9 by Victor Stinner in branch 'master':
    bpo-43244: Remove Yield macro from pycore_ast.h (GH-25243)
    d36d6a9

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Apr 7, 2021

    typed-ast was updated for these changes and to avoid including explicitly pydebug.h (it's included by Python.h: see bpo-35134 change): python/typed_ast#158

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Apr 7, 2021

    New changeset d27f8d2 by Victor Stinner in branch 'master':
    bpo-43244: Rename pycore_ast.h functions to _PyAST_xxx() (GH-25252)
    d27f8d2

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented Apr 7, 2021

    "Remove Yield macro from pycore_ast.h"

    I'm happy that this one is gone, it was a common and annoying compiler warning on Windows!

    @vstinner
    Copy link
    Member Author

    @vstinner vstinner commented May 26, 2021

    I marked bpo-35197 as a duplicate of this issue.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant