Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better explain the GC contract for PyType_FromSpecWithBases #88429

Closed
pablogsal opened this issue May 29, 2021 · 14 comments
Closed

Better explain the GC contract for PyType_FromSpecWithBases #88429

pablogsal opened this issue May 29, 2021 · 14 comments
Labels
3.11 expert-C-API interpreter-core Interpreter core (Objects, Python, Grammar, and Parser dirs)

Comments

@pablogsal
Copy link
Member

pablogsal commented May 29, 2021

BPO 44263
Nosy @vstinner, @pablogsal, @miss-islington, @shihai1991, @erlend-aasland
PRs
  • bpo-44263: Better explain the GC contract for PyType_FromSpecWithBases #26442
  • [3.10] bpo-44263: Better explain the GC contract for PyType_FromSpecWithBases (GH-26442) #26443
  • [3.9] bpo-44263: Better explain the GC contract for PyType_FromSpecWithBases (GH-26442) #26444
  • bpo-44263: Mention PyType_Ready in the gc protocol warning #26445
  • [3.10] bpo-44263: Mention PyType_Ready in the gc protocol warning (GH-26445) #26446
  • [3.9] bpo-44263: Mention PyType_Ready in the gc protocol warning (GH-26445) #26447
  • bpo-44263: Py_TPFLAGS_HAVE_GC requires tp_traverse #26463
  • bpo-44263: Fix _decimal and _testcapi GC protocol #26464
  • [3.10] bpo-44263: Fix _decimal and _testcapi GC protocol (GH-26464) #26465
  • [3.9] bpo-44263: Fix _decimal and _testcapi GC protocol (GH-26464) #26466
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-06-01.21:39:22.556>
    created_at = <Date 2021-05-29.00:00:31.172>
    labels = ['interpreter-core', 'expert-C-API', '3.11']
    title = 'Better explain the GC contract for PyType_FromSpecWithBases'
    updated_at = <Date 2021-06-01.21:39:22.553>
    user = 'https://github.com/pablogsal'

    bugs.python.org fields:

    activity = <Date 2021-06-01.21:39:22.553>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-06-01.21:39:22.556>
    closer = 'vstinner'
    components = ['Interpreter Core', 'C API']
    creation = <Date 2021-05-29.00:00:31.172>
    creator = 'pablogsal'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 44263
    keywords = ['patch']
    message_count = 13.0
    messages = ['394706', '394708', '394709', '394710', '394711', '394712', '394733', '394799', '394800', '394802', '394803', '394872', '394873']
    nosy_count = 5.0
    nosy_names = ['vstinner', 'pablogsal', 'miss-islington', 'shihai1991', 'erlendaasland']
    pr_nums = ['26442', '26443', '26444', '26445', '26446', '26447', '26463', '26464', '26465', '26466']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue44263'
    versions = ['Python 3.11']

    @pablogsal
    Copy link
    Member Author

    pablogsal commented May 29, 2021

    We currently don't properly document how PyType_FromSpecWithBases behaves if the class is inheriting from some other class with GC support.

    @pablogsal
    Copy link
    Member Author

    pablogsal commented May 29, 2021

    New changeset 8b55bc3 by Pablo Galindo in branch 'main':
    bpo-44263: Better explain the GC contract for PyType_FromSpecWithBases (GH-26442)
    8b55bc3

    @miss-islington
    Copy link
    Contributor

    miss-islington commented May 29, 2021

    New changeset a30cbae by Miss Islington (bot) in branch '3.9':
    bpo-44263: Better explain the GC contract for PyType_FromSpecWithBases (GH-26442)
    a30cbae

    @pablogsal
    Copy link
    Member Author

    pablogsal commented May 29, 2021

    New changeset 11049be by Miss Islington (bot) in branch '3.10':
    bpo-44263: Better explain the GC contract for PyType_FromSpecWithBases (GH-26442) (GH-26443)
    11049be

    @pablogsal
    Copy link
    Member Author

    pablogsal commented May 29, 2021

    New changeset 43cf7c8 by Pablo Galindo in branch 'main':
    bpo-44263: Mention PyType_Ready in the gc protocol warning (GH-26445)
    43cf7c8

    @miss-islington
    Copy link
    Contributor

    miss-islington commented May 29, 2021

    New changeset 7bd1304 by Miss Islington (bot) in branch '3.9':
    bpo-44263: Mention PyType_Ready in the gc protocol warning (GH-26445)
    7bd1304

    @pablogsal
    Copy link
    Member Author

    pablogsal commented May 29, 2021

    New changeset d8ce746 by Miss Islington (bot) in branch '3.10':
    bpo-44263: Mention PyType_Ready in the gc protocol warning (GH-26445) (bpo-26446)
    d8ce746

    @vstinner
    Copy link
    Member

    vstinner commented May 31, 2021

    I reopen the issue. I would prefer to change PyType_FromSpecWithBases() to raise an error if a type has the Py_TPFLAGS_HAVE_GC flag but its tp_traverse function is NULL.

    Oops, the Python stdlib has more types than only _ssl.SSLError with this error:

    • _decimal: PyDecSignalDictMixin_Type (internal type)
    • _testcapi: heapctype

    @vstinner vstinner reopened this May 31, 2021
    @vstinner vstinner reopened this May 31, 2021
    @vstinner
    Copy link
    Member

    vstinner commented May 31, 2021

    New changeset 142e5c5 by Victor Stinner in branch 'main':
    bpo-44263: Fix _decimal and _testcapi GC protocol (GH-26464)
    142e5c5

    @pablogsal
    Copy link
    Member Author

    pablogsal commented May 31, 2021

    New changeset 7fe9cad by Miss Islington (bot) in branch '3.10':
    bpo-44263: Fix _decimal and _testcapi GC protocol (GH-26464) (GH-26465)
    7fe9cad

    @miss-islington
    Copy link
    Contributor

    miss-islington commented May 31, 2021

    New changeset 46b16d0 by Miss Islington (bot) in branch '3.9':
    bpo-44263: Fix _decimal and _testcapi GC protocol (GH-26464)
    46b16d0

    @vstinner
    Copy link
    Member

    vstinner commented Jun 1, 2021

    New changeset ee76375 by Victor Stinner in branch 'main':
    bpo-44263: Py_TPFLAGS_HAVE_GC requires tp_traverse (GH-26463)
    ee76375

    @vstinner
    Copy link
    Member

    vstinner commented Jun 1, 2021

    I made sure that it's no longer possible to create a type with Py_TPFLAGS_HAVE_GC flag set but with no traverse function (tp_traverse=NULL). I close again the issue.

    @vstinner vstinner added interpreter-core Interpreter core (Objects, Python, Grammar, and Parser dirs) expert-C-API 3.11 labels Jun 1, 2021
    @vstinner vstinner closed this as completed Jun 1, 2021
    @vstinner vstinner added interpreter-core Interpreter core (Objects, Python, Grammar, and Parser dirs) expert-C-API 3.11 labels Jun 1, 2021
    @vstinner vstinner closed this as completed Jun 1, 2021
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    stefanseefeld pushed a commit to boostorg/python that referenced this issue Apr 26, 2022
    The enum_type_object type inherits from PyLong_Type which is not tracked
    by the GC. Instances doesn't have to be tracked by the GC: remove the
    Py_TPFLAGS_HAVE_GC flag.
    
    The Python C API documentation says:
    
        "To create a container type, the tp_flags field of the type object
        must include the Py_TPFLAGS_HAVE_GC and provide an implementation of
        the tp_traverse handler."
    
    https://docs.python.org/dev/c-api/gcsupport.html
    
    The new exception was introduced in Python 3.11 by:
    python/cpython#88429
    @vstinner
    Copy link
    Member

    vstinner commented May 2, 2022

    Example of bug in the wild: Boost Python enum type defined Py_TPFLAGS_HAVE_GC flag but has no traverse function and instances doesn't have to be tracked by the GC:

    Note: it's an enum such an "enum" in C, like a Python int (1, 2, ...). It's unrelated to Python enum module.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 expert-C-API interpreter-core Interpreter core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants