Rework PEP489: multi-phase module initialization, heap based types #301
Conversation
Sort the constant string decls from the ones to be moved to module state.
Include methods for initializing, traversing, and clearing it. Tidy up module method defs w/ constant docstrings.
Use named members for clarity.
Tidy spelings for pointer vars.
- Store created objects in the module state first, then copy them where they are needed (in the future, into the module's __dict__ so that Python can use them). - At this point, the module and the types are still all static. - Use 'PyModule_AddType' as a clearer statement of intent. We are going to be creating the types on the heap later, and so will have their addresses to hand.
- Stores imported values and flag on the module state struct, rather than in global statics.
Drop statics previously used to call 'PyObject_GetAttr'.
The 'PyModule_AddType' API was added in that release.
And remove the static used to call 'PyDict_GetItem'.
Note that those functions are still ignoring that value: this is prep for the next step, where they will use the '_zic_module_state' machinery
Leave forward decls behind FBO type methods. This step is in preparation for using the '_zic_module_state', rather than the global statics.
This is a stopgap, for the statically-initialized module. It will be replaced by 'PyType_GetModuleByDef' once we have multi-phase init and heap types in place.
Instead, look them up via module state versions.
Look it up instead from module state via type with a new helper, '_get_adapter_hooks'.
Move actual explanatory comment for '_get_adapter_hooks' to its impl.
Look it up instead from from module state via the type with a new helper, '_get_specification_base_class'.
At least one bug remains (some interfaces return a 'member_descriptor' for their '__module__').
Drop vestiges of manual 'weakreflist' management. Rename type structs and methods for consistency.
Make them conditional, along with the manual weakreflist managerment, on Python < 3.12, which added the 'Py_TPFLAGS_MANAGED_WEAKREF' flag. Builds and tests pass on 3.11 with this change.
|
@tseaver Thank you for this PR even though it is way beyond my understanding of C code. |
Nope, the current changes work on 3.11+. I've stubbed my toe a couple of times trying to work out how to solve the problems with the older versions' C APIs, but have given up for the moment. The motivation is laying groundwork for using I should probably mark the PR as a draft. |
This PR is a reworking of PR #297, with a better final outcome: the test suite passes on Python 3.11, 3.12, and 3.13!
Unfortunately, the dance required to juggle module state between dynamically-allocated modules and heap-allocated extension types requires APIs which are too new for all our supported Python versions.
Assuming we have no big plans for the C module, this branch could likely sit around for a bit. Or we could choose to merge it as part of a major version (
7.0?), dropping support for the older versions all at once, perhaps with a "security fixes"6.xmaintenance branch. I haven't yet played with running it under sub-interpreters or the no-gil interpreter, but will likely do some experimentation with them later.