Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow C extensions to declare compatibility with free-threading #116322

Closed
Tracked by #108219
swtaarrs opened this issue Mar 4, 2024 · 3 comments
Closed
Tracked by #108219

Allow C extensions to declare compatibility with free-threading #116322

swtaarrs opened this issue Mar 4, 2024 · 3 comments
Assignees
@swtaarrs
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-free-threading type-feature A feature request or enhancement

Comments

@swtaarrs
Copy link
Member

swtaarrs commented Mar 4, 2024 •
edited by bedevere-app bot

Feature or enhancement

There are a few pieces to this:

  1. Add a Py_mod_gil slot, as described in PEP 703, that multi-phase init modules can use to indicate that they support free-threading.
  2. Add a PyModule_ExperimentalSetGIL() function (discussed here) that single-phase init modules can use in place of Py_mod_gil.
  3. Mark all built-in modules as free-threading compatible with one of the above mechanisms.
  4. Enable the GIL while loading a C module and leave it permanently enabled if the module does not declare free-threading compatibility.

1-3 are addressed in gh-116882; 4 will be addressed in a separate PR.

Linked PRs
@swtaarrs swtaarrs added type-feature A feature request or enhancement interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-free-threading labels Mar 4, 2024
This was referenced Mar 4, 2024
Open
Merged
@swtaarrs swtaarrs self-assigned this Mar 11, 2024
@swtaarrs
Copy link
Member Author

What warning category should this issue? ImportWarning sounds like a good fit at first, but it's ignored by the default warning filters, which we don't want. Of the categories not ignored by default, RuntimeWarning seems like the best fit to me.

This was referenced Mar 13, 2024
Open
Merged
@swtaarrs swtaarrs mentioned this issue Apr 3, 2024
Closed
@swtaarrs swtaarrs changed the title Implement Py_mod_gil slot for free-threading-safe extensions Allow C extensions to declare compatibility with free-threading May 2, 2024
colesbury pushed a commit that referenced this issue May 3, 2024
@swtaarrs
gh-116322: Add Py_mod_gil module slot (#116882)
c2627d6 
This PR adds the ability to enable the GIL if it was disabled at
interpreter startup, and modifies the multi-phase module initialization
path to enable the GIL when loading a module, unless that module's spec
includes a slot indicating it can run safely without the GIL.

PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went
with `Py_MOD_GIL_NOT_USED` for consistency with gh-104148.

A warning will be issued up to once per interpreter for the first
GIL-using module that is loaded. If `-v` is given, a shorter message
will be printed to stderr every time a GIL-using module is loaded
(including the first one that issues a warning).
@swtaarrs swtaarrs mentioned this issue May 3, 2024
Merged
encukou added a commit to encukou/cpython that referenced this issue May 6, 2024
@encukou
pythongh-116322: Rename PyModule_ExperimentalSetGIL to PyUnstable_Mod…
82f3c58 
…ule_SetGIL
@bedevere-app bedevere-app bot mentioned this issue May 6, 2024
Merged
encukou added a commit that referenced this issue May 6, 2024
@encukou
gh-116322: Rename PyModule_ExperimentalSetGIL to PyUnstable_Module_Se…
417dd3a 
…tGIL (GH-118645)
colesbury pushed a commit that referenced this issue May 7, 2024
@swtaarrs
gh-116322: Enable the GIL while loading C extension modules (#118560)
853163d 
Add the ability to enable/disable the GIL at runtime, and use that in
the C module loading code.

We can't know before running a module init function if it supports
free-threading, so the GIL is temporarily enabled before doing so. If
the module declares support for running without the GIL, the GIL is
later disabled. Otherwise, the GIL is permanently enabled, and will
never be disabled again for the life of the current interpreter.
@colesbury
Copy link
Contributor

@swtaarrs is this finished now?

@swtaarrs
Copy link
Member Author

swtaarrs commented May 7, 2024

Yep!

@swtaarrs swtaarrs closed this as completed May 7, 2024
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
@swtaarrs @SonicField
pythongh-116322: Add Py_mod_gil module slot (python#116882)
f301715 
This PR adds the ability to enable the GIL if it was disabled at
interpreter startup, and modifies the multi-phase module initialization
path to enable the GIL when loading a module, unless that module's spec
includes a slot indicating it can run safely without the GIL.

PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went
with `Py_MOD_GIL_NOT_USED` for consistency with pythongh-104148.

A warning will be issued up to once per interpreter for the first
GIL-using module that is loaded. If `-v` is given, a shorter message
will be printed to stderr every time a GIL-using module is loaded
(including the first one that issues a warning).
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
@encukou @SonicField
pythongh-116322: Rename PyModule_ExperimentalSetGIL to PyUnstable_Mod…
7678b39 
…ule_SetGIL (pythonGH-118645)
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
@swtaarrs @SonicField
pythongh-116322: Enable the GIL while loading C extension modules (py…
0d76d9c 
…thon#118560)

Add the ability to enable/disable the GIL at runtime, and use that in
the C module loading code.

We can't know before running a module init function if it supports
free-threading, so the GIL is temporarily enabled before doing so. If
the module declares support for running without the GIL, the GIL is
later disabled. Otherwise, the GIL is permanently enabled, and will
never be disabled again for the life of the current interpreter.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@swtaarrs swtaarrs

Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-free-threading type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@swtaarrs @colesbury