Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Get the "Current" Thread State from a Thread-local #103323

Closed
ericsnowcurrently opened this issue Apr 6, 2023 · 2 comments
Closed

Get the "Current" Thread State from a Thread-local #103323

ericsnowcurrently opened this issue Apr 6, 2023 · 2 comments
Labels
3.12 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement

Comments

@ericsnowcurrently
Copy link
Member

ericsnowcurrently commented Apr 6, 2023 •
edited by bedevere-app bot

(This is a revival of gh-84702, which @vstinner worked on several years ago.)

For a per-interpreter GIL (PEP 684) we need to start storing the "current" thread state in a thread-local variable, rather than _PyRuntime.tstate_current. There may be other benefits to the approach if we can take advantage of native compiler support (e.g. C11's __thread_local).

Also see https://discuss.python.org/t/how-to-get-the-current-thread-state/22655.

Linked PRs
@ericsnowcurrently ericsnowcurrently added type-feature A feature request or enhancement interpreter-core (Objects, Python, Grammar, and Parser dirs) 3.12 bugs and security fixes labels Apr 6, 2023
@ericsnowcurrently
Copy link
Member Author

CC @markshannon

@bedevere-bot bedevere-bot mentioned this issue Apr 6, 2023
Merged
ericsnowcurrently added a commit that referenced this issue Apr 24, 2023
@ericsnowcurrently
gh-103323: Get the "Current" Thread State from a Thread-Local Variable (
f8abfa3 
gh-103324)

We replace _PyRuntime.tstate_current with a thread-local variable. As part of this change, we add a _Py_thread_local macro in pyport.h (only for the core runtime) to smooth out the compiler differences. The main motivation here is in support of a per-interpreter GIL, but this change also provides some performance improvement opportunities.

Note that we do not provide a fallback to the thread-local, either falling back to the old tstate_current or to thread-specific storage (PyThread_tss_*()). If that proves problematic then we can circle back. I consider it unlikely, but will run the buildbots to double-check.

Also note that this does not change any of the code related to the GILState API, where it uses a thread state stored in thread-specific storage. I suspect we can combine that with _Py_tss_tstate (from here). However, that can be addressed separately and is not urgent (nor critical).

(While this change was mostly done independently, I did take some inspiration from earlier (~2020) work by @markshannon (main...markshannon:threadstate_in_tls) and @vstinner (#23976).)
carljm added a commit to carljm/cpython that referenced this issue Apr 24, 2023
@carljm
Merge branch 'main' into superopt
19b8025 
* main: (53 commits)
  pythongh-102498 Clean up unused variables and imports in the email module  (python#102482)
  pythongh-99184: Bypass instance attribute access in `repr` of `weakref.ref` (python#99244)
  pythongh-99032: datetime docs: Encoding is no longer relevant (python#93365)
  pythongh-94300: Update datetime.strptime documentation (python#95318)
  pythongh-103776: Remove explicit uses of $(SHELL) from Makefile (pythonGH-103778)
  pythongh-87092: fix a few cases of incorrect error handling in compiler (python#103456)
  pythonGH-103727: Avoid advancing tokenizer too far in f-string mode (pythonGH-103775)
  Revert "Add tests for empty range equality (python#103751)" (python#103770)
  pythongh-94518: Port 23-argument `_posixsubprocess.fork_exec` to Argument Clinic (python#94519)
  pythonGH-65022: Fix description of copyreg.pickle function (python#102656)
  pythongh-103323: Get the "Current" Thread State from a Thread-Local Variable (pythongh-103324)
  pythongh-91687: modernize dataclass example typing (python#103773)
  pythongh-103746: Test `types.UnionType` and `Literal` types together (python#103747)
  pythongh-103765: Fix 'Warning: py:class reference target not found: ModuleSpec' (pythonGH-103769)
  pythongh-87452: Improve the Popen.returncode docs
  Removed unnecessary escaping of asterisks (python#103714)
  pythonGH-102973: Slim down Fedora packages in the dev container (python#103283)
  pythongh-103091: Add PyUnstable_Type_AssignVersionTag (python#103095)
  Add tests for empty range equality (python#103751)
  pythongh-103712: Increase the length of the type name in AttributeError messages (python#103713)
  ...
carljm added a commit to carljm/cpython that referenced this issue Apr 24, 2023
@carljm
Merge branch 'superopt' into superopt_spec
1972ca4 
* superopt: (82 commits)
  pythongh-101517: fix line number propagation in code generated for except* (python#103550)
  pythongh-103780: Use patch instead of mock in asyncio unix events test (python#103782)
  pythongh-102498 Clean up unused variables and imports in the email module  (python#102482)
  pythongh-99184: Bypass instance attribute access in `repr` of `weakref.ref` (python#99244)
  pythongh-99032: datetime docs: Encoding is no longer relevant (python#93365)
  pythongh-94300: Update datetime.strptime documentation (python#95318)
  pythongh-103776: Remove explicit uses of $(SHELL) from Makefile (pythonGH-103778)
  pythongh-87092: fix a few cases of incorrect error handling in compiler (python#103456)
  pythonGH-103727: Avoid advancing tokenizer too far in f-string mode (pythonGH-103775)
  Revert "Add tests for empty range equality (python#103751)" (python#103770)
  pythongh-94518: Port 23-argument `_posixsubprocess.fork_exec` to Argument Clinic (python#94519)
  pythonGH-65022: Fix description of copyreg.pickle function (python#102656)
  pythongh-103323: Get the "Current" Thread State from a Thread-Local Variable (pythongh-103324)
  pythongh-91687: modernize dataclass example typing (python#103773)
  pythongh-103746: Test `types.UnionType` and `Literal` types together (python#103747)
  pythongh-103765: Fix 'Warning: py:class reference target not found: ModuleSpec' (pythonGH-103769)
  pythongh-87452: Improve the Popen.returncode docs
  Removed unnecessary escaping of asterisks (python#103714)
  pythonGH-102973: Slim down Fedora packages in the dev container (python#103283)
  pythongh-103091: Add PyUnstable_Type_AssignVersionTag (python#103095)
  ...
@hauntsaninja
Copy link
Contributor

Looks like this was completed, thanks!

vstinner added a commit to vstinner/cpython that referenced this issue May 4, 2023
@vstinner
pythongh-103323: Remove PyRuntimeState_GetThreadState()
1257ecf 
This function no longer makes sense, since its runtime parameter is
no longer used. Use directly _PyThreadState_GET() and
_PyInterpreterState_GET() instead.
vstinner added a commit to vstinner/cpython that referenced this issue May 4, 2023
@vstinner
pythongh-103323: Remove PyRuntimeState_GetThreadState()
aed221f 
This function no longer makes sense, since its runtime parameter is
no longer used. Use directly _PyThreadState_GET() and
_PyInterpreterState_GET() instead.
@bedevere-bot bedevere-bot mentioned this issue May 4, 2023
Merged
vstinner added a commit that referenced this issue May 4, 2023
@vstinner
gh-103323: Remove PyRuntimeState_GetThreadState() (#104171)
45398ad 
This function no longer makes sense, since its runtime parameter is
no longer used. Use directly _PyThreadState_GET() and
_PyInterpreterState_GET() instead.
@albertosottile albertosottile mentioned this issue Oct 22, 2023
Open
@kakkoyun kakkoyun mentioned this issue Nov 16, 2023
Open
vstinner added a commit to vstinner/cpython that referenced this issue Jan 26, 2024
@vstinner
pythongh-103323: Remove current_fast_get() unused parameter
ddce5a1 
The current_fast_get() static inline function doesn't use its
'runtime' parameter, so just remove it.
@bedevere-app bedevere-app bot mentioned this issue Jan 26, 2024
Merged
vstinner added a commit that referenced this issue Jan 30, 2024
@vstinner
gh-103323: Remove current_fast_get() unused parameter (#114593)
58f883b 
The current_fast_get() static inline function doesn't use its
'runtime' parameter, so just remove it.
aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
@vstinner @aisk
pythongh-103323: Remove current_fast_get() unused parameter (python#1…
b81e357 
…14593)

The current_fast_get() static inline function doesn't use its
'runtime' parameter, so just remove it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.12 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ericsnowcurrently @hauntsaninja