Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

--disable-gil builds are not compatible with the limited API #111506

Open
Tracked by #108219
colesbury opened this issue Oct 30, 2023 · 5 comments
Open
Tracked by #108219

--disable-gil builds are not compatible with the limited API #111506

colesbury opened this issue Oct 30, 2023 · 5 comments
Labels
3.13 new features, bugs and security fixes topic-C-API topic-free-threading type-bug An unexpected behavior, bug, or error

Comments

@colesbury
Copy link
Contributor

colesbury commented Oct 30, 2023 •
edited by bedevere-app bot

Bug report

After #110764, --disable-gil builds of CPython 3.13 are (temporarily) not be compatible with the limited API or the stable ABI due to modifications to the PyObject header. For context, this was noted in the "Backwards Compatibility" section of PEP 703.

Based on feedback on the related discuss post and discussions with Gregory Smith, Guido, Petr, and Victor at the core dev sprint, I believe we can make changes to the stable ABI so that extension author can target a single stable ABI that supports CPython 3.7+ and both the default build and the --disable-gil builds of CPython 3.13+.

These changes will require a PEP. This issue is intended just to keep track of the known problem.

The rough idea is:

  1. make a number of macro/inline functions (e.g., Py_SIZE, Py_TYPE, Py_SET_REFCNT, Py_REFCNT) "real" function calls in the stable ABI for CPython 3.13+
  2. When targeting CPython <= 3.12, use dynamic symbol lookup (GetProcAddress on Windows) or weak symbols (on Linux/macOS) to determine at runtime if the active CPython version provides the desired functions. Otherwise, fall back to old definitions of the above functions for CPython <= 3.12.

Linked PRs
@colesbury colesbury added type-bug An unexpected behavior, bug, or error 3.13 new features, bugs and security fixes topic-free-threading labels Oct 30, 2023
@colesbury colesbury mentioned this issue Oct 30, 2023
Open
@corona10
Copy link
Member

cc @vstinner

vstinner added a commit to vstinner/cpython that referenced this issue Oct 30, 2023
@vstinner
pythongh-111506: Implement Py_SET_REFCNT() as opaque function in limi…
485ad40 
…ted C API

In the limited C API version 3.13. Py_SET_REFCNT() is now implemented
as an opaque function call.

Add _Py_SetRefcnt() to the stable ABI.
vstinner added a commit to vstinner/cpython that referenced this issue Oct 30, 2023
@vstinner
pythongh-111506: Implement Py_SET_REFCNT() as opaque function in limi…
67638c4 
…ted C API

In the limited C API version 3.13, Py_SET_REFCNT() function is now
implemented as an opaque function call.

Add _Py_SetRefcnt() to the stable ABI.
@bedevere-app bedevere-app bot mentioned this issue Oct 30, 2023
Merged
@vstinner
Copy link
Member

Before creating a whole new ABI, I would like first to avoid __asm__(...); in Include/*.h, in the limited C API. Assembly code is not portable and can cause compilation issues.

A first step for that is to implement Py_SET_REFCNT() as a function call: PR #111508.

vstinner added a commit to vstinner/cpython that referenced this issue Oct 30, 2023
@vstinner
pythongh-111506: Implement Py_SET_REFCNT() as opaque function in limi…
ac0c1dd 
…ted C API

In the limited C API version 3.13, Py_SET_REFCNT() function is now
implemented as an opaque function call.

Add _Py_SetRefcnt() to the stable ABI.
@encukou
Copy link
Member

encukou commented Oct 31, 2023

These changes will require a PEP.

Please, could we have the PEP first, then do the changes?

vstinner added a commit that referenced this issue Nov 3, 2023
@vstinner
gh-111506: Implement Py_SET_REFCNT() as opaque function in limited C …
20cfab9 
…API (#111508)

In the limited C API version 3.13, Py_SET_REFCNT() function is now
implemented as an opaque function call.

Add _Py_SetRefcnt() to the stable ABI.
vstinner added a commit to vstinner/cpython that referenced this issue Dec 5, 2023
@vstinner
pythongh-111506: Implement Py_REFCNT() as opaque function call
d52b1e9 
In the limited C API version 3.13, the Py_REFCNT() function is now
implemented as an opaque function call.
@bedevere-app bedevere-app bot mentioned this issue Dec 5, 2023
Closed
vstinner added a commit to vstinner/cpython that referenced this issue Dec 5, 2023
@vstinner
pythongh-111506: Implement Py_REFCNT() as opaque function call
4fd7ed6 
In the limited C API version 3.13, the Py_REFCNT() function is now
implemented as an opaque function call.
@encukou
Copy link
Member

encukou commented Dec 5, 2023

Please, could we have the PEP first, then do the changes?

@colesbury, do you still plan to write the PEP? Do you need any help?

@colesbury
Copy link
Contributor Author

@encukou, yes I still intend to write the PEP. It just hasn't been as a high a priority as integrating the pieces of PEP 703.

vstinner added a commit to vstinner/cpython that referenced this issue Dec 6, 2023
@vstinner
pythongh-111506: Rename _Py_SetRefcnt() to Py_SET_REFCNT()
22fcb01
vstinner added a commit to vstinner/cpython that referenced this issue Dec 6, 2023
@vstinner
pythongh-111506: Rename _Py_SetRefcnt() to Py_SET_REFCNT()
4c003fb
vstinner added a commit to vstinner/cpython that referenced this issue Dec 6, 2023
@vstinner
pythongh-111506: Rename _Py_SetRefcnt() to Py_SET_REFCNT()
cb415ce 
Add  Objects/object_abi.c file and _Py_STABLE_ABI_IMPL macro.
vstinner added a commit to vstinner/cpython that referenced this issue Dec 6, 2023
@vstinner
pythongh-111506: Rename _Py_SetRefcnt() to Py_SET_REFCNT()
81be63c 
Export Py_SET_REFCNT() opaque function as "Py_SET_REFCNT" symbol in
the stable ABI.

Add  Objects/object_abi.c file and _Py_STABLE_ABI_IMPL macro.
@bedevere-app bedevere-app bot mentioned this issue Dec 6, 2023
Closed
vstinner added a commit to vstinner/cpython that referenced this issue Dec 6, 2023
@vstinner
pythongh-111506: Implement Py_SET_REFCNT() as inline function
a9e8d7a 
Convert Py_SET_REFCNT() and _Py_IsImmortal() static inline functions
to inline functions.

Stable ABI now use "Py_SET_REFCNT" name instead of "_Py_SetRefcnt".
vstinner added a commit to vstinner/cpython that referenced this issue Dec 6, 2023
@vstinner
pythongh-111506: Implement Py_SET_REFCNT() as inline function
0c7cb61 
Convert Py_SET_REFCNT() and _Py_IsImmortal() static inline functions
to inline functions.

_Py_IsImmortal() has to be converted to an inline function, since
non-static Py_SET_REFCNT() function cannot call a static
_Py_IsImmortal() function.

Py_SET_REFCNT() function is now exported as "Py_SET_REFCNT" name in
the stable ABI, instead of "_Py_SetRefcnt".
vstinner added a commit to vstinner/cpython that referenced this issue Dec 6, 2023
@vstinner
pythongh-111506: Implement Py_SET_REFCNT() as inline function
076136e 
Convert Py_SET_REFCNT() and _Py_IsImmortal() static inline functions
to inline functions.

Py_SET_REFCNT() function is now exported as "Py_SET_REFCNT" name in
the stable ABI, instead of "_Py_SetRefcnt".

_Py_IsImmortal() has to be converted to an inline function, since
non-static Py_SET_REFCNT() function cannot call a static
_Py_IsImmortal() function. _Py_IsImmortal() is not part of the stable
ABI.
@bedevere-app bedevere-app bot mentioned this issue Dec 6, 2023
Closed
vstinner added a commit to vstinner/cpython that referenced this issue Dec 7, 2023
@vstinner
pythongh-111506: Implement Py_SET_REFCNT() as inline function
bec2a0f 
Convert Py_SET_REFCNT() and _Py_IsImmortal() static inline functions
to inline functions.

Py_SET_REFCNT() function is now exported as "Py_SET_REFCNT" name in
the stable ABI, instead of "_Py_SetRefcnt".

_Py_IsImmortal(), _Py_IsOwnedByCurrentThread() and _Py_ThreadId()
have to be converted to an inline functions, since non-static
Py_SET_REFCNT() function cannot call a static functions.
_Py_IsImmortal(), _Py_IsOwnedByCurrentThread() and _Py_ThreadId() are
not part of the stable ABI.
vstinner added a commit to vstinner/cpython that referenced this issue Dec 7, 2023
@vstinner
pythongh-111506: Implement Py_SET_REFCNT() as inline function
49d6671 
Convert Py_SET_REFCNT() and _Py_IsImmortal() static inline functions
to inline functions.

Py_SET_REFCNT() function is now exported as "Py_SET_REFCNT" name in
the stable ABI, instead of "_Py_SetRefcnt".

_Py_IsImmortal(), _Py_IsOwnedByCurrentThread() and _Py_ThreadId()
have to be converted to an inline functions, since non-static
Py_SET_REFCNT() function cannot call a static functions.
_Py_IsImmortal(), _Py_IsOwnedByCurrentThread() and _Py_ThreadId() are
not part of the stable ABI.
aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
@vstinner @aisk
pythongh-111506: Implement Py_SET_REFCNT() as opaque function in limi…
f977bc1 
…ted C API (python#111508)

In the limited C API version 3.13, Py_SET_REFCNT() function is now
implemented as an opaque function call.

Add _Py_SetRefcnt() to the stable ABI.
colesbury added a commit to colesbury/cpython that referenced this issue Apr 11, 2024
@colesbury
pythongh-111506: Error if the limited API is used in free-threaded build
5f54ac7 
Issue a build time error if both `Py_LIMITED_API` and `Py_GIL_DISABLED`
are defined.
@bedevere-app bedevere-app bot mentioned this issue Apr 11, 2024
Merged
@colesbury colesbury mentioned this issue Apr 11, 2024
Open
colesbury added a commit that referenced this issue Apr 11, 2024
@colesbury
gh-111506: Error if the limited API is used in free-threaded build (#…
39d381f 
…117762)

Issue a build time error if both `Py_LIMITED_API` and `Py_GIL_DISABLED`
are defined.
diegorusso pushed a commit to diegorusso/cpython that referenced this issue Apr 17, 2024
@colesbury @diegorusso
pythongh-111506: Error if the limited API is used in free-threaded bu…
0e27fe7 
…ild (python#117762)

Issue a build time error if both `Py_LIMITED_API` and `Py_GIL_DISABLED`
are defined.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.13 new features, bugs and security fixes topic-C-API topic-free-threading type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@vstinner @encukou @colesbury @corona10 @AlexWaygood