Removing old buffer support #85275
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
assignee = 'https://github.com/encukou' closed_at = None created_at = <Date 2020-06-24.13:43:42.640> labels = ['expert-C-API', '3.10', '3.11'] title = 'Removing old buffer support' updated_at = <Date 2021-08-05.08:33:02.859> user = 'https://github.com/methane'
activity = <Date 2021-08-05.08:33:02.859> actor = 'petr.viktorin' assignee = 'petr.viktorin' closed = False closed_date = None closer = None components = ['C API'] creation = <Date 2020-06-24.13:43:42.640> creator = 'methane' dependencies =  files =  hgrepos =  issue_num = 41103 keywords = ['patch'] message_count = 12.0 messages = ['372251', '372391', '381854', '381865', '381873', '398459', '398460', '398467', '398476', '398477', '398560', '398974'] nosy_count = 7.0 nosy_names = ['vstinner', 'petr.viktorin', 'methane', 'lukasz.langa', 'hroncok', 'miss-islington', 'tarun.johar'] pr_nums = ['21117', '27437', '27441'] priority = 'normal' resolution = 'wont fix' stage = 'resolved' status = 'open' superseder = None type = None url = 'https://bugs.python.org/issue41103' versions = ['Python 3.10', 'Python 3.11']
The text was updated successfully, but these errors were encountered:
I've just seen a fix of PyQt4 that basically copy pastes (some of) the removed code to PyQt4:
What is the benefit of removing this? Is copy pasting the compatibility layer to (possibly many) different projects worth the "cleanup"?
In Fedora, at least 7 packages are broken so far:
PyObject_AsCharBuffer() is dangerous: it returns a dangling pointer by design. PyObject_GetBuffer() design is safer: the API ensures that the buffer remains valid until PyBuffer_Release() is called.
PyQt5 was updated to use the safe PyObject_GetBuffer()/PyBuffer_Release(). PyQt4 is no longer updated, that's why I proposed a downstream fix which copy/paste the old code. Changing PyQt4 to use the safe API was not worth it, since it's complex to redesign the impacted function qpycore_encode().
Thank you for reporting. I removed these APIs to get such feedback as early as possible.
We are free to revive these APIs if it breaks too much and some projects can not be fixed before Python 3.10 release.
Some project maintainers ignore deprecations and wait compile errors to fix soemthing. (e.g. rogerbinns/apsw#288 (comment))
Oh, no need to copy-paste compatibility layer for all projects. They can migrate to new APIs.
Also filed under https://bugs.python.org/issue44609
The following functions are present in Python 3.9 but have been removed from Python 3.10b3:
PyObject_AsCharBuffer() PyObject_AsReadBuffer() PyObject_AsWriteBuffer() PyObject_CheckReadBuffer()
Without these functions, an extension cannot utilize the stable ABI to access the buffer memory of data structures. The buffer protocol is suggested as an alternative, but the buffer functions PyObject_GetBuffer() and PyBuffer_Release() are not present in the stable ABI.
While these two functions may be added to the stable ABI, removal of the four functions above makes Python 3.10 incompatible with previous versions. It is requested that the four functions be reinstated and maintained as described in PEP-652.
These should be removed from the *limited API*, but stay for the stable ABI.
I assigned this to myself; I'll try to get to it for 3.11.
Yes. PyObject_AsReadBuffer is dangerous *in general*, but in certain specific cases where you're relying on CPython implementations details it can work. For example, borrowing a buffer from a *string* or *array* will work on CPython, and I don't see that changing any time soon. So, using PyObject_AsReadBuffer is still tech debt, but for some projects that's OK.
All this should be documented in the release notes, though, so the projects know what they're doing. (And the release notes should be written *with the change*, not later, so projects testing early can read them.)
I closed my issue #105186 as a duplicate of this one. Copy of my message.
Would it be ok to remove the legacy "buffer" API in Python 3.13, scheduled for October 2024? I'm talking about functions:
This API is deprecated since Python 3.0 and is older than the new
In June 2020, there was a first attempt to remove these functions in issue #85275: commit 6f8a6ee. It caused too many troubles and had to be reverted the following year (commit ce5e1a6, July 2021). The removed functions were still used by at least 6 projects:
The webassets project was also mentioned, but I'm not sure that its failure is related to these removal: see https://bugzilla.redhat.com/show_bug.cgi?id=1899555 I cannot find these functions in https://github.com/Kronuz/pyScss code right now.
Sadly, the 4 functions are part of the stable ABI and have to be kept there. I'm only asking to remove them in the C API. The issue #88775 was complaining that the functions were removed wheareas they are part of the stable ABI.