Skip to content

PEP 797: Revisions based on in-person feedback #4975

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ZeroIntensity merged 1 commit into from
May 18, 2026
+37 −75
Merged

PEP 797: Revisions based on in-person feedback #4975

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 37 additions & 75 deletions peps/pep-0797.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ Post-History: `01-Jul-2025 <https://discuss.python.org/t/97306>`__,
Abstract
========

This PEP introduces a new :func:`~concurrent.interpreters.share` function to
the :mod:`concurrent.interpreters` module, which allows any arbitrary object
to be shared across interpreters using an object proxy, at the cost of being
less efficient to concurrently access across multiple interpreters.
This PEP introduces a new :func:`~concurrent.interpreters.SharedObjectProxy`
type to the :mod:`concurrent.interpreters` module, which allows any arbitrary
object to be shared across interpreters using an object proxy, at the cost of
being less efficient to concurrently access across multiple interpreters.

For example:

Expand All @@ -26,7 +26,7 @@ For example:

with open("spanish_inquisition.txt") as unshareable:
interp = interpreters.create()
proxy = interpreters.share(unshareable)
proxy = interpreters.SharedObjectProxy(unshareable)
interp.prepare_main(file=proxy)
interp.exec("file.write('I didn't expect the Spanish Inquisition')")

Expand All @@ -45,7 +45,7 @@ the list of natively shareable objects can be found in :ref:`the documentation
Motivation
==========

Many Objects Cannot be Shared Between Subinterpreters
Many objects cannot be shared between subinterpreters
-----------------------------------------------------

In Python 3.14, the new :mod:`concurrent.interpreters` module can be used to
Expand All @@ -66,7 +66,7 @@ ideal for multithreaded applications.
Rationale
=========

A Fallback for Object Sharing
A fallback for object sharing
-----------------------------

A shared object proxy is designed to be a fallback for sharing an object
Expand All @@ -82,58 +82,13 @@ Specification
=============


.. function:: concurrent.interpreters.share(obj)

Ensure *obj* is natively shareable.

If *obj* is natively shareable, this function does not create a proxy and
simply returns *obj*. Otherwise, *obj* is wrapped in an instance of
:class:`~concurrent.interpreters.SharedObjectProxy` and returned.

If *obj* has a :meth:`~object.__share__` method, the default behavior of
this function is overridden; the object's ``__share__`` method will be
called to convert *obj* into a natively shareable version of itself, which
will be returned by this function. If the object returned by ``__share__``
is not natively shareable, this function raises an exception.

The behavior of this function is roughly equivalent to:

.. code-block:: python

def share(obj):
if _is_natively_shareable(obj):
return obj

if hasattr(obj, "__share__"):
shareable = obj.__share__()
if not _is_natively_shareable(shareable):
raise TypeError(f"__share__() returned unshareable object: {shareable!r}")

return shareable

return SharedObjectProxy(obj)


.. class:: concurrent.interpreters.SharedObjectProxy(obj)

A proxy type that allows access to an object across multiple interpreters.
Instances of this object are natively shareable between subinterpreters.

Unlike :func:`~concurrent.interpreters.share`, *obj* will always be wrapped,
even if it is natively shareable already or already a ``SharedObjectProxy``
instance. The object's :meth:`~object.__share__` method is not invoked if
it is available. Thus, prefer using ``share`` where possible.


.. function:: object.__share__()

Return a natively shareable version of the current object. This includes
shared object proxies, as they are also natively shareable. Objects composed
of shared object proxies are also allowed, such as a :class:`tuple` whose
elements are :class:`~concurrent.interpreters.SharedObjectProxy` instances.


Interpreter Switching
Interpreter switching
---------------------

When interacting with the wrapped object, the proxy will switch to the
Expand All @@ -155,14 +110,14 @@ accessed in subinterpreters through a proxy:
interp.exec("foo()")


Method Proxying
Method proxying
---------------

Methods on a shared object proxy will switch to their owning interpreter when
accessed. In addition, any arguments passed to the method are implicitly called
with :func:`~concurrent.interpreters.share` to ensure they are shareable (only
types that are not natively shareable are wrapped in a proxy). The same happens
to the return value of the method.
accessed. In addition, any arguments passed to the method are implicitly
ensured to be shareable. If they aren't natively shareable, they are wrapped
in an instance of ``SharedObjectProxy``. The same happens to the return value
of the method.

For example, the ``__add__`` method on an object proxy is roughly equivalent
to the following code:
Expand All @@ -175,7 +130,7 @@ to the following code:
return share(result)


Multithreaded Scaling
Multithreaded scaling
---------------------

To switch to a wrapped object's interpreter, an object proxy must swap the
Expand Down Expand Up @@ -219,13 +174,13 @@ performing the computation can still execute while accessing the proxy.

thread.join()

proxy = interpreters.share(write_log)
proxy = interpreters.SharedObjectProxy(write_log)
for n in range(4):
interp = interpreters.create()
interp.call_in_thread(execute, n, proxy)


Proxy Copying
Proxy copying
-------------

Contrary to what one might think, a shared object proxy itself can only be used
Expand All @@ -242,7 +197,7 @@ For example, in the following code, there are two proxies created, not just one.

interp = interpreters.create()
foo = object()
proxy = interpreters.share(foo)
proxy = interpreters.SharedObjectProxy(foo)

# The proxy crosses an interpreter boundary here. 'proxy' is *not* directly
# send to 'interp'. Instead, a new proxy is created for 'interp', and the
Expand All @@ -251,7 +206,7 @@ For example, in the following code, there are two proxies created, not just one.
interp.prepare_main(proxy=proxy)


Thread-local State
Thread-local state
------------------

Accessing an object proxy will retain information stored on the current
Expand All @@ -271,7 +226,7 @@ This allows the following case to work correctly:
assert thread_local.value == 1

interp = interpreters.create()
proxy = interpreters.share(foo)
proxy = interpreters.SharedObjectProxy(foo)
interp.prepare_main(foo=proxy)
interp.exec("foo()")

Expand Down Expand Up @@ -303,7 +258,7 @@ the thread. In other words, a shared object proxy ensures that thread local
variables and similar state will not disappear.


Memory Management
Memory management
-----------------

All proxy objects hold a :term:`strong reference` to the object that they
Expand Down Expand Up @@ -350,16 +305,16 @@ in the wrapped object's interpreter. To visualize:
interp.exec("import gc; print(gc.get_referents(proxy))")


Interpreter Lifetimes
*********************
Interpreter lifetime management
-------------------------------

When an interpreter is destroyed, shared object proxies wrapping objects
owned by that interpreter may still exist elsewhere. To prevent this
from causing crashes, an interpreter will invalidate all proxies pointing
to any object it owns by overwriting the proxy's wrapped object with ``None``.
to any object it owns, so any subsequent access to a proxy will raise an exception.

To demonstrate, the following snippet first prints out ``Alive``, and then
``None`` after deleting the interpreter:
raises a ``RuntimeError`` after deleting the interpreter:

.. code-block:: python

Expand All @@ -378,11 +333,7 @@ To demonstrate, the following snippet first prints out ``Alive``, and then
wrapped = interp.call(test)
print(wrapped) # Alive
interp.close()
print(wrapped) # None

Note that the proxy is not physically replaced (``wrapped`` in the above example
is still a ``SharedObjectProxy`` instance), but instead has its wrapped object
replaced with ``None``.
print(wrapped) # RuntimeError


Backwards Compatibility
Expand Down Expand Up @@ -410,7 +361,18 @@ A reference implementation of this PEP can be found at
Rejected Ideas
==============

Directly Sharing Proxy Objects
Introducing a generic sharing protocol
--------------------------------------

This PEP used to specify a ``share()`` function that would call a
``__share__()`` method on an object, or otherwise implicitly wrap the object
in a ``SharedObjectProxy``.

It was deemed that this wasn't necessary for this proposal to work, so this
protocol is left to be done by a future PEP.


Directly sharing proxy objects
------------------------------

The initial revision of this proposal took an approach where an instance of
Expand Down
Loading