Skip to content

gh-141518: Add PyUnstable_InterpreterState_SetEvalFrameFunc() #141665

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
+104 −25
Open

gh-141518: Add PyUnstable_InterpreterState_SetEvalFrameFunc() #141665

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
0e322e4
gh-141518: Add PyUnstable_InterpreterState_SetEvalFrameFunc()
vstinner Nov 17, 2025
757eada
Deprecate the old C API
vstinner Nov 17, 2025
4ceff72
Fix PyUnstable_FrameEvalFunction doc
vstinner Nov 17, 2025
0a8d7cd
Add tests
vstinner Nov 17, 2025
f694542
Update Modules/_testcapimodule.c
vstinner Nov 18, 2025
3e48d91
Fix copy/paste mistake
vstinner Nov 19, 2025
c167f44
Merge branch 'main' into eval_frame_func
vstinner Nov 19, 2025
c624f74
_PyFrameEvalFunction changelog
vstinner Nov 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions Doc/c-api/init.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1438,34 +1438,41 @@ All of the following functions must be called after :c:func:`Py_Initialize`.
.. versionadded:: 3.8


.. c:type:: PyObject* (*_PyFrameEvalFunction)(PyThreadState *tstate, _PyInterpreterFrame *frame, int throwflag)
.. c:type:: PyObject* (*PyUnstable_FrameEvalFunction)(PyThreadState *tstate, _PyInterpreterFrame *frame, int throwflag)

Type of a frame evaluation function.

The *throwflag* parameter is used by the ``throw()`` method of generators:
if non-zero, handle the current exception.

.. versionadded:: 3.6
as ``_PyFrameEvalFunction``.

.. versionchanged:: 3.9
The function now takes a *tstate* parameter.

.. versionchanged:: 3.11
The *frame* parameter changed from ``PyFrameObject*`` to ``_PyInterpreterFrame*``.

.. c:function:: _PyFrameEvalFunction _PyInterpreterState_GetEvalFrameFunc(PyInterpreterState *interp)
.. versionchanged:: next
Renamed to :c:type:`!PyUnstable_FrameEvalFunction`. The old private name
is deprecated, but will be available until the API changes.

.. c:function:: PyUnstable_FrameEvalFunction PyUnstable_InterpreterState_GetEvalFrameFunc(PyInterpreterState *interp)

Get the frame evaluation function.

See the :pep:`523` "Adding a frame evaluation API to CPython".

.. versionadded:: 3.9
.. versionadded:: next

.. c:function:: void _PyInterpreterState_SetEvalFrameFunc(PyInterpreterState *interp, _PyFrameEvalFunction eval_frame)
.. c:function:: void PyUnstable_InterpreterState_SetEvalFrameFunc(PyInterpreterState *interp, PyUnstable_FrameEvalFunction eval_frame)

Set the frame evaluation function.

See the :pep:`523` "Adding a frame evaluation API to CPython".

.. versionadded:: 3.9
.. versionadded:: next


.. c:function:: PyObject* PyThreadState_GetDict()
Expand Down
7 changes: 7 additions & 0 deletions Doc/deprecations/c-api-pending-removal-in-3.20.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,10 @@ Pending removal in Python 3.20
representation.

* Macros :c:macro:`!Py_MATH_PIl` and :c:macro:`!Py_MATH_El`.

* :c:func:`!_PyInterpreterState_GetEvalFrameFunc` and
:c:func:`!_PyInterpreterState_SetEvalFrameFunc` functions are deprecated
and will be removed in Python 3.20. Use
:c:func:`PyUnstable_InterpreterState_GetEvalFrameFunc` and
:c:func:`PyUnstable_InterpreterState_SetEvalFrameFunc` instead.
(Contributed by Victor Stinner in :gh:`141518`.)
2 changes: 1 addition & 1 deletion Doc/whatsnew/3.11.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2305,7 +2305,7 @@ Porting to Python 3.11
be used for ``size``.
(Contributed by Kumar Aditya in :issue:`46608`.)

* :c:func:`_PyFrameEvalFunction` now takes ``_PyInterpreterFrame*``
* :c:func:`!_PyFrameEvalFunction` now takes ``_PyInterpreterFrame*``
as its second parameter, instead of ``PyFrameObject*``.
See :pep:`523` for more details of how to use this function pointer type.

Expand Down
15 changes: 15 additions & 0 deletions Doc/whatsnew/3.15.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1091,6 +1091,14 @@ New features
* Add :c:func:`PyTuple_FromArray` to create a :class:`tuple` from an array.
(Contributed by Victor Stinner in :gh:`111489`.)

* Add a PyUnstable API for :pep:`523`:

* :c:type:`PyUnstable_FrameEvalFunction` type
* :c:func:`PyUnstable_InterpreterState_GetEvalFrameFunc`
* :c:func:`PyUnstable_InterpreterState_SetEvalFrameFunc`

(Contributed by Victor Stinner in :gh:`141518`.)

* Add :c:func:`PyUnstable_Object_Dump` to dump an object to ``stderr``.
It should only be used for debugging.
(Contributed by Victor Stinner in :gh:`141070`.)
Expand Down Expand Up @@ -1227,6 +1235,13 @@ Deprecated C APIs
since 3.15 and will be removed in 3.20.
(Contributed by Sergey B Kirpichev in :gh:`141004`.)

* :c:func:`!_PyInterpreterState_GetEvalFrameFunc` and
:c:func:`!_PyInterpreterState_SetEvalFrameFunc` functions are deprecated
and will be removed in Python 3.20. Use
:c:func:`PyUnstable_InterpreterState_GetEvalFrameFunc` and
:c:func:`PyUnstable_InterpreterState_SetEvalFrameFunc` instead.
(Contributed by Victor Stinner in :gh:`141518`.)


.. Add C API deprecations above alphabetically, not here at the end.

Expand Down
24 changes: 20 additions & 4 deletions Include/cpython/pystate.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -302,10 +302,26 @@ PyAPI_FUNC(void) PyThreadState_DeleteCurrent(void);

/* Frame evaluation API */

typedef PyObject* (*_PyFrameEvalFunction)(PyThreadState *tstate, struct _PyInterpreterFrame *, int);
typedef PyObject* (*PyUnstable_FrameEvalFunction)(PyThreadState *tstate, struct _PyInterpreterFrame *, int);

PyAPI_FUNC(_PyFrameEvalFunction) _PyInterpreterState_GetEvalFrameFunc(
PyAPI_FUNC(PyUnstable_FrameEvalFunction) PyUnstable_InterpreterState_GetEvalFrameFunc(
PyInterpreterState *interp);
PyAPI_FUNC(void) _PyInterpreterState_SetEvalFrameFunc(
PyAPI_FUNC(void) PyUnstable_InterpreterState_SetEvalFrameFunc(
PyInterpreterState *interp,
_PyFrameEvalFunction eval_frame);
PyUnstable_FrameEvalFunction eval_frame);

// Deprecated aliases kept for backward compatibility
#define _PyFrameEvalFunction PyUnstable_FrameEvalFunction

Py_DEPRECATED(3.15) static inline PyUnstable_FrameEvalFunction
_PyInterpreterState_GetEvalFrameFunc(PyInterpreterState *interp)
{
return PyUnstable_InterpreterState_GetEvalFrameFunc(interp);
}

Py_DEPRECATED(3.15) static inline void
_PyInterpreterState_SetEvalFrameFunc(PyInterpreterState *interp,
PyUnstable_FrameEvalFunction eval_frame)
{
PyUnstable_InterpreterState_SetEvalFrameFunc(interp, eval_frame);
}
4 changes: 2 additions & 2 deletions Include/internal/pycore_interp_structs.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ struct _ceval_runtime_state {
struct trampoline_api_st trampoline_api;
FILE *map_file;
Py_ssize_t persist_after_fork;
_PyFrameEvalFunction prev_eval_frame;
PyUnstable_FrameEvalFunction prev_eval_frame;
#else
int _not_used;
#endif
Expand Down Expand Up @@ -860,7 +860,7 @@ struct _is {
PyObject *sysdict_copy;
PyObject *builtins_copy;
// Initialized to _PyEval_EvalFrameDefault().
_PyFrameEvalFunction eval_frame;
PyUnstable_FrameEvalFunction eval_frame;

PyFunction_WatchCallback func_watchers[FUNC_MAX_WATCHERS];
// One bit is set for each non-NULL entry in func_watchers
Expand Down
6 changes: 3 additions & 3 deletions Misc/NEWS.d/3.9.0a5.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1305,6 +1305,6 @@ and undefined PY_SSIZE_T_CLEAN whwn an exception is set.
.. section: C API

Add a private API to get and set the frame evaluation function: add
:c:func:`_PyInterpreterState_GetEvalFrameFunc` and
:c:func:`_PyInterpreterState_SetEvalFrameFunc` C functions. The
:c:type:`_PyFrameEvalFunction` function type now takes a *tstate* parameter.
:c:func:`!_PyInterpreterState_GetEvalFrameFunc` and
:c:func:`!_PyInterpreterState_SetEvalFrameFunc` C functions. The
:c:type:`!_PyFrameEvalFunction` function type now takes a *tstate* parameter.
7 changes: 7 additions & 0 deletions Misc/NEWS.d/next/C_API/2025-11-17-16-08-11.gh-issue-141518.Z8ig9U.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Add a PyUnstable API for :pep:`523`:

* :c:type:`PyUnstable_FrameEvalFunction` type
* :c:func:`PyUnstable_InterpreterState_GetEvalFrameFunc`
* :c:func:`PyUnstable_InterpreterState_SetEvalFrameFunc`

Patch by Victor Stinner.
25 changes: 25 additions & 0 deletions Modules/_testcapimodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2595,6 +2595,29 @@ create_managed_weakref_nogc_type(PyObject *self, PyObject *Py_UNUSED(args))
}


static PyObject *
noop_eval(PyThreadState *tstate, struct _PyInterpreterFrame *f, int exc)
Copy link
Member

@encukou encukou Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't _PyInterpreterFrame, need a public name too? Otherwise you can't quite use the functions without private API.

Copy link
Member Author

@vstinner vstinner Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To implement an eval function, you should likely use the internal C API (pycore_interpframe.h) to access frame members, you're right. I don't want to add _PyInterpreterFrame structure members to the PyUnstable API, it's too unstable :-D I prefer to leave it in the internal C API.

I don't think that it's worth it to expose struct _PyInterpreterFrame* type in the PyUnstable API. What do you think?

Copy link
Member

@encukou encukou Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These functions are rather useless without making _PyInterpreterFrame public. At least should be an opaque type, with a function to upgrade it to the full PyFrameObject.

Copy link
Member Author

@vstinner vstinner Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API changed in Python 3.11 to take a _PyInterpreterFrame* instead of a PyFrameObject*. It's basically only used by PyTorch Dynamo which already uses the internal C API:

https://github.com/pytorch/pytorch/blob/be33b7faf685560bb618561b44b751713a660337/torch/csrc/dynamo/eval_frame.c#L15-L17

I wouldn't say that this API is useless without a public API for _PyInterpreterFrame*. It's just "unusual" :-)

At least should be an opaque type, with a function to upgrade it to the full PyFrameObject.

It would make the code (way) slower, _PyInterpreterFrame* idea is to avoid creating a concrete Python object for a frame unless it's strictly needed. I suggest using the internal C API for _PyInterpreterFrame* instead.

Copy link
Member

@encukou encukou Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't say that this API is useless without a public API for _PyInterpreterFrame*.

What is the use, then?

Copy link
Member Author

@vstinner vstinner Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@emmatyping @efimov-mikhail: Do you have an opinion on these questions?

Copy link
Member

@efimov-mikhail efimov-mikhail Nov 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, the main idea of this change is not really to make those functions public. We mark them as PyUnstable_ for sending clear message to its users: "we remember about those function, we don't want to change or remove them in the near future". One can use them and rely on their presence in C API, but can't rely on stability of _PyInterpreterFrame structure. So, there's a need to use some private headers in this case.

I agree that it's not the perfect position for us, but current situation isn't any better.
We have functions that both private and public.
They're private, because they have _Py* names.
But they're "kinda public", since PEP 523 is guaranteed their presence.
PyUnstable_ suits better for those semi-public functions.
But making _PyInterpreterFrame and all its members PyUnstable_ too seems redundant to me.

{
return NULL;
}

static PyObject *
test_interpreter_setevalframefunc(PyObject *self, PyObject *Py_UNUSED(args))
{
PyInterpreterState *interp = PyInterpreterState_Get();
PyUnstable_FrameEvalFunction eval_func;

eval_func = PyUnstable_InterpreterState_GetEvalFrameFunc(interp);

PyUnstable_InterpreterState_SetEvalFrameFunc(interp, noop_eval);
assert(PyUnstable_InterpreterState_GetEvalFrameFunc(interp) == noop_eval);

PyUnstable_InterpreterState_SetEvalFrameFunc(interp, eval_func);

Py_RETURN_NONE;
}


static PyMethodDef TestMethods[] = {
{"set_errno", set_errno, METH_VARARGS},
{"test_config", test_config, METH_NOARGS},
Expand Down Expand Up @@ -2691,6 +2714,8 @@ static PyMethodDef TestMethods[] = {
{"toggle_reftrace_printer", toggle_reftrace_printer, METH_O},
{"create_managed_weakref_nogc_type",
create_managed_weakref_nogc_type, METH_NOARGS},
{"test_interpreter_setevalframefunc",
test_interpreter_setevalframefunc, METH_NOARGS},
{NULL, NULL} /* sentinel */
};

Expand Down
6 changes: 4 additions & 2 deletions Modules/_testinternalcapi.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -657,7 +657,8 @@ static PyObject *
set_eval_frame_default(PyObject *self, PyObject *Py_UNUSED(args))
{
module_state *state = get_module_state(self);
_PyInterpreterState_SetEvalFrameFunc(_PyInterpreterState_GET(), _PyEval_EvalFrameDefault);
PyUnstable_InterpreterState_SetEvalFrameFunc(_PyInterpreterState_GET(),
_PyEval_EvalFrameDefault);
Py_CLEAR(state->record_list);
Py_RETURN_NONE;
}
Expand Down Expand Up @@ -689,7 +690,8 @@ set_eval_frame_record(PyObject *self, PyObject *list)
return NULL;
}
Py_XSETREF(state->record_list, Py_NewRef(list));
_PyInterpreterState_SetEvalFrameFunc(_PyInterpreterState_GET(), record_eval);
PyUnstable_InterpreterState_SetEvalFrameFunc(_PyInterpreterState_GET(),
record_eval);
Py_RETURN_NONE;
}

Expand Down
8 changes: 4 additions & 4 deletions Python/perf_trampoline.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -486,12 +486,12 @@ _PyPerfTrampoline_Init(int activate)
#ifdef PY_HAVE_PERF_TRAMPOLINE
PyThreadState *tstate = _PyThreadState_GET();
if (!activate) {
_PyInterpreterState_SetEvalFrameFunc(tstate->interp, prev_eval_frame);
PyUnstable_InterpreterState_SetEvalFrameFunc(tstate->interp, prev_eval_frame);
perf_status = PERF_STATUS_NO_INIT;
}
else if (tstate->interp->eval_frame != py_trampoline_evaluator) {
prev_eval_frame = _PyInterpreterState_GetEvalFrameFunc(tstate->interp);
_PyInterpreterState_SetEvalFrameFunc(tstate->interp, py_trampoline_evaluator);
prev_eval_frame = PyUnstable_InterpreterState_GetEvalFrameFunc(tstate->interp);
PyUnstable_InterpreterState_SetEvalFrameFunc(tstate->interp, py_trampoline_evaluator);
extra_code_index = _PyEval_RequestCodeExtraIndex(NULL);
if (extra_code_index == -1) {
return -1;
Expand All @@ -517,7 +517,7 @@ _PyPerfTrampoline_Fini(void)
}
PyThreadState *tstate = _PyThreadState_GET();
if (tstate->interp->eval_frame == py_trampoline_evaluator) {
_PyInterpreterState_SetEvalFrameFunc(tstate->interp, NULL);
PyUnstable_InterpreterState_SetEvalFrameFunc(tstate->interp, NULL);
}
if (perf_status == PERF_STATUS_OK) {
trampoline_api.free_state(trampoline_api.state);
Expand Down
8 changes: 4 additions & 4 deletions Python/pystate.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2896,8 +2896,8 @@ PyGILState_Release(PyGILState_STATE oldstate)
/* Other API */
/*************/

_PyFrameEvalFunction
_PyInterpreterState_GetEvalFrameFunc(PyInterpreterState *interp)
PyUnstable_FrameEvalFunction
PyUnstable_InterpreterState_GetEvalFrameFunc(PyInterpreterState *interp)
{
if (interp->eval_frame == NULL) {
return _PyEval_EvalFrameDefault;
Expand All @@ -2907,8 +2907,8 @@ _PyInterpreterState_GetEvalFrameFunc(PyInterpreterState *interp)


void
_PyInterpreterState_SetEvalFrameFunc(PyInterpreterState *interp,
_PyFrameEvalFunction eval_frame)
PyUnstable_InterpreterState_SetEvalFrameFunc(PyInterpreterState *interp,
PyUnstable_FrameEvalFunction eval_frame)
{
if (eval_frame == _PyEval_EvalFrameDefault) {
eval_frame = NULL;
Expand Down
Loading