Skip to content

bpo-46850: Move private _PyEval functions to internal C API #31532

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

bpo-46850: Move private _PyEval functions to internal C API #31532

wants to merge 1 commit into from

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented Feb 23, 2022

Move the following private functions to the internal C API:

  • _PyEval_CallTracing()
  • _PyEval_EvalFrameDefault()
  • _PyEval_GetAsyncGenFinalizer()
  • _PyEval_GetAsyncGenFirstiter()
  • _PyEval_GetCoroutineOriginTrackingDepth()
  • _PyEval_GetSwitchInterval()
  • _PyEval_RequestCodeExtraIndex()
  • _PyEval_SetAsyncGenFinalizer()
  • _PyEval_SetAsyncGenFirstiter()
  • _PyEval_SetProfile()
  • _PyEval_SetSwitchInterval()
  • _PyEval_SetTrace()

_PyEval_EvalFrameDefault() uses "struct _interpreter_frame" which is
part of the internal C API.

Keep the following functions in Include/cpython/ceval.h:

  • _PyEval_GetBuiltin()
  • _PyEval_GetBuiltinId()
  • _PyEval_SliceIndex()
  • _PyEval_SliceIndexNotNone()

https://bugs.python.org/issue46850

@vstinner
Copy link
Member Author

Code search on top 5000 PyPI projects (2022-01-26). This PR seems to impact a bunch of projects.

numpy-1.22.1.zip:

numpy/core/src/multiarray/temp_elide.c: #define PYFRAMEEVAL_FUNC "_PyEval_EvalFrameDefault"

Cython-0.29.26.tar.gz: ok, copy of CPython gdb libpython.py

Cython/Debugger/libpython.py: EVALFRAME = '_PyEval_EvalFrameDefault'
Cython/Debugger/libpython.py: '''Is this a _PyEval_EvalFrameDefault frame, or some other important
Cython/Debugger/libpython.py: '''Is this a _PyEval_EvalFrameDefault frame?'''
Cython/Debugger/libpython.py: # We have a _PyEval_EvalFrameDefault frame:

debugpy-1.5.1.zip:

src/debugpy/_vendored/pydevd/pydevd_attach_to_process/common/python.h: * or most recently, executing _PyEval_EvalFrameDefault. */
src/debugpy/_vendored/pydevd/pydevd_attach_to_process/linux_and_mac/attach.cpp: typedef unsigned long (*_PyEval_GetSwitchInterval)(void);
src/debugpy/_vendored/pydevd/pydevd_attach_to_process/linux_and_mac/attach.cpp: typedef void (*_PyEval_SetSwitchInterval)(unsigned long microseconds);
src/debugpy/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp: typedef unsigned long (_PyEval_GetSwitchInterval)(void);
src/debugpy/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp: typedef void (_PyEval_SetSwitchInterval)(unsigned long microseconds);
src/debugpy/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp: DEFINE_PROC_NO_CHECK(getSwitchInterval, _PyEval_GetSwitchInterval*, "_PyEval_GetSwitchInterval", -260);  // optional
src/debugpy/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp: DEFINE_PROC_NO_CHECK(setSwitchInterval, _PyEval_SetSwitchInterval*, "_PyEval_SetSwitchInterval", -270);  // optional
src/debugpy/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pxd: #define CALL_EvalFrameDefault_39(a, b, c)    _PyEval_EvalFrameDefault(a, b, c)
src/debugpy/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pxd: #define CALL_EvalFrameDefault_38(a, b)    _PyEval_EvalFrameDefault(a, b)
src/debugpy/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pxd: int _PyEval_RequestCodeExtraIndex(freefunc)
src/debugpy/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pxd: # PyObject* _PyEval_EvalFrameDefault(PyThreadState* tstate, PyFrameObject *frame, int exc)
src/debugpy/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pxd: # PyObject* _PyEval_EvalFrameDefault(PyFrameObject *frame, int exc)
src/debugpy/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pxd: PyObject* _PyEval_EvalFrameDefault(...)
src/debugpy/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.template.pyx: _code_extra_index = <int> _PyEval_RequestCodeExtraIndex(release_co_extra)
src/debugpy/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.template.pyx: state.interp.eval_frame = _PyEval_EvalFrameDefault

uvloop-0.16.0.tar.gz: ok, just a comment

uvloop/loop.pyx: # _PyEval_EvalFrameDefault which will process

py_spy-0.3.11.tar.gz: ok, just a comment

src/native_stack_trace.rs: // _PyEval_EvalFrameDefault.cold.2962

pydevd-2.7.0.tar.gz:

pydevd_attach_to_process/common/python.h: * or most recently, executing _PyEval_EvalFrameDefault. */
pydevd_attach_to_process/linux_and_mac/attach.cpp: typedef unsigned long (*_PyEval_GetSwitchInterval)(void);
pydevd_attach_to_process/linux_and_mac/attach.cpp: typedef void (*_PyEval_SetSwitchInterval)(unsigned long microseconds);
pydevd_attach_to_process/windows/attach.cpp: typedef unsigned long (_PyEval_GetSwitchInterval)(void);
pydevd_attach_to_process/windows/attach.cpp: typedef void (_PyEval_SetSwitchInterval)(unsigned long microseconds);
pydevd_attach_to_process/windows/attach.cpp: DEFINE_PROC_NO_CHECK(getSwitchInterval, _PyEval_GetSwitchInterval*, "_PyEval_GetSwitchInterval", -260);  // optional
pydevd_attach_to_process/windows/attach.cpp: DEFINE_PROC_NO_CHECK(setSwitchInterval, _PyEval_SetSwitchInterval*, "_PyEval_SetSwitchInterval", -270);  // optional

ptvsd-4.3.2.zip:

src/ptvsd/_vendored/pydevd/pydevd_attach_to_process/linux_and_mac/attach.cpp: typedef unsigned long (*_PyEval_GetSwitchInterval)(void);
src/ptvsd/_vendored/pydevd/pydevd_attach_to_process/linux_and_mac/attach.cpp: typedef void (*_PyEval_SetSwitchInterval)(unsigned long microseconds);
src/ptvsd/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp: typedef unsigned long (_PyEval_GetSwitchInterval)(void);
src/ptvsd/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp: typedef void (_PyEval_SetSwitchInterval)(unsigned long microseconds);
src/ptvsd/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp: DEFINE_PROC_NO_CHECK(getSwitchInterval, _PyEval_GetSwitchInterval*, "_PyEval_GetSwitchInterval", -260);  // optional
src/ptvsd/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp: DEFINE_PROC_NO_CHECK(setSwitchInterval, _PyEval_SetSwitchInterval*, "_PyEval_SetSwitchInterval", -270);  // optional
src/ptvsd/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pxd: int _PyEval_RequestCodeExtraIndex(freefunc)
src/ptvsd/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pxd: PyObject* _PyEval_EvalFrameDefault(PyFrameObject *frame, int exc)
src/ptvsd/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pyx: _code_extra_index = _PyEval_RequestCodeExtraIndex(release_co_extra)
src/ptvsd/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pyx: return _PyEval_EvalFrameDefault(frame_obj, exc)
src/ptvsd/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pyx: #     return _PyEval_EvalFrameDefault(frame_obj, exc)
src/ptvsd/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pyx: return _PyEval_EvalFrameDefault(frame_obj, exc)
src/ptvsd/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pyx: return _PyEval_EvalFrameDefault(frame_obj, exc)
src/ptvsd/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pyx: return _PyEval_EvalFrameDefault(frame_obj, exc)
src/ptvsd/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pyx: return _PyEval_EvalFrameDefault(frame_obj, exc)
src/ptvsd/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pyx: return _PyEval_EvalFrameDefault(frame_obj, exc)
src/ptvsd/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pyx: return _PyEval_EvalFrameDefault(frame_obj, exc)
src/ptvsd/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.pyx: state.interp.eval_frame = _PyEval_EvalFrameDefault

pickle5-0.0.12.tar.gz: use _PyEval_GetBuiltinId()

pickle5/_pickle.c: st->getattr = _PyEval_GetBuiltinId(&PyId_getattr);
pickle5/compat.h: _PyEval_GetBuiltinId(_Py_Identifier *name)

pydevd-pycharm-213.6777.8.tar.gz:

pydevd_attach_to_process/linux_and_mac/attach.cpp: typedef unsigned long (*_PyEval_GetSwitchInterval)(void);
pydevd_attach_to_process/linux_and_mac/attach.cpp: typedef void (*_PyEval_SetSwitchInterval)(unsigned long microseconds);
pydevd_attach_to_process/windows/attach.cpp: typedef unsigned long (_PyEval_GetSwitchInterval)(void);
pydevd_attach_to_process/windows/attach.cpp: typedef void (_PyEval_SetSwitchInterval)(unsigned long microseconds);
pydevd_attach_to_process/windows/attach.cpp: DEFINE_PROC_NO_CHECK(getSwitchInterval, _PyEval_GetSwitchInterval*, "_PyEval_GetSwitchInterval", -260);  // optional
pydevd_attach_to_process/windows/attach.cpp: DEFINE_PROC_NO_CHECK(setSwitchInterval, _PyEval_SetSwitchInterval*, "_PyEval_SetSwitchInterval", -270);  // optional

osmium-3.2.0.tar.gz: ok, Valgrind suppression file

contrib/pybind11/tests/valgrind-numpy-scipy.supp: fun:_PyEval_EvalFrameDefault
contrib/pybind11/tests/valgrind-numpy-scipy.supp: fun:_PyEval_EvalFrameDefault
contrib/pybind11/tests/valgrind-python.supp: fun:_PyEval_EvalFrameDefault
contrib/pybind11/tests/valgrind-python.supp: fun:_PyEval_EvalFrameDefault
contrib/pybind11/tests/valgrind-python.supp: fun:_PyEval_EvalFrameDefault

tweedledum-1.1.1.tar.gz: ok, Valgrind suppression file

external/pybind11/tests/valgrind-numpy-scipy.supp: fun:_PyEval_EvalFrameDefault
external/pybind11/tests/valgrind-numpy-scipy.supp: fun:_PyEval_EvalFrameDefault
external/pybind11/tests/valgrind-python.supp: fun:_PyEval_EvalFrameDefault
external/pybind11/tests/valgrind-python.supp: fun:_PyEval_EvalFrameDefault
external/pybind11/tests/valgrind-python.supp: fun:_PyEval_EvalFrameDefault

@vstinner
Copy link
Member Author

I modified the PR to keep _PyEval_GetBuiltinId(): it's used by the pickle5 project. It's a backport for Python 3.7 and older: https://pypi.org/project/pickle5/ But it should not harm to keep this function for now.

@vstinner
Copy link
Member Author

@brandtbucher @markshannon: Do you think that it would be a good idea to move functions like _PyEval_EvalFrameDefault(), _PyEval_SetProfile() and _PyEval_SetSwitchInterval() to the internal C API?

First, I created this PR just for _PyEval_EvalFrameDefault() :-)

Remove the following _PyEval private functions (move them to the
internal C API pycore_ceval.h header):

* _PyEval_CallTracing()
* _PyEval_EvalFrameDefault()
* _PyEval_GetAsyncGenFinalizer()
* _PyEval_GetAsyncGenFirstiter()
* _PyEval_GetCoroutineOriginTrackingDepth()
* _PyEval_GetSwitchInterval()
* _PyEval_RequestCodeExtraIndex()
* _PyEval_SetAsyncGenFinalizer()
* _PyEval_SetAsyncGenFirstiter()
* _PyEval_SetProfile()
* _PyEval_SetSwitchInterval()
* _PyEval_SetTrace()

_PyEval_EvalFrameDefault() uses the InterpreterFrame type which is
only part of the internal C API.

Keep the following functions in Include/cpython/ceval.h:

* _PyEval_GetBuiltin()
* _PyEval_GetBuiltinId()
* _PyEval_SliceIndex()
* _PyEval_SliceIndexNotNone()
@vstinner vstinner changed the title bpo-46836: Move private _PyEval functions to internal C API bpo-46850: Move private _PyEval functions to internal C API Feb 24, 2022
@vstinner vstinner marked this pull request as ready for review February 24, 2022 15:44
@vstinner
Copy link
Member Author

I splitted the PR into smaller PRs: see https://bugs.python.org/issue46850

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants