You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Symbols and functions exported by the <token.h> header file cannot be used outside Python internals. There is no public C API to tokenize a Python source file. I propose to simply remove this header file. It's not included by Python.h.
Remove the token.h header file. There was never any public tokenizer
C API. The token.h header file was only designed to be used by Python
internals.
Move Include/token.h to Include/internal/pycore_token.h. Including
this header file now requires that the Py_BUILD_CORE macro is
defined. It no longer checks for the Py_LIMITED_API macro.
Rename functions:
* PyToken_OneChar() => _PyToken_OneChar()
* PyToken_TwoChars() => _PyToken_TwoChars()
* PyToken_ThreeChars() => _PyToken_ThreeChars()
Symbols and functions exported by the
<token.h>
header file cannot be used outside Python internals. There is no public C API to tokenize a Python source file. I propose to simply remove this header file. It's not included byPython.h
.In Python 3.10, I removed the C API related to AST and the parser, since it was not possible to use it outside Python internals. See: https://docs.python.org/dev/whatsnew/3.10.html#id4
The text was updated successfully, but these errors were encountered: