Skip to content

Commit d666cba

Browse files
authored
gh-119786: move 'changing grammar' checklist from devguide to InternalDocs (#1455)
1 parent 0dd2b95 commit d666cba

File tree

1 file changed

+2
-68
lines changed

1 file changed

+2
-68
lines changed

developer-workflow/grammar.rst

Lines changed: 2 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -1,70 +1,4 @@
11
.. _grammar:
22

3-
==========================
4-
Changing CPython's grammar
5-
==========================
6-
7-
Abstract
8-
========
9-
10-
There's more to changing Python's grammar than editing
11-
:cpy-file:`Grammar/python.gram`. Here's a checklist.
12-
13-
.. note::
14-
These instructions are for Python 3.9 and beyond. Earlier
15-
versions use a different parser technology. You probably shouldn't
16-
try to change the grammar of earlier Python versions, but if you
17-
really want to, use GitHub to track down the earlier version of this
18-
file in the devguide.
19-
20-
For more information on how to use the new parser, check the
21-
:ref:`section on how to use CPython's parser <parser>`.
22-
23-
Checklist
24-
=========
25-
26-
Note: sometimes things mysteriously don't work. Before giving up, try ``make clean``.
27-
28-
* :cpy-file:`Grammar/python.gram`: The grammar, with actions that build AST nodes.
29-
After changing it, run ``make regen-pegen`` (or ``build.bat --regen`` on Windows),
30-
to regenerate :cpy-file:`Parser/parser.c`.
31-
(This runs Python's parser generator, :cpy-file:`Tools/peg_generator`).
32-
33-
* :cpy-file:`Grammar/Tokens` is a place for adding new token types. After
34-
changing it, run ``make regen-token`` to regenerate
35-
:cpy-file:`Include/internal/pycore_token.h`, :cpy-file:`Parser/token.c`,
36-
:cpy-file:`Lib/token.py` and :cpy-file:`Doc/library/token-list.inc`.
37-
If you change both ``python.gram`` and ``Tokens``,
38-
run ``make regen-token`` before ``make regen-pegen``.
39-
On Windows, ``build.bat --regen`` will regenerate both at the same time.
40-
41-
* :cpy-file:`Parser/Python.asdl` may need changes to match the grammar.
42-
Then run ``make regen-ast`` to regenerate
43-
:cpy-file:`Include/internal/pycore_ast.h` and :cpy-file:`Python/Python-ast.c`.
44-
45-
* :cpy-file:`Parser/lexer/` contains the tokenization code.
46-
This is where you would add a new type of comment or string literal, for example.
47-
48-
* :cpy-file:`Python/ast.c` will need changes to validate AST objects
49-
involved with the grammar change.
50-
51-
* :cpy-file:`Python/ast_unparse.c` will need changes to unparse AST
52-
involved with the grammar change ("unparsing" is used to turn annotations
53-
into strings per :pep:`563`).
54-
55-
* The :ref:`compiler` has its own page.
56-
57-
* ``_Unparser`` in the :cpy-file:`Lib/ast.py` file may need changes
58-
to accommodate any modifications in the AST nodes.
59-
60-
* :cpy-file:`Doc/library/ast.rst` may need to be updated to reflect changes
61-
to AST nodes.
62-
63-
* Add some usage of your new syntax to ``test_grammar.py``.
64-
65-
* Certain changes may require tweaks to the library module :mod:`pyclbr`.
66-
67-
* :cpy-file:`Lib/tokenize.py` needs changes to match changes to the tokenizer.
68-
69-
* Documentation must be written! Specifically, one or more of the pages in
70-
:cpy-file:`Doc/reference/` will need to be updated.
3+
This document is now part of the
4+
`CPython Internals Docs <https://github.com/python/cpython/blob/main/InternalDocs/changing_grammar.md>`_.

0 commit comments

Comments
 (0)