From 953bcfa0855a12d0c1dcf88067b8792e5954aa87 Mon Sep 17 00:00:00 2001 From: "Gregory P. Smith" Date: Sun, 28 Sep 2025 03:07:42 +0000 Subject: [PATCH 1/4] Initial CLAUDE.md for cpython repo I expect we'll rename this AGENTS.md in the future once the dust settles around the pile of agentic instruction file naming patterns. Starting here as many of the core team now have Claude Max access. --- CLAUDE.md | 90 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000000000..ff5411675813b8 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,90 @@ +# Context + +You are in the CPython repo helping work on the implementation of the Python +language runtime and standard library itself. + +Use the `gh` tool to get information about an issue or PR in the repo. +Including using the GraphQL API via `gh` when appropriate. + +Source files in this repo can be very long. Check their size to consider if +you really need to read the entire thing. + +If tools such as `rg` (ripgrep), `gh`, `jq`, or `pre-commit` are not found, ask +the user to install them. ALWAYS prefer using `rg` rather than `find` or `grep`. + +# Expanding your knowledge + +* ALWAYS load a `.claude/pr-{PR_NUMBER}.md` or `.claude/branch-{branch_name_without_slashes}.md` file when you are told a PR number or when the current git branch is not `main`. +* Keep the file up to date as an engineering notebook of your learnings and project state as you work on a task and after you commit. The goal of our notebook is to make it easier to pick up and resume work later. + +## Optional developer guides and PEPs + +* If `REPO_ROOT/../devguide/` exists, its `developer-workflow/` and `documentation/` subdirs contain detailed info on how we like to work on code and docs. +* PEPs might exist in a `REPO_ROOT/../peps/` tree. + +# Source code + +* The runtime is implemented in C as are many of the core types and extension modules +* The Python standard library (stdlib) itself lives in the `Lib/` tree +* stdlib C extension modules live in the `Modules/` Tree +* builtins, objects, and the runtime itself live in `Objects/` and `Python/` +* NEVER edit files in a `**/clinic/**` subdirectory; those are generated by argument clinic +* Unittests for everything live in the `Lib/test/` tree. Ex: + * `Lib/zipfile/` contains the code for the stdlib `zipfile` module + * `Lib/test/test_zipfile**` are tests for `zipfile` + * `Modules/_csv.c` contains the code for the stdlib `csv` module + * `Lib/test/test_csv.py` are tests for `csv` +* C header files are in the `Include/` tree +* Documentation is written in .rst format in `Doc/` - this is source for the public facing official Python documentation. +* CPython internals are documented for maintainers in @InternalDocs/README.md. +* Build time tools such as Argument Clinic live under the `Tools/` tree + +## Coding style + +* For C, follow PEP-7. For Python, follow PEP-8. +* Be consistent with existing nearby code style unless asked to do otherwise. +* NEVER leave trailing whitespace on any line. +* ALWAYS preserve the newline at the end of files. +* We do not autoformat code in this codebase. If the user asks you to run ruff format on a specific file, it can be found in `Doc/venv/bin/ruff`. +* NEVER add Python type annotations to Python code in the `Lib/` tree. + +## Building + +ONLY build in a `build/` subdirectory that you create at the repo root. + +* Use sub-agents when running configure and make build steps +* `REPO_ROOT` is the root of the cpython git repo +* let `BUILD_DIR=REPO_ROOT/build` +* Setup: `cd BUILD_DIR && ../configure --with-pydebug` +* `make -C BUILD_DIR -j $(nproc)` will rebuild +* Check what OS you are running on. Let `BUILT_PY=BUILD_DIR/python` or `BUILD_DIR/python.exe` on macOS. +* If you are on Windows, ask the user how to build. + +## Running our built Python and tests + +* ALWAYS use sub-agents when running tests +* NEVER use `pytest`. CPython tests are `unittest` based. +* use `BUILT_PY` to run the interpreter we've built +* Individual test files can be run directly using `BUILT_PY Lib/test/test_csv.py` +* `BUILT_PY -m test test_zipfile -j $(nproc)` will properly run all `Lib/test/test_zipfile` tests +* `BUILT_PY -m test` supports a `--match TESTNAME_GLOB` flag to run specific tests, pass `--help` to learn its other capabilities. NEVER try to pass `-k` as this is not pytest based, use `--match` instead. +* `make -C BUILD_DIR clinic` will regenerate argument clinic generated code. Do this after you've edited a corresponding input .c file in a way that changes a C extension module function signature or docstring +* `make -C BUILD_DIR test` will run the entire test suite. Do that sparingly. Focus on specific tests first and ask before running the entire test suite. +* Some tests are packages rather than a single .py file. These require `load_tests()` logic in their `test_package/__init__.py` file in order to work via `BUILT_PY -m test` commands. +* To collect Python code coverage from part of the test suite, use `BUILD_PY -m test -j $(nproc) --coverage test_name --coveragedir .claude/coverage_dir/`; this uses a `trace` based mechanism as implemented using libregrtest. + +### Debugging + +* For interactive debugging (pdb, lldb, or gdb) or when working on an interactive feature such as the REPL: control a tmux session to drive that. + +## Scratch space + +* NEVER create throw away idea exploration files in the top directory of the repo. Use a `.claude/sandbox/` directory for those. They will never be committed. + +## Linting + +* `pre-commit run --all-files` +* Run these steps from the repo root: + * `make -C BUILD_DIR patchcheck` should be made to pass before committing. + * For public documentation changes `make -C Doc check` + From 552db72d4aed0b1443072a6e9a0480567296e0e9 Mon Sep 17 00:00:00 2001 From: "Gregory P. Smith" Date: Sun, 28 Sep 2025 03:52:25 +0000 Subject: [PATCH 2/4] some self improvements suggested by Claude --- CLAUDE.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index ff5411675813b8..7f811b95fd2a94 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -57,12 +57,12 @@ ONLY build in a `build/` subdirectory that you create at the repo root. * let `BUILD_DIR=REPO_ROOT/build` * Setup: `cd BUILD_DIR && ../configure --with-pydebug` * `make -C BUILD_DIR -j $(nproc)` will rebuild -* Check what OS you are running on. Let `BUILT_PY=BUILD_DIR/python` or `BUILD_DIR/python.exe` on macOS. +* Check what OS you are running on. Let `BUILT_PY=BUILD_DIR/python` or `BUILD_DIR/python.exe` on macOS (.exe is used to avoid a case insensitive fs name conflict) * If you are on Windows, ask the user how to build. ## Running our built Python and tests -* ALWAYS use sub-agents when running tests +* ALWAYS use sub-agents when running builds or tests * NEVER use `pytest`. CPython tests are `unittest` based. * use `BUILT_PY` to run the interpreter we've built * Individual test files can be run directly using `BUILT_PY Lib/test/test_csv.py` @@ -71,7 +71,14 @@ ONLY build in a `build/` subdirectory that you create at the repo root. * `make -C BUILD_DIR clinic` will regenerate argument clinic generated code. Do this after you've edited a corresponding input .c file in a way that changes a C extension module function signature or docstring * `make -C BUILD_DIR test` will run the entire test suite. Do that sparingly. Focus on specific tests first and ask before running the entire test suite. * Some tests are packages rather than a single .py file. These require `load_tests()` logic in their `test_package/__init__.py` file in order to work via `BUILT_PY -m test` commands. -* To collect Python code coverage from part of the test suite, use `BUILD_PY -m test -j $(nproc) --coverage test_name --coveragedir .claude/coverage_dir/`; this uses a `trace` based mechanism as implemented using libregrtest. +* To collect Python code coverage from part of the test suite, use `BUILT_PY -m test -j $(nproc) --coverage test_name --coveragedir .claude/coverage_dir/`; this uses a `trace` based mechanism as implemented using libregrtest. + +## Common workflows + +* After editing C code: `make -C BUILD_DIR && BUILT_PY -m test relevant_tests` +* After editing stdlib Python: `BUILT_PY -m test relevant_tests --match specific_test_name_glob` (no rebuild needed) +* After editing .rst documentation: `make -C BUILD_DIR/Doc check` +* Before committing: `make -C BUILD_DIR patchcheck && pre-commit run --all-files` ### Debugging @@ -81,10 +88,3 @@ ONLY build in a `build/` subdirectory that you create at the repo root. * NEVER create throw away idea exploration files in the top directory of the repo. Use a `.claude/sandbox/` directory for those. They will never be committed. -## Linting - -* `pre-commit run --all-files` -* Run these steps from the repo root: - * `make -C BUILD_DIR patchcheck` should be made to pass before committing. - * For public documentation changes `make -C Doc check` - From 7a41d9422f359da1833373e769c518454aae491a Mon Sep 17 00:00:00 2001 From: "Gregory P. Smith" Date: Mon, 29 Sep 2025 03:21:00 +0000 Subject: [PATCH 3/4] rename to AGENTS.md and .agents/ for local state --- .gitignore | 2 ++ AGENTS.md | 90 +++++++++++++++++++++++++++++++++++++++++++++++++++++ CLAUDE.md | 91 ++---------------------------------------------------- 3 files changed, 94 insertions(+), 89 deletions(-) create mode 100644 AGENTS.md diff --git a/.gitignore b/.gitignore index 2bf4925647ddcd..4ea84072e5e3a3 100644 --- a/.gitignore +++ b/.gitignore @@ -174,6 +174,8 @@ Python/frozen_modules/MANIFEST # People's custom https://docs.anthropic.com/en/docs/claude-code/memory configs. /.claude/ CLAUDE.local.md +# Our AGENTS.md tells agents to store local notebooks and temporary files here. +/.agents/ #### main branch only stuff below this line, things to backport go above. #### # main branch only: ABI files are not checked/maintained. diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000000000..0e37ee7bdadc91 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,90 @@ +# Context + +You are in the CPython repo helping work on the implementation of the Python +language runtime and standard library itself. + +Use the `gh` tool to get information about an issue or PR in the repo. +Including using the GraphQL API via `gh` when appropriate. + +Source files in this repo can be very long. Check their size to consider if +you really need to read the entire thing. + +If tools such as `rg` (ripgrep), `gh`, `jq`, or `pre-commit` are not found, ask +the user to install them. ALWAYS prefer using `rg` rather than `find` or `grep`. + +# Expanding your knowledge + +* ALWAYS load a `.agents/pr-{PR_NUMBER}.md` or `.agents/branch-{branch_name_without_slashes}.md` file when you are told a PR number or when the current git branch is not `main`. +* Keep the file up to date as an engineering notebook of your learnings and project state as you work on a task and after you commit. The goal of our notebook is to make it easier to pick up and resume work later. + +## Optional developer guides and PEPs + +* If `REPO_ROOT/../devguide/` exists, its `developer-workflow/` and `documentation/` subdirs contain detailed info on how we like to work on code and docs. +* PEPs might exist in a `REPO_ROOT/../peps/` tree. + +# Source code + +* The runtime is implemented in C as are many of the core types and extension modules +* The Python standard library (stdlib) itself lives in the `Lib/` tree +* stdlib C extension modules live in the `Modules/` Tree +* builtins, objects, and the runtime itself live in `Objects/` and `Python/` +* NEVER edit files in a `**/clinic/**` subdirectory; those are generated by argument clinic +* Unittests for everything live in the `Lib/test/` tree. Ex: + * `Lib/zipfile/` contains the code for the stdlib `zipfile` module + * `Lib/test/test_zipfile**` are tests for `zipfile` + * `Modules/_csv.c` contains the code for the stdlib `csv` module + * `Lib/test/test_csv.py` are tests for `csv` +* C header files are in the `Include/` tree +* Documentation is written in .rst format in `Doc/` - this is source for the public facing official Python documentation. +* CPython internals are documented for maintainers in @InternalDocs/README.md. +* Build time tools such as Argument Clinic live under the `Tools/` tree + +## Coding style + +* For C, follow PEP-7. For Python, follow PEP-8. +* Be consistent with existing nearby code style unless asked to do otherwise. +* NEVER leave trailing whitespace on any line. +* ALWAYS preserve the newline at the end of files. +* We do not autoformat code in this codebase. If the user asks you to run ruff format on a specific file, it can be found in `Doc/venv/bin/ruff`. +* NEVER add Python type annotations to Python code in the `Lib/` tree. + +## Building + +ONLY build in a `build/` subdirectory that you create at the repo root. + +* Use sub-agents when running configure and make build steps +* `REPO_ROOT` is the root of the cpython git repo +* let `BUILD_DIR=REPO_ROOT/build` +* Setup: `cd BUILD_DIR && ../configure --with-pydebug` +* `make -C BUILD_DIR -j $(nproc)` will rebuild +* Check what OS you are running on. Let `BUILT_PY=BUILD_DIR/python` or `BUILD_DIR/python.exe` on macOS (.exe is used to avoid a case insensitive fs name conflict) +* If you are on Windows, ask the user how to build. + +## Running our built Python and tests + +* ALWAYS use sub-agents when running builds or tests +* NEVER use `pytest`. CPython tests are `unittest` based. +* use `BUILT_PY` to run the interpreter we've built +* Individual test files can be run directly using `BUILT_PY Lib/test/test_csv.py` +* `BUILT_PY -m test test_zipfile -j $(nproc)` will properly run all `Lib/test/test_zipfile` tests +* `BUILT_PY -m test` supports a `--match TESTNAME_GLOB` flag to run specific tests, pass `--help` to learn its other capabilities. NEVER try to pass `-k` as this is not pytest based, use `--match` instead. +* `make -C BUILD_DIR clinic` will regenerate argument clinic generated code. Do this after you've edited a corresponding input .c file in a way that changes a C extension module function signature or docstring +* `make -C BUILD_DIR test` will run the entire test suite. Do that sparingly. Focus on specific tests first and ask before running the entire test suite. +* Some tests are packages rather than a single .py file. These require `load_tests()` logic in their `test_package/__init__.py` file in order to work via `BUILT_PY -m test` commands. +* To collect Python code coverage from part of the test suite, use `BUILT_PY -m test -j $(nproc) --coverage test_name --coveragedir .agents/coverage_dir/`; this uses a `trace` based mechanism as implemented using libregrtest. + +## Common workflows + +* After editing C code: `make -C BUILD_DIR && BUILT_PY -m test relevant_tests` +* After editing stdlib Python: `BUILT_PY -m test relevant_tests --match specific_test_name_glob` (no rebuild needed) +* After editing .rst documentation: `make -C BUILD_DIR/Doc check` +* Before committing: `make -C BUILD_DIR patchcheck && pre-commit run --all-files` + +### Debugging + +* For interactive debugging (pdb, lldb, or gdb) or when working on an interactive feature such as the REPL: control a tmux session to drive that. + +## Scratch space + +* NEVER create throw away idea exploration files in the top directory of the repo. Use a `.agents/sandbox/` directory for those. They will never be committed. + diff --git a/CLAUDE.md b/CLAUDE.md index 7f811b95fd2a94..8eed6afa6c6dae 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,90 +1,3 @@ -# Context - -You are in the CPython repo helping work on the implementation of the Python -language runtime and standard library itself. - -Use the `gh` tool to get information about an issue or PR in the repo. -Including using the GraphQL API via `gh` when appropriate. - -Source files in this repo can be very long. Check their size to consider if -you really need to read the entire thing. - -If tools such as `rg` (ripgrep), `gh`, `jq`, or `pre-commit` are not found, ask -the user to install them. ALWAYS prefer using `rg` rather than `find` or `grep`. - -# Expanding your knowledge - -* ALWAYS load a `.claude/pr-{PR_NUMBER}.md` or `.claude/branch-{branch_name_without_slashes}.md` file when you are told a PR number or when the current git branch is not `main`. -* Keep the file up to date as an engineering notebook of your learnings and project state as you work on a task and after you commit. The goal of our notebook is to make it easier to pick up and resume work later. - -## Optional developer guides and PEPs - -* If `REPO_ROOT/../devguide/` exists, its `developer-workflow/` and `documentation/` subdirs contain detailed info on how we like to work on code and docs. -* PEPs might exist in a `REPO_ROOT/../peps/` tree. - -# Source code - -* The runtime is implemented in C as are many of the core types and extension modules -* The Python standard library (stdlib) itself lives in the `Lib/` tree -* stdlib C extension modules live in the `Modules/` Tree -* builtins, objects, and the runtime itself live in `Objects/` and `Python/` -* NEVER edit files in a `**/clinic/**` subdirectory; those are generated by argument clinic -* Unittests for everything live in the `Lib/test/` tree. Ex: - * `Lib/zipfile/` contains the code for the stdlib `zipfile` module - * `Lib/test/test_zipfile**` are tests for `zipfile` - * `Modules/_csv.c` contains the code for the stdlib `csv` module - * `Lib/test/test_csv.py` are tests for `csv` -* C header files are in the `Include/` tree -* Documentation is written in .rst format in `Doc/` - this is source for the public facing official Python documentation. -* CPython internals are documented for maintainers in @InternalDocs/README.md. -* Build time tools such as Argument Clinic live under the `Tools/` tree - -## Coding style - -* For C, follow PEP-7. For Python, follow PEP-8. -* Be consistent with existing nearby code style unless asked to do otherwise. -* NEVER leave trailing whitespace on any line. -* ALWAYS preserve the newline at the end of files. -* We do not autoformat code in this codebase. If the user asks you to run ruff format on a specific file, it can be found in `Doc/venv/bin/ruff`. -* NEVER add Python type annotations to Python code in the `Lib/` tree. - -## Building - -ONLY build in a `build/` subdirectory that you create at the repo root. - -* Use sub-agents when running configure and make build steps -* `REPO_ROOT` is the root of the cpython git repo -* let `BUILD_DIR=REPO_ROOT/build` -* Setup: `cd BUILD_DIR && ../configure --with-pydebug` -* `make -C BUILD_DIR -j $(nproc)` will rebuild -* Check what OS you are running on. Let `BUILT_PY=BUILD_DIR/python` or `BUILD_DIR/python.exe` on macOS (.exe is used to avoid a case insensitive fs name conflict) -* If you are on Windows, ask the user how to build. - -## Running our built Python and tests - -* ALWAYS use sub-agents when running builds or tests -* NEVER use `pytest`. CPython tests are `unittest` based. -* use `BUILT_PY` to run the interpreter we've built -* Individual test files can be run directly using `BUILT_PY Lib/test/test_csv.py` -* `BUILT_PY -m test test_zipfile -j $(nproc)` will properly run all `Lib/test/test_zipfile` tests -* `BUILT_PY -m test` supports a `--match TESTNAME_GLOB` flag to run specific tests, pass `--help` to learn its other capabilities. NEVER try to pass `-k` as this is not pytest based, use `--match` instead. -* `make -C BUILD_DIR clinic` will regenerate argument clinic generated code. Do this after you've edited a corresponding input .c file in a way that changes a C extension module function signature or docstring -* `make -C BUILD_DIR test` will run the entire test suite. Do that sparingly. Focus on specific tests first and ask before running the entire test suite. -* Some tests are packages rather than a single .py file. These require `load_tests()` logic in their `test_package/__init__.py` file in order to work via `BUILT_PY -m test` commands. -* To collect Python code coverage from part of the test suite, use `BUILT_PY -m test -j $(nproc) --coverage test_name --coveragedir .claude/coverage_dir/`; this uses a `trace` based mechanism as implemented using libregrtest. - -## Common workflows - -* After editing C code: `make -C BUILD_DIR && BUILT_PY -m test relevant_tests` -* After editing stdlib Python: `BUILT_PY -m test relevant_tests --match specific_test_name_glob` (no rebuild needed) -* After editing .rst documentation: `make -C BUILD_DIR/Doc check` -* Before committing: `make -C BUILD_DIR patchcheck && pre-commit run --all-files` - -### Debugging - -* For interactive debugging (pdb, lldb, or gdb) or when working on an interactive feature such as the REPL: control a tmux session to drive that. - -## Scratch space - -* NEVER create throw away idea exploration files in the top directory of the repo. Use a `.claude/sandbox/` directory for those. They will never be committed. +@AGENTS.md +Keep agent instructions in AGENTS.md files instead of CLAUDE.md From e8a8f39971493fc6f7009e188eeb9921cc74b115 Mon Sep 17 00:00:00 2001 From: "Gregory P. Smith" Date: Mon, 29 Sep 2025 04:16:32 +0000 Subject: [PATCH 4/4] Add explanation & remind people of responsible use Simplifies a couple other less important statements. (no need to mention graphql and the tmux debugging story works but isn't widely used or well fleshed out yet) --- AGENTS.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 0e37ee7bdadc91..33accfc2b21bab 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,10 +1,14 @@ +# Purpose + +* These brief instructions help LLM agents navigate working in the cpython repo. +* **Humans** are always responsible for changes being proposed and must pre-review all agentic work before turning it into a PR. + # Context You are in the CPython repo helping work on the implementation of the Python language runtime and standard library itself. Use the `gh` tool to get information about an issue or PR in the repo. -Including using the GraphQL API via `gh` when appropriate. Source files in this repo can be very long. Check their size to consider if you really need to read the entire thing. @@ -80,10 +84,6 @@ ONLY build in a `build/` subdirectory that you create at the repo root. * After editing .rst documentation: `make -C BUILD_DIR/Doc check` * Before committing: `make -C BUILD_DIR patchcheck && pre-commit run --all-files` -### Debugging - -* For interactive debugging (pdb, lldb, or gdb) or when working on an interactive feature such as the REPL: control a tmux session to drive that. - ## Scratch space * NEVER create throw away idea exploration files in the top directory of the repo. Use a `.agents/sandbox/` directory for those. They will never be committed.