ast_serialize - Python Parser and AST Serializer

This is a fast Python extension for parsing Python files and serializing the AST using the native binary format used by mypy. This will eventually replace the current mypy parser, which uses the Python stdlib ast module for parsing.

This is work in progress.

Development

Prerequisites:

• Recent Rust toolchain
• Python 3.10+ (3.13t or 3.14t for free-threaded builds)
• maturin: pip install maturin
• Access to mypy new-parser branch for testing

You must use the new-parser branch in the mypy repository to use this with mypy.

Development build (fast compilation, unoptimized):

maturin develop

Optimized development build:

maturin develop --release

Testing

Rust unit tests:

cargo test

Python integration tests: Run end-to-end parser and serialization/deserialization tests using mypy's test suite in the new-parser branch:

cd ~/src/mypy  [or wherever you have mypy]
pytest mypy -k NativeParser

Add new test cases to test-data/unit/native-parser.test (in the mypy repository). See the main parser test cases in parse.test for the expected output format.

Note: Run maturin develop before testing if you've modified Rust code.

Use mypy/test/testcheck.py in the new-parser branch to run mypy type checking tests using ast_serialize. The test runner enables the new parser by default. Note that many tests are still failing.

Creating PRs

You can create PRs in this repository, or you can target the new-parser mypy branch. Contributions are welcome! Run mypy tests (see above) to get ideas about bugs and missing functionality. If your contributions needs changes in both mypy and ast_serialize, please mention this in the PR summary.

Building Wheels

# Build all wheels (stable ABI + free-threaded)
./build_wheels.sh

# Or manually:
# Stable ABI wheel (Python 3.9+)
maturin build --release

# Free-threaded wheel (Python 3.13t)
maturin build --release --no-default-features --features free-threaded --interpreter python3.13t

If you see Python 3.13t not found, you'll need to install the free-threaded build of CPython 3.13 in PATH.

Using Coding Agents

This project is designed to support coding agent assisted development (such as Claude Code, Codex or OpenCode). Notes:

• Ensure your coding agent has access to AGENTS.md (e.g. create a CLAUDE.md symbolic link).
• For the best experience, clone this repository as ~/src/ast_serialize, and clone mypy (with the new-parser branch checked out) as ~/src/mypy.
• Have your mypy virtualenv active when starting the coding agent, or place the virtualenv at ~/venv/mypy (this is referred to in AGENTS.md).

Acknowledgments

This is a wrapper around the Ruff parser. Credits to Ruff maintainers for developing a fast Python parser!

License

MIT