RustPython fuzzing: generic hostile modes + panic-site dedup (core) - #226
Merged
Conversation
RustPython fleets converge fast on the same ~9 panics/segfaults, and neither
--tsan (Rust's Arc/Send/Sync prevent C-style races) nor --oom (Rust OOM aborts)
applies. Add four generic, alt-interpreter-friendly capabilities in core fusil
(the RustPython packaging will live in a separate plugin), and a dedup catalog so
fleets surface only NEW sites.
New modes / options (fusil/python/__init__.py, write_python_code.py, python_source.py):
- --concurrency-stress: emit the --tsan shared-object/many-thread stress region
WITHOUT a TSan build/preflight (reuses --tsan-threads/-iterations/-shared-objects).
The GIL guard is hard only under --tsan; under --concurrency-stress it warns and
runs serialised (threading teardown/re-entrancy bugs surface via context switches
-- this is the class that produced the RustPython _thread RefCell double-borrow).
- --new-uninit: for each type T discovered in the session's module(s) -- including
hidden RESULT types reached via a bounded call sweep (e.g. type(re.match('a','a')),
never a module attribute) -- build obj = T.__new__(T) and hammer its PROTOCOL slots
(subscript, iter/next, call, number, comparison, buffer, context-manager) plus every
dir() method with hostile args. Reproduces the re.Match __new__ subscript segfault
from the mode alone.
- --modules-file FILE: read a curated module list (one per line, '#' comments),
bypassing discovery/blacklists; unions with --modules.
- RUST_BACKTRACE=1 in the target env (harmless no-op for CPython/PyPy): richer panic
frames in the crash dir for dedup + reports.
Dedup engine (fusil/python/rustpython_dedup.py, mirrors tsan_dedup/oom_dedup):
- parse_report -> panic-site signature crates/<path>.rs:<line> (drop the column;
normalise absolute checkout paths to the crates/ tail; handle the newer
'thread NAME (tid) panicked' form and worker panics printed mid-stdout).
- RustPyDeduper.decide -> (keep, label): catalog hit -> RUSTPY-00NN, new panic ->
rustpyNEW, bare segfault -> rustpySEGV (or a gdb-resolved top frame via an injectable
resolver). Wired via --rustpython-dedup-catalog/-keep/-prune on the existing
application.session_keep_policy hook.
Pure-Python engine unit-tested in tests/python/test_rustpython_dedup.py (23 tests).
The two new mode flags are set False in test_oom_fuzz's MagicMock fixture (a bare
Mock attr is truthy and would divert generation into the stress region, same hazard
the existing o.tsan = False guards). Verified end-to-end against RustPython 0.5.0:
--new-uninit reproduces the re.Match segfault and (bonus) a new structseq __new__
panic face; the dedup keep-policy labels crash dirs rustpyNEW / RUSTPY-id. ruff +
1168-test suite green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The first two RustPython fleets converged fast on the same ~9 crashes, and neither
--tsan(Rust'sArc/Send/Syncprevent the C-style races TSan hunts) nor--oom(Rust OOM aborts rather than exposing continuable error paths) applies. Every RustPython finding is instead a.unwrap()/.expect()/panic!/unchecked-index reachable from Python, or unguarded native recursion — all in the ~96 Rust-implemented modules.This PR adds four generic hostile capabilities (they help any alternative interpreter — RustPython, PyPy) in core fusil, plus an in-loop dedup catalog so fleets keep only NEW panic sites. RustPython-specific packaging (curated module list, known-panic suppression) will land in a separate
fusil_rustpython_plugin.What
New modes / options
--concurrency-stress— emit the--tsanshared-object/many-thread stress region without a TSan build or the free-threaded/--with-thread-sanitizerpreflight (reuses--tsan-threads/-iterations/-shared-objects). The GIL guard is hard only under--tsan; under--concurrency-stressit warns and runs serialised — threading teardown/re-entrancy bugs surface via context switches (the class that produced the RustPython_threadRefCelldouble-borrow).--new-uninit— for each typeTdiscovered in the session's module(s), including hidden result types reached via a bounded call sweep (e.g.type(re.match('a','a')), never a module attribute), buildobj = T.__new__(T)and hammer its protocol slots (subscript, iter/next, call, number, comparison, buffer, context-manager) plus everydir()method with hostile args. Protocol slots that touch payload without re-validating segfault a type-confused instance where ordinary methods downcast-fail cleanly.--modules-file FILE— read a curated module list (one per line,#comments), bypassing discovery/blacklists; unions with--modules.RUST_BACKTRACE=1in the target env (harmless no-op for CPython/PyPy) for richer panic frames.Dedup engine —
fusil/python/rustpython_dedup.py, mirroringtsan_dedup.py/oom_dedup.py:parse_report→ panic-site signaturecrates/<path>.rs:<line>(drops the column, normalises absolute checkout paths to thecrates/tail, handles the newerthread 'NAME' (tid) panickedform and worker panics printed mid-stdout).RustPyDeduper.decide→(keep, label): catalog hit →RUSTPY-00NN, new panic →rustpyNEW, bare segfault →rustpySEGV(or a gdb-resolved top frame via an injectable resolver). Wired via--rustpython-dedup-catalog/-keep/-pruneon the existingapplication.session_keep_policyhook.Testing
tests/python/test_rustpython_dedup.py(23 tests).Falseintest_oom_fuzz'sMagicMockfixture (a bare Mock attr is truthy and would divert generation into the stress region — the same hazard the existingo.tsan = Falseguards).--new-uninitreproduces there.Match.__new__subscript segfault and a newstructseq.__new__panic face;--concurrency-stressruns the stress region on RustPython with no TSan preflight; the dedup keep-policy labels crash dirsrustpyNEW/RUSTPY-id.ruff check+ruff format --checkclean; fullunittestsuite (1168 tests) green.Non-
--tsan/non---new-uninitoutput is unchanged (both are gated early-return branches).🤖 Generated with Claude Code