Skip to content

stop scoring the fuzzer's own noise as target crashes - #259

Merged
devdanzin merged 1 commit into
mainfrom
fuzzer-noise-signatures
Aug 18, 2026
Merged

stop scoring the fuzzer's own noise as target crashes#259
devdanzin merged 1 commit into
mainfrom
fuzzer-noise-signatures

Conversation

@devdanzin

Copy link
Copy Markdown
Owner

Two independent false-positive sources, both measured on a PyPy 3.11 stdlib fleet
(fusil-pypy311_fleet_02, 255 kept dirs). Together they account for 23 of 255 (9%)
every one a session the fuzzer killed itself.

1. Three bomb signatures missing from the stdout ignore regex

The regex has been extended once per bomb family, but three raise sites were added later
without it:

signature raised at covered?
fusil instancecheck bomb_objects.py:685 (metaclass bomb)
fusil monitoring callback bomb write_python_code.py:1807
fusil junk return write_python_code.py:1811

instancecheck is raised as SystemError — a 1.0 word, so this isn't a cosmetic gap: it
manufactures crashes. 7 sessions kept in that one fleet.

test_ignore_regex_covers_every_raised_bomb_signature scrapes the raise sites and fails on
any signature the regex misses, so the next bomb family can't repeat this. Verified: with the
three alternatives removed it reports exactly those three.

2. default_int_handler kills the session

signal.default_int_handler / _signal.default_int_handler raise KeyboardInterrupt — a
BaseException, so it blows straight through the generated script's except Exception
handlers and takes the session with it. That's the #192 class.

Called directly as a fuzz target it tagged 16 dirs -sigint in this fleet, and the
rustpython fleets show the same pattern. It exists on every interpreter, so this is not
PyPy-specific. Its neighbours (pause/alarm/setitimer/pthread_kill) were already
blacklisted for exactly this reason — this one was just missed.

Full suite green (1254); ruff clean.

🤖 Generated with Claude Code

Two independent false-positive sources, both measured on a PyPy 3.11 stdlib fleet
(fusil-pypy311_fleet_02, 255 kept dirs):

1. Three bomb signatures were never added to the stdout ignore regex. The regex has been
   extended once per bomb family, but `instancecheck` (the metaclass bomb) and the
   monitoring-callback bomb's `junk return` / `monitoring callback bomb` were added later
   without it. `instancecheck` is raised as SystemError -- a 1.0 word -- so the gap does
   not merely add noise, it manufactures crashes: 7 sessions kept in that one fleet.

2. signal.default_int_handler / _signal.default_int_handler raise KeyboardInterrupt, a
   BaseException, which blows straight through the generated script's `except Exception`
   handlers and kills the session (the #192 class). Called directly as a fuzz target it
   tagged 16 dirs `-sigint`; the rustpython fleets show the same, and the function exists
   on every interpreter. Its neighbours (pause/alarm/setitimer/pthread_kill) were already
   blacklisted for the same reason.

Together these accounted for 23 of 255 kept dirs (9%) in that fleet, every one a session
the fuzzer killed itself.

test_ignore_regex_covers_every_raised_bomb_signature scrapes the raise sites and fails on
any signature the regex misses, so the next bomb family cannot repeat this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WhcpLoyjUWLbETGZnA9boj
@devdanzin
devdanzin merged commit bddcd40 into main Aug 18, 2026
1 of 2 checks passed
@devdanzin
devdanzin deleted the fuzzer-noise-signatures branch August 18, 2026 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant