Skip to content

Fuzzing baseline blockers: libpng libpng16 pin, experiment-runner Rich crash, dead fuzzing CLI #148

Description

@Ella-Hoeppner

Surfaced while validating the fuzzing pipeline before the baseline sweep (haiku).

1. libpng Docker build broken (pin libpng16) — FIX INCLUDED

targets/libpng/Dockerfile did git clone …/libpng with no branch, pulling libpng18; build.sh looks for libpng16.aERROR: could not find libpng static library. Fixed by pinning git clone -b libpng16 (the documented CLAUDE.md gotcha).

2. Experiment runner crashes on teardown in non-TTY — FIX INCLUDED

Background/piped runs end with IndexError: pop from empty list in rich Progress.__exit__ → Live.stop → console.clear_live. Inspect-AI drives its own Rich Live per eval inside the runner's Progress loop; in non-interactive mode the shared console's live stack underflows on teardown. Non-fatal (per-run manifest + per-sample Logfire export both happen before it), but it prints a traceback and skips the summary table every run. Fixed by disable=not sys.stdout.isatty() on the runner's Progress (makes Rich skip live.stop()); per-run log lines still report progress.

3. fuzzing CLI command is dead — NOT fixed here (follow-up)

parser-security-eval fuzzing calls live_fuzzing(engine=…, max_rounds=…, round_duration=…), but the task signature is live_fuzzing(targets_dir, target, fuzzing_engine, sanitizer, fuzz_duration, message_limit)TypeError: unexpected keyword argument 'engine'. The CLI was never updated when the task signature changed. The experiment runner calls live_fuzzing correctly, so the sweep path is unaffected; this is the direct-CLI convenience path only. Left as a follow-up so the fix PR stays focused on the two run-critical blockers (the right mapping of max_rounds/round_duration → fuzz_duration/message_limit needs a small design call).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions