Skip to content

v0.4.11

Choose a tag to compare

@github-actions github-actions released this 05 Oct 13:15
· 4 commits to main since this release
v0.4.11
6b2cc8d

proofpath update keeps proofpath current, and it keeps the extras you installed. A
judge that hits Groq's per-minute limit now returns to Groq once the limit resets,
instead of staying on the local model for the rest of the run. The AVeriTeC numbers
were measured again, with the browser and the judge: search with judge-written queries
scores 0.483, the one gain that cleared the noise. That is still below the 0.708
baseline, and the report now says how often search calls a false claim supported.

Added

  • AVeriTeC eval: --browser, --judge and --fresh (scripts/eval_averitec.py).
    Every source now records the ladder step that read it and, under --judge, the
    judge's opinion when its verdict was in the escalation band. The report adds two
    numbers beside the product's: the accuracy with the browser's pages dropped, and a
    hypothetical one in which the judge decides the low band. The product never does the
    second, and the report says so. Each run setup gets its own results file and report.
    Measured on 2026-10-05: gold URLs 0.371; search with sentence queries 0.371; search
    with the judge writing the queries 0.483 (14 claims fixed, 4 broken, p = 0.031). The
    majority baseline is 0.708.
  • proofpath update installs the latest release from PyPI, and
    proofpath update --check only says whether there is one. In the TUI they are
    /update and /update --check. Running the command is the consent, so nothing
    asks. uv tool, pipx and pip installs are each updated with their own tool; pip
    falls back to uv pip when it fails and uv is on PATH. An editable install is
    never touched: the command names its source directory and says to update it with
    git pull and uv sync.
  • The installed extras are kept. They are written into the install spec
    (proofpath[browser]), never with a version pin. A plain uv tool upgrade was
    measured to drop packages added to the tool's environment after it was installed,
    which would have removed the browser wheels you had consented to.
  • The new version is read back from a fresh process. An update the resolver held
    back below the latest version is a failure that names the version it installed.
    On Windows a failed command adds "close proofpath and run the command above",
    since a running proofpath.exe can be locked.
  • Temporary environments are left alone. In a uvx or pipx run environment
    nothing runs. The command says that uvx proofpath@latest or
    pipx run --no-cache proofpath already fetches the newest.
  • Commands paste back on every platform. Each command is logged as $ <cmd>; the
    one printed for running by hand has no $. On Windows both use cmd.exe quoting,
    and so does the browser install's log.
    The version check after the update runs the interpreter isolated (-I), so a
    checkout in the working directory cannot answer for the install.
  • In the TUI, an update and a run never overlap. A bare /update is refused while
    a run or a mirrored verb is still going, because the install replaces the code they
    are executing; /update --check is not refused. While an update is installing,
    every /check and mirrored verb is refused until it finishes, including a second
    /update.
  • Exit codes: 0 for up to date, updated, ahead of PyPI, editable, temporary, or
    (with --check) an update available. 1 means the check or the update failed.
    permissions.network is read by the fetch ladder's own rule: deny, and ask
    without a terminal, send nothing to PyPI and fail rather than report "up to date".
    ask in a terminal goes ahead with a note.

Changed

  • At 80x24 the / list now caps. /update makes eleven verbs and 80x24 has
    room for ten rows. The last two, /quit and /update, sit behind ... 2 more.
    update is last on purpose: the banner's hint line already names /quit, and
    /help stays visible. One more row, at 80x25, shows them all.
  • packaging is a declared dependency. It was already installed through other
    packages; update now imports it directly.
  • The judge goes back to Groq once its minute limit resets. A 429 whose
    Retry-After is at most 120 seconds (60 when it names none) now switches to the
    local model only until that wait has passed: Groq limit reached — judging with local ollama qwen3.5:9b until it resets (~30s). The first request after it asks
    Groq again, from one thread only; if Groq answers, the judge stays on it and says
    so once (Groq answering again — local ollama qwen3.5:9b stood in for 4 requests),
    on stderr, in the TUI and in the report's footer. Another short 429 starts a new
    wait, quietly. Nobody waits: the local model answers while Groq cools down. A daily
    limit (a longer Retry-After), a 413, a rejected key, a 5xx or no answer still
    switches for the rest of the run, with the same notice as before.