v0.4.12

The judge now has to quote the passage it judges, and an opinion whose quote is not
there is dropped and said to be. The check is exact at word and number boundaries, so
"2 million" is not found in "1.2 million". The AVeriTeC eval can now show the judge
the NEIs too. It fixed 5 claims and broke none, but at p = 0.062 that is short of the
bar set beforehand, so the product does not do it.

Changed

• The judge's quote is checked, and shown. A SUPPORTED or REFUTED opinion now
  has to return the exact words from the passage that decide it, in a new quote
  field. An opinion is dropped, with a note saying why, when its quote has no words,
  when a part of it is shorter than two words, or when it is not in the passage it was
  shown at word boundaries ("5 million" is not in "15 million"). The match folds case,
  curly quotes, dashes and the minus sign, spaces, the fraction slash and invisible
  characters such as the soft hyphen, and nothing else: a superscript stays a
  superscript, and a sign or a percent is never trimmed. ... may elide the middle of
  a quote as long as every part is in the passage, in order. The verified quote is
  printed in the judge line, judge (<model>): SUPPORTED — "<quote>" — <rationale>,
  and JSON gains a quote field. An NEI opinion asserts nothing and may quote
  nothing. Cached opinions are now keyed by the passage they were checked against, and
  those cached before this change are not served again: the next run asks.

Added

• AVeriTeC eval: --judge-nei and --no-fallback (scripts/eval_averitec.py).
  --judge-nei (requires --judge) also sends every read source whose verdict is NEI
  to the judge, with the passage the models came closest to deciding on, and the
  hypothetical column gains a second number in which those opinions vote too, with its
  per-label counts, its wrong-way count beside the product's, and an exact McNemar test
  against the product claim by claim. The report also counts opinions by model and kind
  of escalation, and the ones the quote check dropped by reason. --no-fallback
  (requires --judge) turns the judge's local fallback off for the run, so every
  opinion is from the configured model. Both
  flags get their own results file and report name. Measurement only: the product
  still never sends an NEI to the judge.

v0.4.11

proofpath update keeps proofpath current, and it keeps the extras you installed. A
judge that hits Groq's per-minute limit now returns to Groq once the limit resets,
instead of staying on the local model for the rest of the run. The AVeriTeC numbers
were measured again, with the browser and the judge: search with judge-written queries
scores 0.483, the one gain that cleared the noise. That is still below the 0.708
baseline, and the report now says how often search calls a false claim supported.

Added

• AVeriTeC eval: --browser, --judge and --fresh (scripts/eval_averitec.py).
  Every source now records the ladder step that read it and, under --judge, the
  judge's opinion when its verdict was in the escalation band. The report adds two
  numbers beside the product's: the accuracy with the browser's pages dropped, and a
  hypothetical one in which the judge decides the low band. The product never does the
  second, and the report says so. Each run setup gets its own results file and report.
  Measured on 2026-10-05: gold URLs 0.371; search with sentence queries 0.371; search
  with the judge writing the queries 0.483 (14 claims fixed, 4 broken, p = 0.031). The
  majority baseline is 0.708.
• proofpath update installs the latest release from PyPI, and
  proofpath update --check only says whether there is one. In the TUI they are
  /update and /update --check. Running the command is the consent, so nothing
  asks. uv tool, pipx and pip installs are each updated with their own tool; pip
  falls back to uv pip when it fails and uv is on PATH. An editable install is
  never touched: the command names its source directory and says to update it with
  git pull and uv sync.
• The installed extras are kept. They are written into the install spec
  (proofpath[browser]), never with a version pin. A plain uv tool upgrade was
  measured to drop packages added to the tool's environment after it was installed,
  which would have removed the browser wheels you had consented to.
• The new version is read back from a fresh process. An update the resolver held
  back below the latest version is a failure that names the version it installed.
  On Windows a failed command adds "close proofpath and run the command above",
  since a running proofpath.exe can be locked.
• Temporary environments are left alone. In a uvx or pipx run environment
  nothing runs. The command says that uvx proofpath@latest or
  pipx run --no-cache proofpath already fetches the newest.
• Commands paste back on every platform. Each command is logged as $ <cmd>; the
  one printed for running by hand has no $. On Windows both use cmd.exe quoting,
  and so does the browser install's log.
  The version check after the update runs the interpreter isolated (-I), so a
  checkout in the working directory cannot answer for the install.
• In the TUI, an update and a run never overlap. A bare /update is refused while
  a run or a mirrored verb is still going, because the install replaces the code they
  are executing; /update --check is not refused. While an update is installing,
  every /check and mirrored verb is refused until it finishes, including a second
  /update.
• Exit codes: 0 for up to date, updated, ahead of PyPI, editable, temporary, or
  (with --check) an update available. 1 means the check or the update failed.
  permissions.network is read by the fetch ladder's own rule: deny, and ask
  without a terminal, send nothing to PyPI and fail rather than report "up to date".
  ask in a terminal goes ahead with a note.

Changed

• At 80x24 the / list now caps. /update makes eleven verbs and 80x24 has
  room for ten rows. The last two, /quit and /update, sit behind ... 2 more.
  update is last on purpose: the banner's hint line already names /quit, and
  /help stays visible. One more row, at 80x25, shows them all.
• packaging is a declared dependency. It was already installed through other
  packages; update now imports it directly.
• The judge goes back to Groq once its minute limit resets. A 429 whose
  Retry-After is at most 120 seconds (60 when it names none) now switches to the
  local model only until that wait has passed: Groq limit reached — judging with local ollama qwen3.5:9b until it resets (~30s). The first request after it asks
  Groq again, from one thread only; if Groq answers, the judge stays on it and says
  so once (Groq answering again — local ollama qwen3.5:9b stood in for 4 requests),
  on stderr, in the TUI and in the report's footer. Another short 429 starts a new
  wait, quietly. Nobody waits: the local model answers while Groq cools down. A daily
  limit (a longer Retry-After), a 413, a rejected key, a 5xx or no answer still
  switches for the rest of the run, with the same notice as before.

v0.4.10

An opt-in, larger NLI model for scientific sources, chosen by a measured bake-off and
downloaded only with your consent. Reports now admit when a model has no real low
confidence tier, and CI type-checks the whole package.

Added

• An opt-in accurate NLI model. The bake-off's winner,
  MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli, runs at k=2 with cuts
  calibrated on SciFact train. Its SciFact dev macro-F1 is 0.697, against 0.580 for
  the default model. It is not better on news: on AVeriTeC claims with a readable
  source it scores 0.475, against 0.537 for the default model at k=2. It is
  recommended for scientific sources, not for news. It is 643 MB, against 244 MB,
  and the default install does not change.
• proofpath check --accurate uses the accurate model for one run.
  [models] nli = "accurate" uses it for every run; set it with
  proofpath config set models.nli accurate or in the /config panel.
• permissions.install_model guards the 643 MB download. It is ask by
  default: the run asks once, naming the model and its size, with the browser
  question's answers (once, always, no, never), and "always" and "never" are saved.
  With no terminal, ask is treated as deny and the denial is reported. A denied
  or failed download falls back to the default model and says so in three places:
  a note with the reason and what would change it (dropped by -q, and on stderr
  under --format json or sarif); in text mode, an
  nli default model chosen; requested accurate (<reason>) line that -q keeps;
  and nli_requested: accurate (<reason>) in the report's models block. That
  block now also names the k that ran.

Changed

• A note when a run has no real low tier. A run whose medium cut sits within
  0.01 of decide now says "this model has no real low tier on SciFact, so an
  asserted verdict is almost always medium or better". Default runs say it too: the
  default's medium (0.457948) sits 0.008 above decide (0.45), which the
  2026-09-12 tier sweep already measured. When the no-high-tier note also applies,
  the two are joined with "; ".
• The no-high-tier note now reads "this model earns no high tier on SciFact dev,
  so medium is the strongest confidence shown".
• CI runs strict mypy over the whole of src/.

Measured

• An NLI bake-off (scripts/eval_nli_bakeoff.py, report in
  docs/eval/2026-10-01-nli-bakeoff.md). It compares three NLI models, three
  values of k and three aggregation rules, 27 combinations in all. Each combination
  is calibrated on SciFact train and scored on SciFact dev and a frozen AVeriTeC
  snapshot. The winner is picked by a rule written down before any number was
  seen.
  • Winner: MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli with
    k=2. SciFact dev macro-F1 is 0.697, against 0.580 for today's default and 0.608
    for the best configuration of the default model.
  • AVeriTeC stays below its majority baseline: 0.475 on claims with a readable
    source, against 0.708.
  • The report lists its own limits. The main one is that dev both chose the winner
    and reports its score.

v0.4.9

Wikipedia can be read again, and a verdict now says more about how much it rests
on: an abstract, or a claim too short to pin down. The Tavily key is pasted straight
into /config.

Added

• Tavily API: in /config. The search section of the settings panel has a key
  row: press Enter, paste the key, Enter again. It is saved to the .env in your
  config directory (mode 0600 on macOS/Linux), never to config.toml or the project's
  .env, and it is never shown: the row says only whether a key is saved and where.
  Backspace removes it. Saving a key does not turn search on: while
  search.provider is not tavily the row says the key is unused, and with
  tavily selected and no key it says one is needed.
• A note on claims too short to pin down. A claim like "openai won" is supported
  by almost any page about any win. A claim with two content words or fewer, and no
  linking verb (is, are, was, …) between them, now carries a "short claim"
  note on each of its verdict rows, SARIF included. The note tells you to check
  that the quoted passage is about the event you meant. It is a rule, not a model,
  and it changes no verdict, tier or exit code.

Fixed

• Wikipedia pages no longer read as UNVERIFIED (blocked). The bot-wall check
  matched captcha inside MediaWiki's inline configuration script, so every Wikipedia
  page was reported blocked at all four ladder steps despite answering 200. Challenge
  phrases are now matched outside <script>/<style> contents and only on small
  pages. cf-chl and Cloudflare's _cf_chl_opt still mark a wall wherever they
  appear. Other cf_chl tokens (such as __cf_chl_f_tk) count only on small
  pages, because they linger in the URLs of real articles.
• A verdict judged against an abstract now says so on its own row. A short page
  or an abstract-only source already had its LOW CONFIDENCE (abstract only) row, but
  the claim's verdict row (SUPPORTED (found by proofpath), NOT SUPPORTED, …) did
  not, so read alone it looked like a full-text verdict. It now carries "judged
  against the abstract only, not the full text", in the terminal, Markdown, JSON and
  SARIF.

v0.4.8

A text that cites nothing can now be checked: proofpath searches the web for it. And
the judge no longer goes quiet when Groq's free tier runs out.

Added

• Evidence search for claims with no source (experimental). A pasted post or plain
  text with no link, no [12] marker and no bibliography used to end in a parse error.
  With search.provider set to tavily (a free key in TAVILY_API_KEY) or searxng
  (plus search.base_url), up to 5 check-worthy sentences are searched, up to 3 pages
  each. Every page is fetched and read like any other source; the search result's own
  snippet is never evidence. Findings are marked FOUND BY PROOFPATH (not cited by the author): SUPPORTED/REFUTED (found by proofpath) carry their passage, and a claim
  nothing confirmed is NO EVIDENCE FOUND (searched), never REFUTED. Setting the
  provider is the consent; --no-search turns it off for one run and
  permissions.web_search = deny for good. A run that searched only some of its
  check-worthy claims says so and exits 1. AVeriTeC search mode scores 0.404
  against a 0.708 majority baseline, which is why it is marked experimental.
• Queries and translation by the judge. With --judge, the model writes the search
  queries and translates a non-English claim; the report shows your sentence and a
  checked as: … line. Without a judge, a claim that is not in English is
  UNVERIFIED (language not supported) rather than checked by English-only models.
• Local fallback judge. When Groq hits its rate limit, rejects the key or stops
  answering, the run switches at once to qwen3.5:9b in your local Ollama, if it is
  installed. There is no retry and no wait, and the request that failed is sent again,
  so no batch is lost. The switch and its cause appear on stderr, in the TUI just above
  the prompt, and in the report. The next run tries the configured API first again.
  proofpath never pulls the model; judge.fallback off turns this off. Without the
  model, the run says how to add it (install Ollama, ollama pull qwen3.5:9b).
• /config rows for [search] and permissions.web_search, and footer lines for what
  was searched.
• scripts/eval_averitec.py --search measures the product's own search path, leaving
  out the claim's fact-checking site.

Changed

• The ollama judge's default model is now qwen3.5:9b (was llama3.1), always
  asked with reasoning off.
• The X hint says a pasted text with no link is searched when a provider is set.
• Pages proofpath found on its own are fetched without the contact address.

Fixed

• A bibliography entry that is only a DOI or arXiv URL is never reported as a
  GHOST REFERENCE. It resolves, or it is AMBIGUOUS. A cache migration (v5) drops
  ghost results cached under the old rule, so they are checked again.

v0.4.7

Two more platforms a post can be read from without an account: Lobste.rs and Lemmy.

Added

• Lobste.rs. check --url https://lobste.rs/s/<id> reads a story — its title, its
  submitted address first and the links in its text — and /c/<id> or a #c_<id>
  fragment reads one comment, never the thread above it. Read from the .json twin
  of each page; no key, no account. A moderated comment is UNVERIFIED (blocked), a
  deleted one UNVERIFIED (unreachable), as spec §15 keeps them apart. An anchor
  pointing back at the story's own discussion page is not a source and is left out.
• Lemmy. check --url https://<instance>/post/<n> and /comment/<n> read from
  the /api/v3 every instance serves without a login. A post's submitted address
  comes first, then the markdown links in its body. removed (a moderator's doing)
  is UNVERIFIED (blocked); deleted is UNVERIFIED (unreachable). A private
  instance answers an anonymous reader with the same 400 as a missing post and
  not_logged_in in the body, so the body's token decides: not_logged_in is
  UNVERIFIED (blocked), couldnt_find_post is UNVERIFIED (unreachable), and a
  400 naming neither is UNVERIFIED (unavailable) — never a guess about the post.
  Lemmy is known by its host — the
  lemmy. prefix or one of lemm.ee, sh.itjust.works, beehaw.org,
  programming.dev, feddit.org — never by /post/<n> alone, which is how any
  blog addresses an article; a post on an instance outside that rule is read as a
  page, as before, and no /api/v3 request is ever sent to a host that did not say
  it is Lemmy.

Changed

• The "platform not read in this version" hint, the check --url help and the README
  table name all seven platforms.
• polite.ProviderError carries code: the one snake_case token a JSON error body's
  error field named, when it named one, and nothing else from the body. It is what
  lets the Lemmy reader tell the two facts above apart; no other caller reads it yet.

v0.4.6

The terminal looks like the product: the wordmark in both themes, a settings
panel, and commands that show themselves as you type.

Added

• TUI: slash suggestions. Typing / lists every command above the bar with what
  it wants and what it does; the list narrows per keystroke (/h → /help), Tab
  completes the selected row, ↑/↓ move the selection while the list shows,
  Enter runs what is in the bar. On a short terminal the list windows around the
  selection and says … n more
  (docs/superpowers/specs/2026-09-18-wordmark-banner-design.md §10).
• TUI: /config is a settings panel. Arrow keys move between the settings,
  ←/→ change a choice (permissions.install_browser, permissions.network,
  fetch.respect_robots, judge.provider), Enter edits a text setting in place,
  Backspace restores its default, Esc closes; every change is written at once and
  reported under the panel; the key line says where the judge's API key was found
  and never shows it. /config show|path|set|check are unchanged
  (docs/superpowers/specs/2026-09-18-wordmark-banner-design.md §12).

Changed

• TUI: the wordmark everywhere. The plain theme draws the same ProofPath mark
  in ASCII and the raven is gone from the terminal; rich runs it through five
  crimson bands left to right with the shadow one tone darker; a full-width rule
  closes the banner in both themes; at 129 columns and up the version and hint rows
  sit beside the mark, narrower terminals keep them under it, and below 69 columns
  only the two rows and the rule are drawn
  (docs/superpowers/specs/2026-09-18-wordmark-banner-design.md §8).
• TUI: the bar's frame runs the same gradient. The rounded frame around the input
  is drawn in the wordmark's five crimson bands, light at the left, dark at the
  right; the caret keeps the next run's accent
  (docs/superpowers/specs/2026-09-18-wordmark-banner-design.md §9).

Fixed

• TUI: typing always reaches the bar. A click on the log's background, the banner
  or a panel no longer takes the keyboard away; a printable key typed while a finding
  line or a permission button is focused goes to the bar
  (docs/superpowers/specs/2026-09-18-wordmark-banner-design.md §11).
• Tests: the cache CLI test no longer expires with the calendar. Its "fresh" entry
  was dated 2026-09-11 and crossed the 7-day TTL on its own.

v0.4.5

The bar takes a whole post, and the rich theme opens with the wordmark.

Changed

• TUI: the rich banner is a wordmark. ProofPath in block letters, crimson on
  deep crimson, with the version and hint lines under it; the Braille raven is gone.
  The ASCII raven of the plain theme and the website's raven stay
  (docs/superpowers/specs/2026-09-18-wordmark-banner-design.md).

Fixed

• TUI: a multi-line paste is no longer cut to its first line. Textual's input
  keeps the first line of a bracketed paste; the bar now holds the whole text behind
  a one-line summary (pasted · 3 lines · 118 chars · enter to check, esc to drop),
  Enter checks it whole and Esc drops it. The run is labelled /check pasted text and the paste stays out of the command history
  (docs/superpowers/specs/2026-09-17-multiline-paste-design.md).

v0.4.4

Paste any link: a page is read as the document itself.

Added

• A page as the target. A bare address on any host that is not a social platform
  — pasted into the TUI bar or given to check --url — is read as the document
  itself instead of being refused as "a page, not a post". The page is fetched up
  the same ladder its sources are, cut into one paragraph per block, and cites the
  way a post does: each paragraph's sentences are checked against the pages that
  paragraph links to. A page that prints a reference list is paired by citation
  number like a paper. A page the ladder could not read ends the run in the ladder's
  own words (UNVERIFIED (blocked), … (unreachable), …), with the setting to type
  when the browser step was the missing one; a page with no links is reported rather
  than passed as clean; the page's own address is never one of its sources
  (docs/superpowers/specs/2026-09-17-page-target-design.md).

v0.4.3

The TUI says what to type when it stops you, and every tag gets a GitHub Release.

Added

• GitHub Releases. The release workflow now creates the GitHub Release for each
  tag, with the version's CHANGELOG section as its notes (scripts/release_notes.py)
  and the wheel and sdist attached; the eight earlier tags were released the same way
  by hand on 2026-09-17.

Changed

• Permission hints. The TUI's consent question names the /allow answers instead
  of the terminal's keys; a run that skipped sources because the browser was not
  permitted says which setting turns it on (/config set … in the TUI,
  proofpath config set … elsewhere), and a run the network setting stopped says so
  in its once-per-run network: not permitted note; a page given where a post was
  expected says to paste the claim as text; /allow with nothing waiting says where
  the question appears.
• README screenshots are real runs with the raven (docs/eval/tui-raven-{rich,plain}.svg,
  method in docs/eval/2026-09-17-tui-raven.md).

Fixed

• loading models … no longer outlives the loading in the TUI. The note is
  transient and is taken back when the Verifying stage ends; a finished run no longer
  looks like it is still loading.