Skip to content

fieldproof 0.2.1

Choose a tag to compare

@antonsoo antonsoo released this 02 Oct 09:37
· 134 commits to main since this release
pip install fieldproof

Added

  • Published to PyPI: pip install fieldproof. The package includes the built review UI, so fieldproof serve works without a Node toolchain (scripts/bundle_ui.py builds it into the package before a release).
  • fieldproof --version.

Fixed

  • A value quoted from inside a longer word could be marked verified. The exact matcher refused a quote that cuts a word, and the fuzzy fallback then scored the same fragment 100/100 as a perfect partial match. On the example invoice, a vendor name of wind (inside "Northwind") and an invoice number of 0214 (inside "NW-20260214") both verified. A match now has to cover whole words; when the best fuzzy window cuts one, the quote is scored against the whole words, so those two are unsupported and 458.00 is a weak match for $2,458.00. Punctuation glued to a word may be left out of a quote (1 for 1,, 2,458.00 for $2,458.00), and every such occurrence counts when deciding whether a quote is ambiguous.
  • A string value one digit off verified: NW-20260215 for a document reading NW-20260214 is 96% similar, which passed both the grounding and the value check. Every run of digits in a string value now has to appear in the evidence as written.
  • --provider anthropic against the real SDK. The provider had only met a mocked client, which returned parsed_output=None for a reply that isn't the schema. The SDK's messages.parse raises instead: an extraction cut off at max_tokens (a long invoice) or refused came out as a pydantic traceback, Invalid JSON: EOF while parsing a string. The provider now streams the reply, reads its stop_reason and says which it was in one line: error: Claude's reply was cut off at 16000 output tokens, before the Invoice was complete; raise the limit (--max-tokens) or extract a shorter document. New --max-tokens option. The tests run the real SDK over a mocked HTTP transport.
  • The declared minimum anthropic>=0.40 had no messages.parse: AttributeError: 'Messages' object has no attribute 'parse'. The minimum is 0.77, the first release with structured outputs on the Messages API, and the provider tests pass on it.
  • Two more declared minimums did not work. With typer below 0.15.4 every command stopped (Type not yet supported: pathlib.Path | None on 0.12.0; make_metavar() takes 1 positional argument on --help with a current click), and with pydantic below 2.10 a typed result could not be built from Evidenced values. The minimums are typer>=0.15.4 and pydantic>=2.10, and CI has a job that installs the oldest allowed version of every direct dependency on the oldest supported Python and runs the tests there.
  • A match could begin or end inside a ligature (inal matched "final").
  • A result or fixture file that isn't valid JSON, isn't an object, or doesn't fit the schema printed a traceback from extract and verify, and was an unexplained 500 in the review UI. The CLI prints one line (error: result.json does not match the Invoice schema: total.value: Input should be a valid number, unable to parse string as a number) and the server answers 422 with the same message. A file that isn't a PDF, a missing API key and a failed API request are one line too.

Added

  • Seeded fuzz tests for the promise above: an exact match always covers whole words, a piece of a longer word never verifies, a value with a number the document lacks never verifies, and a damaged result file ends in a report or a one-line error.