You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Published to PyPI: pip install fieldproof. The package includes the built review UI, so fieldproof serve works without a Node toolchain (scripts/bundle_ui.py builds it into the package before a release).
fieldproof --version.
Fixed
A value quoted from inside a longer word could be marked verified. The exact matcher refused a quote that cuts a word, and the fuzzy fallback then scored the same fragment 100/100 as a perfect partial match. On the example invoice, a vendor name of wind (inside "Northwind") and an invoice number of 0214 (inside "NW-20260214") both verified. A match now has to cover whole words; when the best fuzzy window cuts one, the quote is scored against the whole words, so those two are unsupported and 458.00 is a weak match for $2,458.00. Punctuation glued to a word may be left out of a quote (1 for 1,, 2,458.00 for $2,458.00), and every such occurrence counts when deciding whether a quote is ambiguous.
A string value one digit off verified: NW-20260215 for a document reading NW-20260214 is 96% similar, which passed both the grounding and the value check. Every run of digits in a string value now has to appear in the evidence as written.
--provider anthropic against the real SDK. The provider had only met a mocked client, which returned parsed_output=None for a reply that isn't the schema. The SDK's messages.parse raises instead: an extraction cut off at max_tokens (a long invoice) or refused came out as a pydantic traceback, Invalid JSON: EOF while parsing a string. The provider now streams the reply, reads its stop_reason and says which it was in one line: error: Claude's reply was cut off at 16000 output tokens, before the Invoice was complete; raise the limit (--max-tokens) or extract a shorter document. New --max-tokens option. The tests run the real SDK over a mocked HTTP transport.
The declared minimum anthropic>=0.40 had no messages.parse: AttributeError: 'Messages' object has no attribute 'parse'. The minimum is 0.77, the first release with structured outputs on the Messages API, and the provider tests pass on it.
Two more declared minimums did not work. With typer below 0.15.4 every command stopped (Type not yet supported: pathlib.Path | None on 0.12.0; make_metavar() takes 1 positional argument on --help with a current click), and with pydantic below 2.10 a typed result could not be built from Evidenced values. The minimums are typer>=0.15.4 and pydantic>=2.10, and CI has a job that installs the oldest allowed version of every direct dependency on the oldest supported Python and runs the tests there.
A match could begin or end inside a ligature (inal matched "final").
A result or fixture file that isn't valid JSON, isn't an object, or doesn't fit the schema printed a traceback from extract and verify, and was an unexplained 500 in the review UI. The CLI prints one line (error: result.json does not match the Invoice schema: total.value: Input should be a valid number, unable to parse string as a number) and the server answers 422 with the same message. A file that isn't a PDF, a missing API key and a failed API request are one line too.
Added
Seeded fuzz tests for the promise above: an exact match always covers whole words, a piece of a longer word never verifies, a value with a number the document lacks never verifies, and a damaged result file ends in a report or a one-line error.