Skip to content

andon 0.7.0 — a check that examined nothing is not a PASS

Latest

Choose a tag to compare

@gulmezeren2-byte gulmezeren2-byte released this 04 Aug 06:59
· 1 commit to main since this release

Fixes a false PASS. A plausibility check that could not read a single value was reporting that nothing fell outside the bounds — true only in the way that vacuous statements are true.

plausibility.bounds coerced its column to numbers, dropped whatever would not convert, then found zero values out of bounds in the empty remainder. Given a Turkish Excel export — 1.234,56, ;-separated, the ordinary shape in this tool's home market — the whole column reads as text, and a bound of min: 999999999 came back green:

before:  PASS    tutar stays within bounds
after:   REVIEW  no numeric values in tutar; nothing was compared against the bounds

For a tool that prints "a PASS is not an opinion about the analysis; it is arithmetic about these claims" under every report, this was the one defect it could not afford.

The policy already existed and was already written down — numeric_series says "andon never coerces text to numbers: a corrupted cell must surface as ERROR, not vanish into a NaN that sum() happily skips", and the hard checks obey it. The two plausibility checks bypassed the helper and did exactly what its docstring forbids. Being heuristics, they cannot fail a run, so their version of surfacing it is REVIEW.

Partial reads carry their denominator

A statement about the half of a column that parsed is not a statement about the data you handed over:

REVIEW  bounds  tutar stays within bounds, but only read 2 of 4 value(s) of tutar

plausibility.mean_shift does the same rather than presenting the mean of whatever converted as the mean of the column. Both now report values_checked / values_total / values_skipped in evidence.

The skipped count was itself wrong: computed as a coercion delta, it missed everything pandas had already turned into NaN while reading. n/a, -, #N/A are all in its default na_values, so a column half full of them reported zero skipped while dropping half its rows.

CSV sources take decimal and thousands

encoding and delimiter existed for precisely this reason and stopped one step short: without saying how numbers are punctuated, a Turkish file could be read but never measured.

sources:
  sales: { path: satis.csv, delimiter: ";", decimal: ",", thousands: "." }

The two must differ; a source setting both to the same character is refused rather than guessed at.

Clean numeric data still passes silently — pinned by a regression test, because a correctness fix that turns every ordinary run into a REVIEW is its own defect.

163 tests, ruff + mypy clean.

🤖 Generated with Claude Code