fix(data): adversarial hardening of trained DataClass classifiers (self-review) - #265
Merged
Merged
Conversation
…lf-review, no Copilot on #264) Adversarial + functional review of #264 (which Copilot didn't review) found: - the monotone constraint was VACUOUS — the fixture held decimal_places constant, so its weight trained to 0 and the monotone teeth passed trivially. New fixture makes BOTH monotone features (token_magnitude, decimal_places) the primary label signal → both weights bind strongly positive, accuracy realistic (<1.0), monotonicity genuinely exercised. - standardiser was fit on FULL data (mild train/test leakage) → now fit on the TRAIN split only. - ModelManifest overclaimed 'wide-and-deep'/lattice → renamed 'monotone-logistic' (an honest realisation of the declared tf-lattice-wide-and-deep contract, not a full calibrated lattice). - validator now checks BOTH per-class heads monotone + asserts the constraint BINDS (non-vacuous), and — a latent bug the trivial 1.0-accuracy fixture had masked — compares reproduced accuracy at the stored 4-decimal precision instead of full-precision vs 1e-6 (per-class 0.7857 now reproduces). Tamper teeth still refuses corrupted weights. 8 checks green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adversarial + functional review of #264 (which Copilot didn't review)
Since #264 merged without Copilot's second pass, I ran an adversarial review myself and found three real rigor gaps (plus a latent validator bug the fix surfaced):
decimal_placesconstant, so its weight trained to 0 and the monotone teeth passed trivially. New fixture makes both monotone features (token_magnitude,decimal_places) the primary label signal → both weights bind strongly positive, accuracy realistic (<1.0), monotonicity genuinely exercised.wide-and-deep/lattice but a linear monotone logistic was trained → renamedmonotone-logistic(an honest realisation of the declaredtf-lattice-wide-and-deepcontract, not a full calibrated lattice).0.7857spuriously failed. Now compares at stored precision.Validator strengthened: checks both per-class heads monotone AND asserts the constraint binds (non-vacuous). 8 checks; tamper teeth still refuses corrupted weights.
This is the adversarial pass #264 was owed.