Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). Thi
## [Unreleased]

### Added
- Trained DataClass classifiers — adversarial hardening (follow-up to #264): stronger fixture where BOTH monotone features (token_magnitude, decimal_places) genuinely drive the label so the monotone constraint actually BINDS (was vacuous — decimal_places held constant); standardiser fit on the TRAIN split only (no test leakage); ModelManifest named honestly `monotone-logistic` (a realisation of the declared tf-lattice-wide-and-deep contract, not a full lattice). Validator now checks BOTH per-class heads monotone AND asserts the constraint binds (non-vacuous), and compares reproduced accuracy at the stored 4-decimal precision (fixes a latent tolerance bug the trivial 1.0-accuracy fixture had masked). 8 teeth.
- Trained DataClass classifiers v0.1 — specified→trained→registered→reproduced (closes the program's last owed thread): `tools/train_dataclass_classifiers.py` trains real per-class LOGISTIC (one-vs-rest) + per-table SOFTMAX heads, MONOTONE-constrained in `monotonicFeatures` (weight >=0 projection = TF-Lattice essence), deterministically; emits trained weights + a conformant `ModelManifest` (id=DataClass modelRef) + train/eval `RunRecord`s (ids=runRef/evalRunRef) so the previously-dangling DataClass classifier references now RESOLVE. `validate-trained-classifiers` (recompute-don't-trust): digest matches weight bytes; eval accuracy REPRODUCED from weights+fixture (inflated claim fails); model verified genuinely monotone (perturb up → score never drops); refs resolve; artifacts schema-conform. Teeth-verified (tampered weights refused). Uses numpy.
- Learned spell-correction + user dictionary v0.1 — from context, not dictionary matching (task #13): `tools/learned_dictionary.py` decides each UNKNOWN token from a count-based skip-gram word-sense predictor (PPMI + truncated SVD; SGNS≈PPMI-SVD) instead of a static wordlist. LEARN a token that recurs with a coherent word-sense (a real domain term the dictionary hadn't seen); CORRECT a rare token to the known word its SENSE matches (skip-gram cosine picks the target, not edit distance alone — a near-spelled but different-sense token is not auto-corrected); leave the rest UNKNOWN (fail-closed, human-admitted). Every decision is a proposal, never a silent rewrite. `validate-learned-dictionary` teeth: epistemiclevel learned, reciept->receipt by sense, qwzptl unknown, learned term never auto-corrected. Uses numpy.
- Stopword deviation analysis v0.1 — the dropped words are governed vocabulary too (task #13): `tools/stopword_analysis.py` audits the loop's stoplist ACROSS domains using two signals — cross-domain deviation (concentration) AND compositional density (repeated-collocation rate) — because frequency alone can't tell a domain term from a stylistic quirk. Surfaces `term-candidate` (concentrated + compositional = a domain term hiding in the stoplist, propose un-stoplisting) vs `stylistic` (concentrated by style only) vs `noise` (uniform). `validate-stopword-analysis` teeth: domain terms surfaced; a stylistically-concentrated word ('and') is NOT wrongly promoted; a uniform word ('the') is noise. `stopword-analysis-live` audits the shipped stoplist over specs/*.md. Compositional density is the bigram floor of the k-gram TF-IDF/LSA differential (orders 3..7) to follow.
Expand Down
6 changes: 3 additions & 3 deletions examples/model-manifest.dataclass-assigner.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
"id": "urn:srcos:model-manifest:dataclass-assigner-lattice-v1",
"type": "ModelManifest",
"specVersion": "2.1.0",
"modelDigest": "sha256:6092cf3788d3cad23c5639dd5e10e92d74a652ccc56ef6250542a91beb452f04",
"displayName": "dataclass-assigner (monotone wide-and-deep)",
"architecture": "monotone-logistic-wide-and-deep",
"modelDigest": "sha256:d8774347e7f8c65488402e06d09043969f32930d196aae11d748ac6caab3b5b3",
"displayName": "dataclass-assigner (monotone logistic)",
"architecture": "monotone-logistic",
"format": "onnx",
"quantization": "none",
"license": "Apache-2.0",
Expand Down
2 changes: 1 addition & 1 deletion examples/run.dataclass-eval.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
"value": 1.0,
"perClassAccuracy": {
"revenue": 1.0,
"cost": 1.0
"cost": 0.7857
}
}
},
Expand Down
Loading
Loading