Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). Thi
## [Unreleased]

### Added
- Assay fleet-tier contracts (cloud-mesh): `AssayRollup` (fleet/cohort aggregate of `ReasoningAssay` verdicts over a window — ok/sad/bad distribution, calibration-drift view of which `AssayStandard` versions are live across the fleet, unassayed-reason breakdown) and `AssayStandardRollout` (canary→widen/halt promotion of a new `AssayStandard` version across cohorts, gated on an observed `AssayRollup`; rides the release-bundle + lifecycle machinery). New `validate-assay-fleet-examples` target enforces aggregation/rollout soundness (distribution sums, drift-flag agreement, rolloutPct vs node share, decision/phase consistency, and no-promotion-by-hope), all verified biting. This is the boundary between prophet-mesh single-user-local (node self-view, no fleet dashboards, structural) and cloud-mesh (rollup + rollout + dashboards). See `schemas/assay-fleet.README.md`.
- Trained DataClass classifiers — adversarial hardening (follow-up to #264): stronger fixture where BOTH monotone features (token_magnitude, decimal_places) genuinely drive the label so the monotone constraint actually BINDS (was vacuous — decimal_places held constant); standardiser fit on the TRAIN split only (no test leakage); ModelManifest named honestly `monotone-logistic` (a realisation of the declared tf-lattice-wide-and-deep contract, not a full lattice). Validator now checks BOTH per-class heads monotone AND asserts the constraint binds (non-vacuous), and compares reproduced accuracy at the stored 4-decimal precision (fixes a latent tolerance bug the trivial 1.0-accuracy fixture had masked). 8 teeth.
- Trained DataClass classifiers v0.1 — specified→trained→registered→reproduced (closes the program's last owed thread): `tools/train_dataclass_classifiers.py` trains real per-class LOGISTIC (one-vs-rest) + per-table SOFTMAX heads, MONOTONE-constrained in `monotonicFeatures` (weight >=0 projection = TF-Lattice essence), deterministically; emits trained weights + a conformant `ModelManifest` (id=DataClass modelRef) + train/eval `RunRecord`s (ids=runRef/evalRunRef) so the previously-dangling DataClass classifier references now RESOLVE. `validate-trained-classifiers` (recompute-don't-trust): digest matches weight bytes; eval accuracy REPRODUCED from weights+fixture (inflated claim fails); model verified genuinely monotone (perturb up → score never drops); refs resolve; artifacts schema-conform. Teeth-verified (tampered weights refused). Uses numpy.
- Learned spell-correction + user dictionary v0.1 — from context, not dictionary matching (task #13): `tools/learned_dictionary.py` decides each UNKNOWN token from a count-based skip-gram word-sense predictor (PPMI + truncated SVD; SGNS≈PPMI-SVD) instead of a static wordlist. LEARN a token that recurs with a coherent word-sense (a real domain term the dictionary hadn't seen); CORRECT a rare token to the known word its SENSE matches (skip-gram cosine picks the target, not edit distance alone — a near-spelled but different-sense token is not auto-corrected); leave the rest UNKNOWN (fail-closed, human-admitted). Every decision is a proposal, never a silent rewrite. `validate-learned-dictionary` teeth: epistemiclevel learned, reciept->receipt by sense, qwzptl unknown, learned term never auto-corrected. Uses numpy.
Expand Down
8 changes: 6 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
.PHONY: validate validate-resource-contract validate-measurement validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-ingestion-pipeline-examples validate-data-acquisition-examples validate-glossary-alignment-examples validate-data-class-examples validate-table-keys validate-dag-loop validate-vocab-currency-loop vocab-currency-dogfood validate-glossary-promotion validate-agreement validate-stopword-analysis stopword-analysis-live validate-kgram-differential validate-learned-dictionary train-dataclass-classifiers validate-schema-references
.PHONY: validate validate-resource-contract validate-measurement validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-ingestion-pipeline-examples validate-data-acquisition-examples validate-glossary-alignment-examples validate-data-class-examples validate-table-keys validate-dag-loop validate-vocab-currency-loop vocab-currency-dogfood validate-glossary-promotion validate-agreement validate-stopword-analysis stopword-analysis-live validate-kgram-differential validate-learned-dictionary train-dataclass-classifiers validate-assay-fleet-examples validate-schema-references

validate: validate-data-class-examples validate-glossary-alignment-examples validate-data-acquisition-examples validate-ingestion-pipeline-examples validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-resource-contract validate-table-keys validate-dag-loop validate-vocab-currency-loop validate-glossary-promotion validate-agreement validate-stopword-analysis validate-kgram-differential validate-learned-dictionary validate-trained-classifiers validate-multiverseal-twin-examples validate-schema-references
validate: validate-data-class-examples validate-glossary-alignment-examples validate-data-acquisition-examples validate-ingestion-pipeline-examples validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-resource-contract validate-table-keys validate-dag-loop validate-vocab-currency-loop validate-glossary-promotion validate-agreement validate-stopword-analysis validate-kgram-differential validate-learned-dictionary validate-trained-classifiers validate-multiverseal-twin-examples validate-assay-fleet-examples validate-schema-references
@echo "OK: validate"

validate-assay-fleet-examples:
python3 -m pip install --user jsonschema >/dev/null
python3 tools/validate_assay_fleet_examples.py

# General referential-integrity gate: every schema is a valid JSON Schema and every
# $ref resolves. The per-example validators only exercise schemas that ship examples;
# this catches a dangling/renamed/typo'd $ref in ANY of the 300+ schemas — a
Expand Down
43 changes: 43 additions & 0 deletions examples/assay_rollup.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
{
"id": "urn:srcos:assay-rollup:fleet-2026-07-05",
"type": "AssayRollup",
"specVersion": "2.0.0",
"scope": {
"mode": "fleet",
"nodeCount": 12
},
"window": {
"from": "2026-07-05T00:00:00Z",
"to": "2026-07-05T01:00:00Z"
},
"totalAssays": 100,
"distribution": {
"ok": 40,
"sad": 55,
"bad": 5
},
"byMethod": {
"computed": 30,
"retrieved": 25,
"generated": 45
},
"unassayedReasons": {
"post-hoc-binding": 30,
"uncalibrated-verifier": 15,
"correlated-arms": 10
},
"standardAdoption": [
{
"calibrationRef": "urn:srcos:assay-standard:narration-fidelity-verifier:0.2.0",
"nodeCount": 9,
"calibrated": true
},
{
"calibrationRef": "urn:srcos:assay-standard:deployed-nli:0.1.0",
"nodeCount": 3,
"calibrated": false
}
],
"driftDetected": true,
"capturedAt": "2026-07-05T01:00:01Z"
}
30 changes: 30 additions & 0 deletions examples/assay_standard_rollout.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"id": "urn:srcos:assay-standard-rollout:narration-fidelity-0.3.0",
"type": "AssayStandardRollout",
"specVersion": "2.0.0",
"standardRef": "urn:srcos:assay-standard:narration-fidelity-verifier:0.3.0",
"supersedes": "urn:srcos:assay-standard:narration-fidelity-verifier:0.2.0",
"releaseBundleRef": "urn:srcos:runtime-release-bundle:assay-standards-2026-07-05",
"strategy": "canary",
"phase": "widening",
"cohorts": [
{
"cohortId": "canary-a",
"nodeCount": 2,
"state": "promoted"
},
{
"cohortId": "fleet-remainder",
"nodeCount": 10,
"state": "pending"
}
],
"guard": {
"observedRollupRef": "urn:srcos:assay-rollup:fleet-2026-07-05",
"metric": "bad_rate_delta",
"decision": "continue"
},
"rolloutPct": 16.7,
"startedAt": "2026-07-05T00:30:00Z",
"updatedAt": "2026-07-05T01:05:00Z"
}
75 changes: 75 additions & 0 deletions schemas/AssayRollup.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://schemas.srcos.ai/v2/AssayRollup.json",
"title": "AssayRollup",
"description": "A fleet/cohort aggregate of ReasoningAssay verdicts over a window. This is the CLOUD-MESH tier: it exists only where verdicts are aggregated across nodes. A single-user local deployment (scope.mode 'node') MAY emit a self-rollup of its own runs, but the fleet/cohort rollup and its dashboards are structurally absent on-device — the aggregation path is not built there, not merely permission-gated. Records the ok/sad/bad distribution, plus the two things a fleet operator cannot see from a single node: which AssayStandard versions are actually live across the fleet (calibration drift), and the breakdown of why claims are unassayed.",
"type": "object",
"additionalProperties": false,
"required": ["id", "type", "specVersion", "scope", "window", "totalAssays", "distribution", "capturedAt"],
"properties": {
"id": { "type": "string", "pattern": "^urn:srcos:assay-rollup:", "description": "Stable URN for this rollup." },
"type": { "const": "AssayRollup" },
"specVersion": { "type": "string" },
"scope": {
"type": "object",
"additionalProperties": false,
"required": ["mode", "nodeCount"],
"description": "What this rollup aggregates over. 'node' is the on-device self-view (single-user local); 'cohort' and 'fleet' are cloud-mesh only.",
"properties": {
"mode": { "enum": ["node", "cohort", "fleet"] },
"cohortId": { "type": "string", "description": "Present when mode is 'cohort'." },
"nodeCount": { "type": "integer", "minimum": 1, "description": "Number of nodes contributing. 1 for a local self-view." }
}
},
"window": {
"type": "object",
"additionalProperties": false,
"required": ["from", "to"],
"properties": {
"from": { "type": "string", "format": "date-time" },
"to": { "type": "string", "format": "date-time" }
}
},
"totalAssays": { "type": "integer", "minimum": 0, "description": "Total assays in the window. distribution counts MUST sum to this." },
"distribution": {
"type": "object",
"additionalProperties": false,
"required": ["ok", "sad", "bad"],
"description": "Count of assays by projectedState. Sums to totalAssays (validator-enforced).",
"properties": {
"ok": { "type": "integer", "minimum": 0 },
"sad": { "type": "integer", "minimum": 0 },
"bad": { "type": "integer", "minimum": 0 }
}
},
"byMethod": {
"type": "object",
"additionalProperties": { "type": "integer", "minimum": 0 },
"description": "Optional count by method (computed/retrieved/generated). Each value <= totalAssays."
},
"unassayedReasons": {
"type": "object",
"additionalProperties": { "type": "integer", "minimum": 0 },
"description": "Optional breakdown of the 'sad' band by unassayedReason. Sum <= distribution.sad (validator-enforced): tells the operator WHY the fleet is amber."
},
"standardAdoption": {
"type": "array",
"description": "Which AssayStandard versions are live across the fleet — the calibration-drift view. A fleet where some nodes verify against an uncalibrated or stale standard is silently weaker than its ok-count suggests.",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["calibrationRef", "nodeCount", "calibrated"],
"properties": {
"calibrationRef": { "type": "string", "pattern": "^urn:srcos:assay-standard:" },
"nodeCount": { "type": "integer", "minimum": 1, "description": "Nodes verifying against this standard version." },
"calibrated": { "type": "boolean", "description": "Whether that standard is calibrated (mirrors AssayStandard.calibrated)." }
}
}
},
"driftDetected": {
"type": "boolean",
"description": "True when nodes are split across divergent standard versions or any live standard is uncalibrated. Validator cross-checks this against standardAdoption."
},
"capturedAt": { "type": "string", "format": "date-time" }
}
}
Loading
Loading