feat(calc-golden): v2 — rule families + --derive + era bucketing (T2a) - #76
Merged
Conversation
Calc-golden v1 could only fit ONE calculation shape (net x rate); it reported the gross-inclusive second paths as unexplained residuals. v2 adds calculation SHAPE families so the audit can reverse-engineer WHICH formula the legacy used, not just which rounding mode. - Rule families (RuleFamily: base_role + formula): net, vat_from_gross (round(gross*rate/(100+rate))), net_from_gross (net by division+round, vat by subtraction). audit_corpus fits every (family, mode) whose base is resolvable; reports family_match_pct / best_family / unexplained_by_families ALONGSIDE the v1 net-only keys (fully backward compatible). - --derive 'gross=net+vat' computes extra inputs Decimal-exactly (two operands, +/- only). Derived-from-expect inputs embed the answer, so they are STRIPPED from --runner payloads — the implementation under test never sees them. - --bucket-by @index:N: era pseudo-bucket (record ordinal // N), a regime-change proxy needing no date column. Documented as an active-record-ordinal approximation (a PACK/re-sort shifts it). Validated on the full 948K INVO_HB 13%-bucket, and it OVERTURNED the v1 hand-hypothesis: the dominant shape is gross-based, not net x rate. vat_from_gross:half_up is the best single family in every era (beating net by 3-6pp), and the independent net_from_gross corroborates it exactly (ruling out a self-reference artifact from gross=net+vat). Era bucketing auto-surfaced a regime change (era0-1 ~97-99% -> era2+ ~74-85%). The remaining ~15% late-era residual is explained by NO single rounding family = the per-invoice aggregation path, correctly scoped to v3 (--group-by allocation replay). This is the conservative best-single-family audit superseding my looser union-of-rules estimate — oracle over prior. 7 new tests (30 total); backward compat intact; prepush 7/7.
roam-code AnalysisMode: incremental ( Health Score: 78/100 health: Fair codebase (78/100) — 49 critical, 7 warnings, focus: god_components Health Metrics
PR Risk
Quality Gate: PASSEDGate expression: SARIF Upload
Full analysis outputhealth{
"_meta": {
"cache_ttl_s": 300,
"cacheable": true,
"index_age_s": 3,
"index_status": {
"dirty_files": 0,
"fresh": false,
"head_commit": "9691629a69a4",
"hint": "index latest commit 7551c0ee7298 != HEAD 9691629a69a4 — git-derived metrics (commits, churn, co-change, weather) may be stale. Run `roam index --force`.",
"indexed_commit": "7551c0ee7298"
},
"latency_ms": null,
"response_tokens": 4087,
"roam_version": "13.9.0",
"timestamp": "2026-07-16T12:35:32Z"
},
"actionable_count": 16,
"actionable_cycles": 0,
"agent_contract": {
"confidence": null,
"facts": [
"Fair codebase (78/100) — 49 critical, 7 warnings, focus: god_components",
"health score 78",
"tangle ratio 0.0",
"0.0017 propagation cost findings",
"issue count 65"
],
"next_commands": [
"roam debt",
"roam trends --days 30"
],
"risks": []
},
"algebraic_connectivity": null,
"algebraic_connectivity_available": false,
"bottleneck_thresholds": {
"p70": 1040.6,
"p90": 9504.7,
"population": 1195,
"utility_multiplier": 1.5
},
"category_severity": {
"bottlenecks": {
"critical": 15,
"info": 0,
"warning": 0
},
"cycles": {
"critical": 0,
"info": 0,
"warning": 0
},
"god_components": {
"critical": 34,
"info": 9,
"warning": 7
},
"layer_violations": {
"critical": 0,
"info": 0,
"warning": 0
}
},
"command": "health",
"cycles_actionable": 0,
"cycles_total": 17,
"framework_filtered": 0,
"health_score": 78,
"ignored_cycles": 17,
"imported_coverable_lines": 0,
"imported_coverage_files": 0,
"imported_coverage_pct": null,
"imported_covered_lines": 0,
"index_status": {
"dirty_files": 0,
"fresh": false,
"head_commit": "9691629a69a4",
"hint": "index latest commit 7551c0ee7298 != HEAD 9691629a69a4 — git-derived metrics (commits, churn, co-change, weather) may be stale. Run `roam index --force`.",
"indexed_commit": "7551c0ee7298"
},
"issue_count": 65,
"list_counts": {
"bottlenecks": 15,
"cycle_break_suggestions": 0,
"cycles": 17,
"god_components": 50,
"layer_violations": 0,
"next_steps": 2,
"score_breakdown": 5
},
"project": "roam-code",
"propagation_cost": 0.0017,
"schema": "roam-envelope-v1",
"schema_version": "1.1.0",
"severity": {
"critical": 49,
"info": 26,
"warning": 7
},
"summary": {
"actionable_cycles": 0,
"algebraic_connectivity": null,
"algebraic_connectivity_available": false,
"category_severity": {
"bottlenecks": {
"critical": 15,
"info": 0,
"warning": 0
},
"cycles": {
"critical": 0,
"info": 0,
"warning": 0
},
"god_components": {
"critical": 34,
"info": 9,
"warning": 7
},
"layer_violations": {
"critical": 0,
"info": 0,
"warning": 0
}
},
"cycles_actionable": 0,
"cycles_definition": "Cycle counts derived from `roam.graph.cycles.find_cycles(G, min_size=2)` on the symbol graph. `cycles_total` = all SCCs of size >= 2; `cycles_actionable` = SCCs spanning >=2 files AND no test files (same-file and test-only cycles are informational). Run `roam health` for the per-cycle breakdown.",
"cycles_total": 17,
"detail_available": true,
"god_components": 50,
"god_components_definition": "God components: symbols where `(in_degree + out_degree) > 20` from the `graph_metrics` table, with utility-aware severity bands (standard >50=CRITICAL >30=WARNING; utility >150=CRITICAL >90=WARNING). Run `roam health` for the per-symbol breakdown. Legacy aliases: `god_objects` (fingerprint), `god_classes` (rules).",
"health_score": 78,
"health_score_definition": "weighted geometric mean (0-100) of 5 sigmoid health factors: tangle_ratio, god_components, bottlenecks, layer_violations, file_health (+coverage if available).",
"ignored_cycles": 17,
"imported_coverage_files": 0,
"imported_coverage_pct": null,
"issue_count": 65,
"partial_success": true,
"preserved_list_truncations": {},
"propagation_cost": 0.0017,
"severity": {
"critical": 49,
"info": 26,
"warning": 7
},
"tangle_ratio": 0,
"tangle_ratio_definition": "fraction of symbols inside non-trivial SCCs; higher = more cyclic coupling.",
"total_cycles": 17,
"truncated": true,
"verdict": "Fair codebase (78/100) — 49 critical, 7 warnings, focus: god_components",
"warnings_out": [
"health_algebraic_connectivity_warning:RuntimeWarning:algebraic_connectivity compute failed (ModuleNotFoundError): No module named 'numpy'; returning 0.0 sentinel — value is NOT a legitimate disconnected-graph reading"
]
},
"tangle_ratio": 0,
"total_cycles": 17,
"utility_count": 34,
"version": "13.9.0",
"warnings_out": [
"health_algebraic_connectivity_warning:RuntimeWarning:algebraic_connectivity compute failed (ModuleNotFoundError): No module named 'numpy'; returning 0.0 sentinel — value is NOT a legitimate disconnected-graph reading"
]
}pr-risk{
"_meta": {
"cache_ttl_s": 60,
"cacheable": true,
"index_age_s": 5,
"index_status": {
"dirty_files": 0,
"fresh": false,
"head_commit": "9691629a69a4",
"hint": "index latest commit 7551c0ee7298 != HEAD 9691629a69a4 — git-derived metrics (commits, churn, co-change, weather) may be stale. Run `roam index --force`.",
"indexed_commit": "7551c0ee7298"
},
"latency_ms": null,
"response_tokens": 1316,
"roam_version": "13.9.0",
"timestamp": "2026-07-16T12:35:34Z"
},
"actor": null,
"agent_contract": {
"confidence": null,
"facts": [
"High risk (61/100) — careful review needed (risk_level high) (driver: blast_pct)",
"risk score 61",
"3 risk rank findings",
"3 changed files",
"382 lines added"
],
"next_commands": [],
"risks": [
"pr-risk: high (61/100) on c37556a75d3e72dcb8ac48c4b0c2c18265c9b84c..HEAD — driver: blast_pct",
"High blast radius: 12675 of 41912 symbols affected (30.2%) on c37556a75d3e72dcb8ac48c4b0c2c18265c9b84c..HEAD"
]
},
"author": null,
"blast_radius_pct": 30.2,
"bus_factor_risk": 0.5,
"change_shape": "mixed",
"changed_files": 3,
"closest_historical_pattern": [
"src/roam/commands/cmd_calc_golden.py",
"src/roam/index/golden_calc.py",
"tests/test_calc_golden.py"
],
"closest_similarity": 1,
"cluster_spread": 0,
"clusters_touched": 42,
"command": "pr-risk",
"coupling_score": 0.33,
"dead_code": [
{
"file": "src/roam/commands/cmd_calc_golden.py",
"kind": "function",
"name": "extract"
},
{
"file": "src/roam/commands/cmd_calc_golden.py",
"kind": "function",
"name": "audit"
},
{
"file": "src/roam/index/golden_calc.py",
"kind": "class",
"name": "RuleFit"
},
{
"file": "src/roam/index/golden_calc.py",
"kind": "method",
"name": "pct"
},
{
"file": "src/roam/index/golden_calc.py",
"kind": "method",
"name": "failed"
}
],
"dead_exports": 5,
"familiarity": {
"avg_familiarity": 1,
"files": [],
"files_assessed": 0
},
"findings": [
{
"claim": "pr-risk: high (61/100) on c37556a75d3e72dcb8ac48c4b0c2c18265c9b84c..HEAD — driver: blast_pct",
"confidence": "heuristic",
"evidence": {
"actor": null,
"author": null,
"blast_radius_pct": 30.2,
"bus_factor_risk": 0.5,
"changed_files_count": 3,
"commit_range": "c37556a75d3e72dcb8ac48c4b0c2c18265c9b84c..HEAD",
"coupling_score": 0.333,
"created_at_epoch": 1784205334,
"diff_id": "96737e788050",
"familiarity_risk": 0,
"file_list": [
"src/roam/commands/cmd_calc_golden.py",
"src/roam/index/golden_calc.py",
"tests/test_calc_golden.py"
],
"hotspot_score": 0.785,
"label": "c37556a75d3e72dcb8ac48c4b0c2c18265c9b84c..HEAD",
"lines_added": 382,
"lines_removed": 47,
"minor_risk": 0,
"novelty_score": 0,
"reductive_change": false,
"risk_level": "high",
"risk_score": 61,
"staged": false,
"test_coverage_pct": 50,
"top_driver": "blast_pct"
},
"finding_id_str": "pr-risk:composite-risk-score:96737e788050",
"kind": "pr-risk:composite-risk-score",
"severity": "high",
"source_detector": "pr-risk",
"source_version": "1.0.0",
"subject_id": null,
"subject_kind": "commit"
},
{
"claim": "High blast radius: 12675 of 41912 symbols affected (30.2%) on c37556a75d3e72dcb8ac48c4b0c2c18265c9b84c..HEAD",
"confidence": "structural",
"evidence": {
"affected_symbols": 12675,
"blast_radius_pct": 30.2,
"changed_files_count": 3,
"changed_symbol_ids_count": 106,
"commit_range": "c37556a75d3e72dcb8ac48c4b0c2c18265c9b84c..HEAD",
"created_at_epoch": 1784205334,
"diff_id": "96737e788050",
"file_list": [
"src/roam/commands/cmd_calc_golden.py",
"src/roam/index/golden_calc.py",
"tests/test_calc_golden.py"
],
"label": "c37556a75d3e72dcb8ac48c4b0c2c18265c9b84c..HEAD",
"staged": false,
"total_symbols": 41912
},
"finding_id_str": "pr-risk:high-blast-radius-symbol-touched:96737e788050",
"kind": "pr-risk:high-blast-radius-symbol-touched",
"severity": "high",
"source_detector": "pr-risk",
"source_version": "1.0.0",
"subject_id": null,
"subject_kind": "commit"
}
],
"hotspot_score": 0.78,
"label": "c37556a75d3e72dcb8ac48c4b0c2c18265c9b84c..HEAD",
"layer_spread": 0.35,
"layers_touched": 11,
"lines_added": 382,
"lines_removed": 47,
"minor_risk": {
"files": [],
"files_assessed": 0,
"minor_files": 0
},
"novelty_score": 0,
"per_file": [
{
"blast": 12671,
"churn": 816,
"is_test": false,
"lines_added": 215,
"lines_removed": 34,
"path": "src/roam/index/golden_calc.py",
"symbols": 63
},
{
"blast": 51,
"churn": 443,
"is_test": false,
"lines_added": 46,
"lines_removed": 13,
"path": "src/roam/commands/cmd_calc_golden.py",
"symbols": 7
},
{
"blast": 17,
"churn": 599,
"is_test": true,
"lines_added": 121,
"lines_removed": 0,
"path": "tests/test_calc_golden.py",
"symbols": 36
}
],
"project": "roam-code",
"reductive_change": false,
"reductive_discount_applied": false,
"risk_level": "high",
"risk_level_canonical": "high",
"risk_rank": 3,
"risk_score": 61,
"schema": "roam-envelope-v1",
"schema_version": "1.1.0",
"suggested_reviewers": [
{
"actor": "Dimitris",
"author": "Dimitris",
"lines": 924
},
{
"actor": "Cranot",
"author": "Cranot",
"lines": 261
}
],
"summary": {
"change_shape": "mixed",
"changed_files": 3,
"findings_count": 2,
"lines_added": 382,
"lines_removed": 47,
"partial_success": false,
"risk_level": "high",
"risk_level_canonical": "high",
"risk_rank": 3,
"risk_score": 61,
"score_classification": "classified",
"verdict": "High risk (61/100) — careful review needed (risk_level high) (driver: blast_pct)"
},
"test_coverage_pct": 50,
"total_clusters": 13109,
"total_layers": 31,
"version": "13.9.0",
"warnings_out": []
}roam-code analysis | Commands: |
Roam Agent ReviewVerdict: SAFE (risk_level low) blast-radius 0/100 · ai-likelihood 19/100 · rule violations 0 · critique high-severity 0 Verdict: SAFE. All structural signals clean at the configured thresholds. Next steps
Powered by roam-code — Apache 2.0, 100% local. Customize thresholds in |
Owner
Author
|
Independent adversarial verification vs the BUILD-PLAN T2a falsifier — fresh worktree at 7551c0e, full real INVO_HB Reproduced:
Falsifier verdicts (plan item 6):
Two small defects to fix at/before merge:
Verification script: session scratchpad |
Cranot
added a commit
that referenced
this pull request
Jul 16, 2026
…x keys (#80) The v2 audit verdict printed "N unexplained by ALL families" while the counter behind it was per-bucket best-single-family — and the residual examples were gated on the (much looser) union metric, so an era-3-shaped bucket reports ~26% unexplained while only ever listing the ~1.7% union residuals. Emit BOTH counters (unexplained_by_best_family / unexplained_by_family_union, plus per-bucket family_union_match_pct), word the verdict to say which is which, and gate the examples on the union explicitly. The union is soft by construction (a derived gross + 6 rounding modes degenerates to roughly |error| <= ~1 cent), so headline claims should quote the best-single number — now documented in the audit docstring. Also: multiple @index:N entries in --bucket-by silently kept only one era size; now a loud ValueError. Both defects were found by the full-INVO_HB adversarial verification posted on PR #76, which merged verbatim before the fixes landed; this is the follow-up.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tranche-2 (C1 redesigned per the hardened plan). v1 could fit only one shape (net×rate) and reported the gross-inclusive second paths as residuals. v2 adds calculation-SHAPE families so audit reverse-engineers which formula the legacy used.
auditfits every (family, mode); reportsfamily_match_pct/best_family/unexplained_by_familiesalongside the v1 net-only keys (backward compatible).--derive gross=net+vat— Decimal-exact; derived-from-expect keys are stripped from--runnerpayloads (they'd hand the oracle's answer to the implementation under test).--bucket-by @index:N— era pseudo-bucket (regime-change proxy, no date column; documented as an ordinal approximation).Validated on the full 948K corpus — and it overturned my v1 hand-hypothesis: the dominant shape is gross-based, not net×rate.
vat_from_gross:half_upis the best single family in every era (beating net by 3–6pp), and the independentnet_from_grosscorroborates it exactly (rules out a self-reference artifact). Era bucketing auto-surfaced the regime change (era0–1 ~97–99% → era2+ ~74–85%). The remaining ~15% is explained by no single rounding family = the per-invoice aggregation path, correctly scoped to v3. Conservative best-single-family audit superseding a looser union estimate — oracle over prior.7 new tests (30 total); prepush 7/7.