Skip to content

fix(summarize): outside-in HDI tower + mass-aware tip — resolve minority-duplicate collapse (C-44)#100

Merged
Polichinel merged 1 commit into
developmentfrom
fix/tower-point-zero-spike-collapse
Jun 24, 2026
Merged

fix(summarize): outside-in HDI tower + mass-aware tip — resolve minority-duplicate collapse (C-44)#100
Polichinel merged 1 commit into
developmentfrom
fix/tower-point-zero-spike-collapse

Conversation

@Polichinel

Copy link
Copy Markdown
Contributor

Outside-in HDI tower + mass-aware tip + fail-loud config — resolve C-44 (Tier 1)

Fixes a silent output-correctness bug in the v1.1 tower estimators (register C-44):
a minority duplicated draw (a couple of exact zeros, a lone pair, any repeated value)
could hijack the degenerate ~2-sample 5% floor and collapse both tower_point and the
nested hdi_tower bands. Confirmed on real faoapi forecast cells (2–3 exact zeros + a
clear positive body → tower_point = 0.0, hdi 50% = [0, 1.49]).

The fix

  • Outside-in construction — the tower is built from the widest floor down; each
    narrower floor is the shortest interval contained in its wider parent. Robust by
    construction: outliers are shed by the well-determined wide floors and the containment
    constraint forbids a narrower floor from re-selecting them. Keeps the whole tower; nesting
    • reproducibility laws preserved.
  • Mass-aware tiptower_point reads the median of the configurable tip_mass
    floor (default 0.5, the "shorth"), not the degenerate 5% floor. A k<=0 floor collapses
    to a real sample (not an averaged median) so containment stays well-defined.
  • Fail-loud config (views_frames_summarize.config) — every tower-family tunable in one
    dict, no silent defaults: a missing key raises ValueError naming it (ADR-008/009).
    The tower-family functions drop their tunable kwargs (read from config); masses stays
    per-call. Frozen estimators (ADR-018) untouched; CONFORMANCE_FLOOR stays 1.0.0.

Verification

  • 275 tests (+44: truth table A–L, real faoapi cells, duplicate-count sweep, NaN/inf
    locality, multimodality, config fail-loud) at 100% line+branch coverage.
  • ruff · mypy (3.10 + 3.11) · numpy floor 1.26.4 · validate_docs.sh · 1.2.0 wheel builds.
  • /review-diff CLEAN; /falsify SURVIVED (12 probes, no hard/soft falsifications).

Truth table (was wrong → now)

A 2×0+30×2 → 2.0 · E 3×0+29×3 → 3.0 · F 30×2+2×5 → 2.0 ·
L distinct body+2×3 → body (not 3.0) · majority 25×0+7×5 → 0.0 ·
real faoapi cells → robust body (no collapse).

Governance

ADR-019 amended (inside-out→outside-in, tip_mass, config); Summarize CIC updated (+ the
nesting-tradeoff and robust-mode-vs-MAP caveats); register C-44 Resolved (Tier 1),
C-32/C-34 amended; version 1.1.1 → 1.2.0; CHANGELOG [1.2.0].

Two /falsify observations, documented in the CIC (not new bugs): hdi_tower bands are
coherent nested bands, not the unconstrained shortest HDI (use frozen hdi for that);
and tower_point returns the robust dense mode, which will disagree (downward) with a
histogram MAP on skewed cells (largely C-32 bias being corrected) — validate on adoption.

🤖 Generated with Claude Code

…ity-duplicate collapse (C-44)

The v1.1 tower was built inside-out from the degenerate ~2-sample 5% floor, so
any minority duplicated draw (a couple of exact zeros, a lone pair) was the
unbeatably "shortest" interval and hijacked the foundation — silently collapsing
tower_point AND the nested hdi_tower bands. Confirmed on real faoapi cells.

Rebuild the tower OUTSIDE-IN (widest floor first, each narrower floor the shortest
interval contained in its wider parent): robust by construction — outliers are
shed by the wide floors and containment forbids a narrower floor from re-selecting
them. The tip now reads the configurable tip_mass floor (default 0.5, the shorth),
not the degenerate 5% floor. A k<=0 floor collapses to a real sample (not an
averaged median) so containment stays well-defined.

All tower-family tunables move to a fail-loud config.py (TOWER_CONFIG + REQUIRED_KEYS
+ validate_config/get) — no silent defaults: a missing key raises ValueError naming
it (ADR-008/009). The tower-family public functions drop their tunable kwargs (read
from config); masses stays per-call. Frozen estimators (ADR-018) untouched;
CONFORMANCE_FLOOR stays 1.0.0.

- engine: reversed _dense_tower; new _shortest_seed/_shortest_contained_in/
  _median_in/_mid_sample_in/_tip_floor_index; deleted _select_window + old helpers
- conformance: tip law restated to tip-in-tip_mass-floor
- tests: +44 (truth table A-L, real faoapi cells, dup-count sweep, NaN/inf locality,
  multimodality, config fail-loud); 275 pass at 100% line+branch
- governance: ADR-019 amended (inside-out->outside-in, tip_mass, config); Summarize
  CIC updated (+ the nesting-tradeoff / robust-mode-vs-MAP caveats); register C-44
  resolved (Tier 1), C-32/C-34 amended; version 1.1.1 -> 1.2.0; CHANGELOG [1.2.0]

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Polichinel
Polichinel merged commit 4cdf0d4 into development Jun 24, 2026
6 checks passed
@Polichinel
Polichinel deleted the fix/tower-point-zero-spike-collapse branch June 24, 2026 03:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant