Skip to content

CONFORMANCE_FLOOR reads as a safe dependency floor, but estimator semantics changed above it #237

Description

@Polichinel

Filed from views-datafactory, where we got this wrong and shipped it.

The value is correct — the framing is what misleads

To be clear up front: CONFORMANCE_FLOOR = "1.0.0" is right by your own governance rule. GOVERNANCE.md says the floor tracks the published conformance surface and is "bumped whenever a breaking change is made to any of them. Additive surface (a new law or method) is MINOR and does not bump the floor." The 1.9.0 MAP-containment law was additive. Nothing broke. The number is doing what it is documented to do.

This issue is not "the value is wrong."

What actually happened

CONFORMANCE_FLOOR reads like "the version a consumer can safely depend on." It isn't — it covers the conformance suite, not estimator outputs. Those changed three times:

Version Change
1.2.0 Outside-in HDI tower + mass-aware tip — "fixes a silent" error
1.3.0 No magnitude-based zeroing by default (tower_point / hdi_tower / summarize_tower)
1.9.0 Tower-tip MAP: tip_mass 0.5 → 0.25. Your changelog: "Behavior change to tower_point/summarize_tower outputs, shipped MINOR"

All three sit above the floor. All three shipped MINOR, correctly per your SemVer-for-a-contract policy.

The consumer incident

views-datafactory declared views-frames>=1.0,<2. We do not import the estimators — we use FeatureFrame, FrameMetadata, SpatialLevel, SpatioTemporalIndex and views_frames.conformance — so an audit that asked "does our code work at the floor?" answered yes and stopped.

Then the second-order effect: uv lock keeps an existing pin while it still satisfies the constraint, and >=1.0 satisfies 1.0.0 forever. Our lockfile pinned views-frames 1.0.0 from June until 2026-08-02. Roughly six weeks of CI ran against pre-amendment MAP/HDI semantics. A loose floor did not merely permit stale behaviour — it froze it.

Since views_frames_summarize ships in the same wheel, any environment resolving against our floor could have picked up the old estimators too. Two systems on different versions produce different numbers from the same posterior, with no error.

Our floor is now >=1.10.2. Registered on our side as C-337.

What would have prevented it

The reasoning that failed was: "CONFORMANCE_FLOOR is 1.0.0, the changelog shows no breaking change, therefore >=1.0 is a safe floor." Every step is supported by what the package says about itself. The conclusion is wrong.

Suggestions, in rough order of value — all cheap, and yours to weigh:

  1. Say what the floor does not cover. One line in GOVERNANCE.md and the conformance/__init__.py docstring: "This is the contract-suite version, not a recommended dependency floor. Estimator outputs (views_frames_summarize) have changed within 1.x — see the changelog."
  2. Publish a recommended runtime floor alongside it, moving when estimator semantics change. That is the number consumers actually want, and right now they infer it from the wrong one.
  3. Flag estimator-semantics releases distinctly in the changelog — e.g. a ### Changed — estimator output heading. 1.9.0 does say "Behavior change to tower_point/summarize_tower outputs", which is admirably explicit; it is just not greppable as a class, so a consumer scanning for BREAKING misses it. I did exactly that.

Happy to send a PR for (1) if useful.

Not asking you to change SemVer

Your policy of a high bar for MAJOR looks deliberate and reasonable. The gap is that the most authoritative-looking number in the package answers a different question from the one consumers bring to it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions