The question
docs/CONTRACT.md says:
The contract is immutable within a version. A change that alters detected
output — a new or changed rule, a new scenario, a threshold change, or OCPP
2.0.1 — is a new contract version (contract-v2, …), regenerated against the
matching toolkit release and re-tagged, never an in-place edit of contract-v1.
#93 changed three rules. By the letter of that paragraph it is a contract-v2
trigger. But no golden moved: all 15 still match, in both implementations, and no
fixture or golden was edited. So contract-v1 is still a true and checkable
statement about both engines, which is what the paragraph exists to protect.
#93 deliberately left the contract untouched rather than settle this. It needs
deciding once, because it recurs on every rule fix, and #90 (two ported rule
fixes, goldens unchanged) already hit the same situation without anyone naming it.
Two readings
A. Version on observable change. A rule correction that leaves every pinned
golden matching is not a contract change, because the contract is the fixtures
and the goldens. The trigger is "detected output changed on the corpus", not
"a rule was edited". Nothing to do here beyond making the wording say that.
B. Version on any rule change. Take the sentence literally: the rule set is
part of what contract-v1 names, so correcting a rule means the tag now describes
different behavior than it did at freeze, even where the corpus cannot see it. Cut
contract-v2.
Recommendation
Reading A, with a wording fix. The contract's purpose, in its own words, is to
keep "two implementations, one format" checkable rather than hoped-for, and that
is satisfied exactly when both engines match the pinned goldens. Under B, the tag
would bump on changes that no consumer and no check can observe, and the
regenerate-and-retag cost would fall on every spec fix, which discourages exactly
the kind of correction #93 was.
The gap reading A leaves is real and worth naming instead of papering over: the
corpus is a sample, so two engines can pass contract-v1 while disagreeing
outside it. That is what happened between toolkit 0.4.5 and Studio before #93. The
answer is to widen the corpus, not to bump the tag. #93's parity check (97 probe
traces: 10 firmware statuses, 6 authorization statuses, all 81 ordered
ChargePointStatus pairs, identical code sets on both engines) is the shape of
what the corpus should eventually cover.
If the answer is B, or when the corpus is widened
The natural moment for contract-v2 is folding in the three toolkit scenarios
added after the freeze, which would take the corpus from 15 to 18:
firmware-update-success
firmware-update-failure
refused-authorization (added in toolkit#156, and it exercises the
authorization statuses contract-v1 never covered)
That is an additive regeneration against a known toolkit release, and it closes
part of the sampling gap above rather than just renaming the tag.
Either way
Update docs/CONTRACT.md so the trigger is unambiguous, and record the decision
in an ADR alongside ADR-0012,
which froze the contract but does not answer this.
The question
docs/CONTRACT.mdsays:#93 changed three rules. By the letter of that paragraph it is a
contract-v2trigger. But no golden moved: all 15 still match, in both implementations, and no
fixture or golden was edited. So
contract-v1is still a true and checkablestatement about both engines, which is what the paragraph exists to protect.
#93 deliberately left the contract untouched rather than settle this. It needs
deciding once, because it recurs on every rule fix, and #90 (two ported rule
fixes, goldens unchanged) already hit the same situation without anyone naming it.
Two readings
A. Version on observable change. A rule correction that leaves every pinned
golden matching is not a contract change, because the contract is the fixtures
and the goldens. The trigger is "detected output changed on the corpus", not
"a rule was edited". Nothing to do here beyond making the wording say that.
B. Version on any rule change. Take the sentence literally: the rule set is
part of what
contract-v1names, so correcting a rule means the tag now describesdifferent behavior than it did at freeze, even where the corpus cannot see it. Cut
contract-v2.Recommendation
Reading A, with a wording fix. The contract's purpose, in its own words, is to
keep "two implementations, one format" checkable rather than hoped-for, and that
is satisfied exactly when both engines match the pinned goldens. Under B, the tag
would bump on changes that no consumer and no check can observe, and the
regenerate-and-retag cost would fall on every spec fix, which discourages exactly
the kind of correction #93 was.
The gap reading A leaves is real and worth naming instead of papering over: the
corpus is a sample, so two engines can pass
contract-v1while disagreeingoutside it. That is what happened between toolkit 0.4.5 and Studio before #93. The
answer is to widen the corpus, not to bump the tag. #93's parity check (97 probe
traces: 10 firmware statuses, 6 authorization statuses, all 81 ordered
ChargePointStatuspairs, identical code sets on both engines) is the shape ofwhat the corpus should eventually cover.
If the answer is B, or when the corpus is widened
The natural moment for
contract-v2is folding in the three toolkit scenariosadded after the freeze, which would take the corpus from 15 to 18:
firmware-update-successfirmware-update-failurerefused-authorization(added in toolkit#156, and it exercises theauthorization statuses
contract-v1never covered)That is an additive regeneration against a known toolkit release, and it closes
part of the sampling gap above rather than just renaming the tag.
Either way
Update
docs/CONTRACT.mdso the trigger is unambiguous, and record the decisionin an ADR alongside ADR-0012,
which froze the contract but does not answer this.