Skip to content

feat(conformance): widen the corpus to 18 scenarios and cut contract-v2 #96

Description

@sepehr-safari

Summary

The vendored conformance corpus has been frozen at 15 scenarios since 0.5.0
(ADR-0012). The toolkit's corpus has since grown to 18: firmware-update-success
and firmware-update-failure landed in toolkit 0.4.3/0.4.4, and
refused-authorization in toolkit#156 (released as 0.4.5). Those three have never
been part of the contract.

That gap is not cosmetic. The harness compares only the pinned fixtures, so the
corpus is a sample, and both implementations can pass contract-v1 while
disagreeing on anything it does not reach. #93 is the proof: three rules were wrong
here for as long as they were wrong in the toolkit, and 15/15 stayed green
throughout, because no pinned fixture exercised a firmware status, a non-Invalid
idTagInfo, or any of the 22 transitions the old matrix got wrong. Widening the
corpus is the durable fix; the ad-hoc parity probe #93 used is not a regression
guard.

The three additions land exactly on the blind spots:

Scenario Golden Covers
firmware-update-success [] FirmwareStatusNotification progress states are not failures
firmware-update-failure ["FIRMWARE_UPDATE_FAILURE"] InstallationFailed, the spec value the old rule missed
refused-authorization ["FAILED_AUTHORIZATION"] Blocked, Expired, ConcurrentTx, the refusals the old rule ignored

This is a contract version bump

ADR-0012 and docs/CONTRACT.md both name "a new
scenario" as a trigger for a new contract version, regenerated against the
matching toolkit release, with contract-v1 never edited in place. So unlike #94
(which asks about rule corrections that move no golden, and stays open), this case
is unambiguous: it is contract-v2.

Worth recording while cutting it: regenerating all 18 against the published
toolkit 0.4.5 reproduces the existing 15 fixtures and goldens byte for byte.
contract-v2 is therefore a strict superset of contract-v1, not a revision of
it. Every scenario v1 pinned, v2 pins identically. "Never edited in place" holds
through git history and the release tags: v0.5.4's tree carries v1, so every
released Studio stays pinned to a reproducible reference, which is what ADR-0012's
consequences describe.

Work

  • Regenerate src/ocpp/conformance/fixtures/ and goldens/ for all 18 from the
    published toolkit 0.4.5, using the recipe in that directory's README (the
    existing 15 must come out unchanged, which is the check that the recipe and the
    toolkit release are the right ones).
  • Add the three names to scenario_names in harness.zig; everything else there
    is derived, so the harness, studio ci and studio scenario pick them up.
  • Update the counts and the contract label in harness.zig, the conformance
    README.md, docs/CONTRACT.md, and anywhere else that states 15.
  • New ADR recording the contract-v2 cut, refining ADR-0012 rather than
    superseding it.
  • native test -Dplatform=null and studio ci both at 18/18.

Not in scope

#94 stays open. Cutting contract-v2 for a corpus expansion says nothing about
whether a rule correction that moves no golden needs its own bump, and the
docs/CONTRACT.md wording fix that question calls for should land with the
decision, not here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions