Skip to content

v1.4.2 — eval coverage for the v2 corrections

Choose a tag to compare

@LucaQapla LucaQapla released this 03 Sep 13:19
· 4 commits to main since this release

A validation release, not a content release: the 1.4.1 corrections now have eval
coverage, and adding it turned up one place 1.4.1 had missed.

Fixed

  • GET /v2/sandbox: the updatedAfter / updatedBefore rows had no timezone
    warning.
    1.4.1 said the Europe/Rome caveat was called out "wherever the
    filter appears" — it was in v2/overview.md only, while the query-param table
    you actually read while calling the endpoint still described them as plain
    "ISO 8601 datetime filter". Both rows now carry the warning, what to do about
    it (convert the cursor, or overlap the window and de-duplicate), and a pointer
    that the shipments resource has the same filter.

    ⚠️ If you poll incrementally with a UTC cursor, this is still the failure that
    costs you data silently — see the 1.4.1 notes.

Added

  • Six eval scenarios (#20–25), one per correction shipped in 1.4.1:
    updatedAfter's timezone, the mixed timestamp formats, the sandbox casing
    break, the v2 rate limit, second-tier resources being published rather than in
    flight, and the published spec's info.version proving nothing about the
    contracts inside it. Three are negative controls on a wrong premise, which is
    the shape these failures take in practice. Full suite: 25/25.

    The previous 19 scenarios covered none of what 1.4.1 corrected — which is why
    that drift was found in a live integration rather than by the suite.

Changed

  • evaluation/ is no longer installed with the skill. It is export-ignore
    in .gitattributes, so the install step (now git archive, see the README)
    leaves it out. It listed the expected answer for every validation prompt right
    beside the content it validates, and a run caught an agent reading it while
    answering one of those prompts. The suite stays in the repo — it is a
    maintainer tool, not part of the knowledge pack.

Full detail in CHANGELOG.md.