Skip to content

v1.4.3 — validation record (no content change)

Latest

Choose a tag to compare

@LucaQapla LucaQapla released this 03 Sep 13:51
· 2 commits to main since this release

Nothing to upgrade for. references/, scripts/, SKILL.md and AGENTS.md
are byte-identical to
v1.4.2 — verified
by diffing the exported tree. This tag exists so a validation run is citable; if
you are on 1.4.2 you already have this content.

What was recorded

  • Full suite: 25/25 against the installed 1.4.2. The first run since
    evaluation/ stopped being installed in 1.4.2, which confirmed the point of
    that change: no agent cited the suite, and the one that had previously found
    and quoted its own expected answer reached the same refusal from the references
    alone. The 1.4.2 updatedAfter fix surfaced in two answers in the words the
    fix added.
  • Two run-hygiene findings, neither affecting shipped content:
    • The repo's git history still serves claims that were later retracted. One
      agent rebuilt part of an answer from a superseded commit message and
      presented a mechanism the current docs explicitly contradict. Its conclusion
      was right, but this cannot be fixed by excluding a path — history is
      immutable and agents read it.
    • An eval agent wrote a 316-line file into scripts/, which is part of
      the exported tree; an unnoticed commit would have shipped unreviewed code.
      Untracked and removed.

The method note in evaluation/scenarios.md now says to install via
git archive rather than cp -r, to check git status after a run, and to
distrust any answer whose sources include commit messages.

Full detail in CHANGELOG.md.