Skip to content

ToppleCat 0.0.4

Choose a tag to compare

@samzhu samzhu released this 26 Jul 23:52
· 64 commits to main since this release

ToppleCat 0.0.4

Release: 0.0.4 · 2026-07-27

ToppleCat 0.0.4 fixes a misleading mutation result: PIT could finish without
running the public acceptance test that defines an AC. The build looked
successful, but the mutation score had not checked the contract you cared
about.

Mutation must exercise the acceptance contract

ToppleCat now checks that PIT actually ran every required public canonical
@ToppleTest. This also works when production code and acceptance tests use
different Java packages.

The mutation result is now:

  • PASS when the required acceptance tests ran and killed the required mutants;
  • FAIL when PIT produced a usable report but skipped a required canonical
    test; or
  • INCOMPLETE when PIT produced no usable current-run report.

Old mutation artifacts cannot fill a missing result. Safe agent feedback says
that public acceptance coverage is missing without revealing reviewer test
names, paths, or failures.

ToppleCat finds the canonical tests from compiler-generated contract
descriptors. This is an implementation detail; most projects do not need new
configuration.

Reviewer rows no longer need a placeholder Java test

A reviewer can keep independent JSON or YAML cases under src/hiddenTest and
run them through the public canonical @ToppleTest. A Java helper without an
executable JUnit method no longer prevents this rows-only retest.

If reviewer rows and hidden Java tests both exist, both must pass. If neither
exists while hidden retest is enabled, REVIEWER_JUNIT remains INCOMPLETE
instead of becoming an empty pass.

Do I need to change anything?

Most projects do not.

If your build sets PIT targetTests, ToppleCat keeps that setting. Make sure it
includes every public canonical acceptance test. A usable report that skips one
now fails the mutation gate.

Custom mutation producers still control their own targeting and report output.
They must produce a usable current-run report or the gate is INCOMPLETE. The
first mutation run after upgrading may take longer while ToppleCat builds the
contract descriptors and PIT matrix.

What the mutation score covers

The default mutation score measures the strength of the public executable
contract. Reviewer rows and hidden Java tests are separate hidden-retest
evidence; they do not raise the public PIT score.

ToppleCat 0.0.4

發佈版本:0.0.4 · 2026-07-27

ToppleCat 0.0.4 修正一種容易誤判的突變測試結果:PIT 雖然執行完成,卻可能
根本沒跑到某個 AC 對應的公開驗收測試。畫面看起來是綠燈,突變分數其實
沒有檢查你在意的合約。

突變測試必須真的跑到驗收合約

ToppleCat 現在會確認 PIT 是否執行了每一個必要的公開主要 @ToppleTest。
即使正式程式與驗收測試放在不同的 Java 套件,也能正確對應。

突變測試結果會明確分成:

  • 必要驗收測試有執行,而且結果符合設定的突變測試要求:PASS;
  • PIT 報告可用,但漏掉必要的主要驗收測試:FAIL;
  • 當次沒有產生可用的 PIT 報告:INCOMPLETE。

舊的突變測試產物不能拿來補上這次缺少的結果。給代理的安全回饋只會說
公開驗收測試涵蓋不足,不會透露審閱者測試的名稱、路徑或原始錯誤。

ToppleCat 會從編譯器產生的合約描述檔找出主要驗收測試。多數專案不需要
為此新增設定。

審閱案例不再需要空的 Java 測試

審閱者可以只在 src/hiddenTest 放獨立設計的 JSON 或 YAML 案例,再用
公開的主要 @ToppleTest 執行。只有輔助類別、沒有可執行 JUnit 方法的
Java 原始碼,不會阻擋只有案例資料列的隱藏重測。

審閱者案例與隱藏 Java 測試同時存在時,兩邊都必須通過。如果兩者都沒有,
但隱藏案例重測仍處於啟用狀態,REVIEWER_JUNIT 會維持
INCOMPLETE,不會因為沒有測試而自動通過。

升級後需要改設定嗎?

大多數專案不用。

如果 Gradle 建置已設定 PIT targetTests,ToppleCat 會保留原設定。請確認
它包含所有公開的主要驗收測試;只要可用的報告漏掉其中一個,突變測試關卡
現在就會失敗。

自訂的突變測試執行方式,仍由專案自行管理測試目標與報告。這次執行沒有
產生可用報告時,該關卡會是 INCOMPLETE。升級後第一次執行突變測試可能
會比較久,因為 ToppleCat 需要建立合約描述檔與 PIT 執行資料。

突變分數算的是什麼

預設的突變分數衡量公開可執行合約的強度。審閱者案例與隱藏 Java 測試是
另一組隱藏案例重測證據,不會提高公開合約的 PIT 突變分數。