Skip to content

[infra] IDD plugin test framework — adversarial fixtures for SKILL.md regression coverage (sister concern from #154) #156

@kiki830621

Description

@kiki830621

Problem

Source: surfaced during /idd-diagnose #154 sister concern surfacing (Step 3.6)

IDD plugin 目前無 systematic test framework。 每個 SKILL.md 有 dogfood scenario(README-style)但沒 executable test runner#154 diagnosis Block A3 需 build 9 adversarial test fixtures(R1/R2/R3 regression set)來確保 parser 不再 introduce bugs:

  1. --scope=value (eq form)
  2. --scope value (space form)
  3. --scope as last arg (missing value)
  4. --scope --body "x" (next-flag eats value)
  5. --body-file=/missing/path (unreadable file)
  6. Multi-line body (newline content)
  7. Single-line body
  8. Section with ### subsections (level-aware end)
  9. Section with no closing heading (EOF as end)

問題:這 9 fixtures 無 framework → 寫 ad-hoc shell script 跑一次 → 之後 regression 無法自動 catch。 R1/R2/R3 verify 之所以失敗 = AI 寫了「應該 work」的 happy-path smoke test,沒 adversarial coverage。

Type

enhancement / infrastructure

Background

IDD plugin 既有 testing surface:

  • idd-verifyruntime verification (issue → diagnose → impl → verify 6-AI ensemble) — catches implementation regression
  • 每個 SKILL.md 有 README 風格 dogfood example — manual one-off
  • 沒有 tests/idd-edit/test_parser.sh 之類 unit-level CI-able test

Closest precedent:scripts/process-attachments.sh 是 IDD helper script,但無對應 test。

Open questions

  • Test framework choice:bats(bash test framework)/ plain set -e shell script / Python pytest wrapping bash?
  • Trigger:single SKILL.md needs it → 順便建?或等 ≥3 SKILL.md surface 同需求?
  • Scope:
    • (a) idd-edit-only fixtures(本 issue 範疇)
    • (b) IDD plugin-level test infrastructure(generalizable to all SKILL.md)
  • CI integration:GitHub Actions runs on PR?或 pre-commit hook?(plugin distribution model 對 CI 期望低)

Trigger conditions (parking-lot — but lower threshold than #154 layer issue)

Type

enhancement / infrastructure

Priority

P2 — higher than typical parking-lot (#154 implementation immediately need),但本身仍 follow-up scope

Refs #154 #150

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions