Skip to content

v0.3.0 — Journey Test Honesty

Choose a tag to compare

View all tags
@Hulupeep Hulupeep released this 04 Apr 11:25
· 86 commits to main since this release
v0.3.0
0f910a6

Journey Test Honesty

Specflow no longer trusts that test files exist. It verifies they exercise the real path.

What's new

run-journey-tests.sh now audits test files before running them:

  • No page.goto or request.* calls → fail (not a real journey test)
  • All tests use test.skip() → fail (skipped is not passed)
  • Mocking detected (jest.fn, vi.fn) → fail (must exercise real path)
  • Contract required_patterns checked against test content
  • Contract forbidden_patterns checked against test content

Why

HOOK-724–727 incident on hooktunnel-web. Four test files existed, passed all Specflow checks. All were regex scans or test.skip() — none exercised the real path. Platform canary was broken for weeks.

Also in this release

  • verify-setup.sh: Board CLI installation check, Repository vs git remote validation
  • git log depth increased from 5 to 20 for feature branch coverage
  • install-hooks.sh: dynamic hook copy (new hooks auto-installed)
  • specflow-writer.md: MANDATORY OUTPUT RULES at top
  • CONTRACT-SCHEMA.md: test_hooks.required_patterns and forbidden_patterns
  • Full CHANGELOG.md covering all 14 versions

Install / Update

npx @colmbyrne/specflow update . --ci
npx @colmbyrne/specflow verify
Assets 2
Loading