Skip to content

v3.0.0 — 50 QA Skills, TDD-validated

Latest

Choose a tag to compare

@petrkindlmann petrkindlmann released this 10 Jun 18:06

50 skills, TDD-validated, with an executable eval runner

v3.0.0 grows the library from 43 to 50 skills, re-templates every existing skill to one house format, and turns the evals from inert data into something a runner actually checks.

➕ 7 new skills (43 → 50)

Found via a multi-lens gap analysis + judge panel, then built through a research → author → review pipeline (the review stage caught a fabricated Jest fake-timers claim, a 16-digit secret-scan that would have flagged every legitimate test card, and several missing sections before they shipped):

Skill Category What it does
test-case-management Process Manual/hybrid cases in TestRail/Xray/Zephyr/Qase — anatomy, bulk authoring from stories, ambiguous-step linting, CSV/API import-export, traceability
test-suite-curation Process Evidence-based regression-suite pruning — coverage fingerprinting, AST duplicate clustering, CI-history mining, smoke/core/extended tiering, quarantine-before-delete
bug-reproduction AI-QA Vague report → verified minimal repro → failing regression test; git bisect, deterministic repro, red-before-fix
agentic-browser-testing AI-QA Goal-driven E2E via a browser agent (Playwright MCP / computer-use) — determinism controls, cost budgets, accessibility-tree interaction, graduation to scripted tests
email-testing Specialized Signup/reset/magic-link/OTP flows via Mailpit/Mailosaur/MailSlurp — inbox polling, link/OTP extraction, deliverability checks
payment-testing Specialized Stripe (+ Adyen/Braintree/PayPal) sandboxes — test cards, 3DS/SCA iframes, test clocks, webhook signature/idempotency, refunds
analytics-tracking-testing Specialized GA4/dataLayer/pixel correctness — tracking-plan contract, beacon interception, param/value/timing assertions, Consent Mode v2, CI gating

🛡️ AI-agent / LLM security coverage

Prompted by a real indirect-prompt-injection payload disguised as a security finding:

  • security-testing gains an OWASP LLM Top 10 (2025) section and routes LLM-layer attacks to ai-system-testing.
  • ai-system-testing gains indirect-injection (tool output / RAG / scan reports), self-propagating directives, exfil-via-agent, and a defend-the-tester section.
  • Ships skills/ai-system-testing/scripts/detect_injection.py — a self-tested scanner (10 attack-marker rule classes) plus references/injection-detector.md.

🔁 All 43 existing skills re-templated

Every SKILL.md brought to one house format (docs/SKILL_TEMPLATE.md): <objective>, Discovery Questions, Core Principles, domain sections, Anti-Patterns, Verification, objectively-checkable Done When, Related Skills. Heavy code pushed into references/. Currency pass on every named tool/version.

✅ Evals are now executable

  • scripts/run_evals.py--static (content proxy, CI-safe), --baseline (RED), --live (GREEN, via claude -p).
  • Extended scripts/validate_skills.py and a generated scripts/build_index.py (skills_index.json had drifted to v2.5.0).
  • A CI workflow runs the validator + static evals + index check + injection-detector selftest on every PR.
  • Library health: validator 0 errors across 50 skills; static evals 92% checkable pass.

📚 Docs & site

README.md, CLAUDE.md, AGENTS.md, and the landing page updated to 50 skills with new disambiguation rules. docs/SKILL_TEMPLATE.md is the new authoring contract.

Install: npx skills add petrkindlmann/qa-skills