QA Suite 0.9.0: 126-run evaluation on three models

7 new independent blind trials (API 3.1, data migration, requirements analysis, test design, hard mobile, hard test data, exploratory), with/without skills, Opus / Sonnet 5.5 / Haiku, 3 runs each (126 runs, blind graders, low effort).

Result: the skills help most on the weakest model (Haiku mean 89.3 vs 85.4; test design 100 vs 86). On strong models the gap is small (+1.9 / +2.3) and several trials hit the ceiling. The revised exploratory skill now beats no-skill on Opus (90 vs 74) and Sonnet (79 vs 74) but still loses on Haiku (65 vs 74). Mobile plan review: no difference.

Caveats (n=3, low effort, grader model differs from the 0.7.0 round) are in docs/EVALUATION.md section 8.

QA Suite 0.8.0: international test data, API security probes, OWASP LLM 2026

• preparing-test-data: card_test (published sandbox test cards only, Luhn), phone_intl (US/UK fictional ranges), iban (TR, DE, GB, FR, NL, ES, IT, mod-97).
• testing-apis: opt-in --security-probes adds one SQL-injection and one XSS smoke probe per operation (no 5xx, no leaked DB/stack errors, no unescaped script in HTML).
• testing-ai-features: OWASP Top 10 for LLM Applications 2026 as the primary edition, 2025 IDs kept as cross-reference.
  All tests pass; see CHANGELOG.md.

QA Suite 0.7.1: hardening

Fixes from an external adversarial review: CSV formula injection (CWE-1236) neutralised in all CSV exports (--no-formula-escape to opt out); OpenAPI 3.1 type lists no longer crash openapi_tests.py; decision_table.py refuses combinatorial explosions (default limit 2048, --max-combinations); ep_bva.py rejects zero/negative steps and inverted ranges. New tests/test_hardening.py.

QA Suite 0.7.0: deeper, proven, launch-ready

Descriptions trimmed to fit the skill-listing budget (15.1k → 7.4k chars); per-requirement traceability for API/mobile/NFR/AI generators and migration results; shared TCKN/VKN/IBAN and JUnit (Selenium, Cypress, pytest…) result ingestion; xlsx review import; export formats doc-verified with an import kit; volatile-facts CI check; quickstart, manual tester guide (TR), privacy section, CONTRIBUTING and issue templates.

Evidence: 5 new independent blind trials, 30 skill vs no-skill runs on Opus (low effort), blind grading. Results are mixed and published as-is in docs/EVALUATION.md §7: small gains in Excel review (96 vs 92, no false positives) and AI chatbot testing (65 vs 59), a loss in exploratory testing (81 vs 90), ceiling on mobile and test data. Routing 84/84 on Opus and Haiku. 266 tests.

Install: claude plugin marketplace add alinurettin/testskills → claude plugin install qa-suite@qa-suite-marketplace, or upload the zips below.

QA Suite 0.6.0: 17 skills, API/mobile/AI/migration testing, blind-trial evidence

QA Suite: 17 Agent Skills that make your AI assistant work like a senior test analyst (TR/EN).

New in 0.6.0

• 6 new skills:
  • testing-apis: OpenAPI → contract tests and an executable Playwright API suite, including BOLA
  • running-exploratory-tests: SBTM charters and session sheets turned into defects and regression tests
  • testing-ai-features: LLM/RAG eval sets mapped to the OWASP LLM Top 10 2025, with a scoring gate
  • preparing-test-data: synthetic TCKN/VKN/IBAN data and KVKK/GDPR masking
  • testing-mobile-apps: iOS/Android checklists and a device matrix
  • testing-data-migrations: reconciliation and sign-off
• Export to TestRail, Azure DevOps Test Plans and Qase.
• Risk-based regression selection with a time budget, reporting residual risk.
• Domain packs for insurance/pensions and telecom (6 in total).

Evidence (blind trials: the agent never saw the code or the answer key)

• Demo Bank API: 5/5 planted defects, including BOLA, plus 2 real unplanted ones. An independent rerun gave the identical result (31 passed / 11 failed).
• Customer data migration (cp1254 → UTF-8): 9/9 planted defects, with no false positives on the Turkish-casing traps.
• FAST transfer web app, end to end (0.5.0): 5/5.
• Proxy routing: 84/84 requests across 17 skills. 150 unit tests run in CI.

Methodology and limits: docs/EVALUATION.en.md · Real outputs: examples/

Install

• Claude Code: claude plugin marketplace add alinurettin/testskills → claude plugin install qa-suite@qa-suite-marketplace
• claude.ai / Claude API: download the zips below and upload them under Settings → Skills. Upload all 17; the skills hand work to each other.
• Other agents (Codex, Copilot, Cursor, Gemini CLI): copy the skills/ folders.

Full history: CHANGELOG.md

QA Suite 0.5.1

QA Suite: 11 Agent Skills that make your AI assistant work like a senior test analyst (TR/EN).

Kurulum / Install

• Claude Code: claude plugin marketplace add alinurettin/testskills → claude plugin install qa-suite@qa-suite-marketplace
• claude.ai / Claude API: download the zips below and upload them under Settings → Skills. Upload all 11; the skills hand work to each other.
• Other agents (Codex, Copilot, Cursor, Gemini CLI): copy the skills/ folders.

Evidence (blind trial on a demo bank app with 5 hidden defects):

• 5/5 defects found, plus one real unplanted defect
• 46/46 candidate tests automated with Playwright
• "Not ready" release verdict, which was correct

See examples/fast-transfer for the outputs. Full history: CHANGELOG.md