v4.3.0 — Test Intelligence Skills, --version flag, canary upgrade

Pairs with Capillary/canary-capillary v4.3.0.

New skills — Test Intelligence

Five new bundled skills for active test quality intelligence:

/canary-ci-ready

Analyses a test suite for CI readiness across 5 checks: coverage depth, flakiness (documented failures with linked open issues count as verified), assertion quality, critical path coverage, and suite runtime. Investigates config/auth failures via user_catalog_skill in .canary/company.json. Standalone gate and convergence check for the pipeline.

/canary-test-pipeline

Multi-phase orchestrator (Gate → Assess → Discover → Impact → Generate → Verify) with a convergence loop and health report on exit. Follows the harness:docs-pipeline pattern. Loops until CI-ready passes or the user stops.

/canary-critical-areas

Risk-ranked area list from git churn, downstream dependents (harness graph → static fallback), business-critical signals, and existing coverage depth. Optional critical-areas.json artifact for cross-skill context.

/canary-edge-cases

Surfaces edge cases across 6 categories (boundary values, race conditions, locale/timezone, partial network, unexpected input shapes, accessibility). Explanation depth scales via --level sdet|junior|manual.

/canary-failure-impact

Traces downstream effects of a test, function, or code path failing undetected. Domain heuristics (billing, auth, compliance) boost severity. Produces Critical/High/Medium/Low label with affected dependency list.

CLI improvements

• canary --version / canary -V — conventional flag form alongside the existing canary version subcommand
• canary upgrade — upgrades via pipx (preferred) with pip fallback; --dry-run shows what would change
• canary (bare) — now shows help instead of erroring (no_args_is_help=True)

Overlay

Pair with canary-capillary v4.3.0 for Coverage Intelligence Arc (depth scoring, test inventory) and Test Intelligence close-out.

v4.2.0 — wdio Migration Support, Skill Python Venv Execution

What's new

feat(migrate): wdio framework support

canary migrate now detects and scaffolds WebdriverIO projects.

• Config probes: wdio.conf.ts, wdio.conf.js, wdio.conf.mjs
• package.json scripts detection
• Scaffold template: wdio.conf.ts + tests/ directory
• Framework shape: mobile

feat(skills): Skill CLIs run in canary's own venv

.py skill CLIs are now invoked via sys.executable instead of the shebang, so canary's venv dependencies (including newly bundled openpyxl>=3.1) are always available to skills. Fixes capillary-optum-user-catalog xlsx import on machines where the system Python lacks openpyxl.

chore: Oracle → Canary in CANARY_STATE.md

Bat emoji swapped to 🐤; bat is Oracle's character mark, not Canary's.

Overlay

Pair this with canary-capillary v4.2.0 (Capillary/canary-capillary) for the full release.

v4.1.0 — Company Knowledge, Skill Deployment, Global Discovery

What's new in v4.1.0

Company Knowledge (canary company-knowledge)

Ground AI generation in your organisation's internal context — Confluence spaces, Jira projects, internal doc URLs, MCP servers, skill slugs — without committing proprietary content.

• Three-source merge cascade: ~/.canary/company.json (org defaults) → .canary/company.json (project-local) → .canary/company.<env>.json (env override)
• canary company-knowledge init — interactive scaffolder
• canary company-knowledge show --validate-mcp — verify MCP server registration locally
• canary company-knowledge show --env <name> — inspect a specific env layer

Skill deployment via canary migrate --overlay

canary migrate --overlay path/to/company-overlay --apply

Skills in the overlay with a deploy_to frontmatter field matching the detected project shape are copied into .canary/skills/ automatically. Dry-run shows what would be deployed.

deploy_to: [api, e2e_ui]   # copy to API and E2E test repos

Global skill discovery (~/.canary/skills/)

Skills installed in ~/.canary/skills/ are available in every Canary session regardless of working directory — including from the Claude web extension and scratch directories.

mkdir -p ~/.canary/skills/my-skill
cp SKILL.md ~/.canary/skills/my-skill/
canary skills list   # shows "Global skills" group from anywhere

Pre-commit proprietary gate

python3 hooks/check-proprietary.py --install

Runs the CI proprietary-identifier check locally before every git commit. Requires .proprietary-denylist (gitignored).

Docs

• New docs/guides/company-knowledge.md — full operational guide
• docs/specs/skill-discovery.md updated to v3 (global tier, deploy_to)
• docs/wiki/For-Manual-Testers.md — global skill install for Claude web extension users

Upgrade

pipx upgrade canary-test-ai
# or pin to this release:
pipx install "git+https://github.com/bop-clocktower/canary@v4.1.0"

v4.0.0 — Oracle → Canary

First release of the rebranded Canary plugin. Continues the existing release line (descends from v3.0.0) — no prior release was modified.

Highlights (since v3.0.0)

• Oracle → Canary full rebrand — package, slash commands, plugin name, and branding
• Plugin relocated to the repo root (was plugins/oracle/)
• Docs overhaul: brand-kit reorg, removal of stale API-key / removed-command references, added CI guard
• Company-leak scrub + open-core proprietary guard
• CI: actions/setup-python v5 → v6

Install

```
pipx install git+https://github.com/bop-clocktower/canary@v4.0.0
```

Verification

• ✅ 452 unit tests passing

Full changelog: v3.0.0...v4.0.0

v3.0.0 — Keyless Architecture (LLM Layer Removed)

Breaking Changes

⚡ Keyless Architecture — LLM Abstraction Layer Removed

Oracle no longer manages LLM providers, API keys, or model selection. All generation runs through the host Claude Code session. The oracle generate CLI command and multi-provider abstraction have been removed.

Migration: Remove any ORACLE_LLM_* environment variables and oracle generate calls. Use the /oracle-write-test slash command (or equivalent) inside a Claude Code session instead. See AGENTS.md for the updated workflow.

New Features

🎫 Ticket Updater

oracle ticket-update — posts a run summary as a Jira comment and transitions the ticket to the configured post-run status.

Fixes

• fix(plugin): rename MCP server key oracle → oracle-mcp to avoid conflicts with other MCP servers
• fix(v3): remove broken AGENTS.md links left over from deleted v3 workflow files

Upgrading

```bash
pipx upgrade oracle-test-ai
oracle version # 3.0.0
```

v2.4.0 — Workflow Discovery, Framework Picker, Host-LLM Migration

What's New

🔍 Project Workflow Discovery

Never hardcode Jira status names again. Oracle now discovers per-project issue workflows and persists them locally.

• oracle workflow discover [--project <key>] [--refresh] [--dry-run] — fetches Jira issue types, statuses, and transitions via the Jira REST API (ATLASSIAN_URL / ATLASSIAN_USER / ATLASSIAN_TOKEN); synthesizes open/closed for GitHub repos without a project board
• oracle workflow show [--project <key>] [--roles-only] [--json] — inspect the cached mapping at a glance
• Semantic-role heuristics map status names to roles (qa_passed, ready_to_deploy, in_review, in_qa, in_progress, blocked) with priority ordering
• resolve_role(project_key, role) module-level shortcut for downstream consumers — returns the status name or None; callers must never fall back to a hardcoded string
• Cache-first: reads .oracle/workflow-<key>.json; only calls Jira/GitHub when missing or --refresh is passed
• Reads project keys from .oracle/company.json jira_projects when --project is omitted

🎯 Framework Picker (Stages 1–3, #128–#130)

• 16 categories — expanded registry covering synthetic_data, accessibility, contract, snapshot, visual_regression, and more
• Ranked recommendations — recommend() returns a sorted candidate list with per-framework confidence scores
• Observability-sink routing — Stage 2 matches suites with reporting destinations (Datadog, Prometheus, etc.)
• Enterprise license gate — Stage 3 strips commercially-licensed tools (Tricentis Tosca, NeoLoad, LambdaTest) unless the matching ORACLE_LICENSE_* env var is set; OSS defaults always remain

🤖 Host-LLM Migration (Phases 1–2)

• Phase 1 — oracle-test-author no longer requires an API key; runs in the Claude Code host session
• Phase 2 — oracle-heal-test self-heal loop as a keyless /oracle-heal-test slash command
• oracle generate marked deprecated (removed in v3.0); ADR-0003 documents the migration path

🗣 Voice Profiles

Named tone presets (concise, pedagogical, assertive, …) configurable per-project via .oracle/voice.json

🔢 Magic-Number Detection

QualityScorer now flags hardcoded numeric literals in generated tests as a quality gate

🛠 oracle skills run + CLI/entry Loader

• oracle skills run <name> [args…] invokes skills with a cli: or entry: field
• --allow-executable-skills required for non-interactive / CI contexts

🔌 Plugin Lifecycle Hooks

block-no-verify, protect-config, quality-gate, and pre-compact-state hooks wired into .claude/settings.json

🧹 Cleanups

• Removed obsolete API-key setup modules (env_setup, setup_wizard) — no API key required for core workflows
• oracle version now derived from package metadata

Upgrading

pipx upgrade oracle-test-ai
oracle version  # should show 2.4.0

Stats since v2.1.0

515 unit tests passing · 39 new tests for workflow discovery · no regressions

v2.1.0 — Test Quality Scoring, Language-Aware Recommendations, CI/CD Improvements

What's new in v2.1.0

Test Quality Scoring (#104)

Every oracle generate run now includes a static-analysis quality score — no extra flags needed.

• Coverage breadth: test function count + negative/error path detection + parametrize bonus
• Assertion density: framework-aware assertion count per test (pytest `assert`, playwright `expect`, vitest `expect`, k6 `check`)
• Flakiness risk: deductions for hardcoded `waitForTimeout`/`sleep` calls, `random.*`, timestamp-dependent assertions
• Composite 0–100 score with letter grade A–F
• Surfaced in CLI text output, `--json`, GitHub Action PR comment table, and SARIF `properties`

Language-Aware Framework Selection (#94)

The recommender now reads project metadata (detected from `package.json`, `tsconfig.json`, `requirements.txt`) and filters framework candidates to those matching the project's languages. TypeScript/JavaScript projects route API tests to Playwright; Python projects route to pytest. Falls back to the unfiltered order if no language match is detected.

Explicit Framework Hints (#98)

Prompts can now name a framework directly — `oracle generate "write vitest tests for …"` — and the classifier honours it, bypassing the automatic recommender. Useful when you want to override the default for a specific generation.

Commit-to-PR Workflow (#96)

New `oracle-commit-test.yml` workflow: triggered manually via `workflow_dispatch`, it checks out a PR branch, re-runs Oracle, commits the generated file, and posts a follow-up comment. The PR comment now includes a "Add to your regression suite" section linking directly to this workflow.

Environment Utilities (#99)

New CLI subcommands and core modules for first-run configuration:

• `oracle env-setup` — interactive wizard
• `oracle env-setup-legacy` — non-interactive flow for scripts
• `agent/core/provider_ping.py` — no-cost API-key validation via `models.list`
• `agent/core/env_writer.py` — idempotent `.env` merge (refuses to rewrite quoted/multiline values)

Other

• #97 Spec: skills can bundle executable code via `cli:`/`entry:` fields
• #100 Runnable example prompts for all four frameworks (Playwright, pytest, Vitest, k6)
• #101 `scripts/extract_python_docstrings.py` — materialises module docstrings as harness knowledge
• #95 Action: pass extracted JSON values via temp files (fixes empty PR comment fields)
• #92/#93 Classifier: HTTP verb/path signals route to API tests; CI test-runner fix

Stats

• 437 tests passing (up from 361 at v2.0.0)
• All CI checks green: Architecture Enforcer, Security Reviewer, Harness, Docs Lint, Quality & Integrity

Upgrading

pip install --upgrade git+https://github.com/bri-stevenski/oracle-test-ai-agent@v2.1.0
# or
pipx install git+https://github.com/bri-stevenski/oracle-test-ai-agent@v2.1.0

GitHub Action users: bump your pin to @v2.1.0 or move the floating @v2 tag.

v2.0.0 — Claude Code Plugin, Skill Discovery, Interactive Onboarding

What's new in v2

Claude Code Plugin

Oracle is now a first-class Claude Code plugin. Load the repo root as a plugin to get:

• 6 MCP tools via FastMCP: oracle__analyze_file, oracle__write_test_file, oracle__run_tests, oracle__init_suite, oracle__list_frameworks, oracle__migrate
• 3 slash-command skills: /oracle:generate, /oracle:init, /oracle:migrate
• 7 agent definitions: oracle-test-generator, oracle-initializer, oracle-migrator, oracle-test-author, oracle-test-reviewer, oracle-framework-advisor, oracle-flake-hunter
• 4 slash commands wired to the four MVP personas: /oracle-write-test, /oracle-review-test, /oracle-pick-framework, /oracle-debug-flake

Skill Discovery Convention

Downstream overlay repositories can extend Oracle with zero application code — just .oracle/skills/<name>/SKILL.md directories. Local skills override bundled skills of the same name. Discovery walks from CWD up to the git root.

oracle skills list           # show all discoverable skills
oracle skills list --verbose # include file paths

Interactive Guided Onboarding

First-run SetupWizard asks permission before any unconfigured command, guides provider selection, verifies the API key, and optionally runs a sample generation. Re-runnable with oracle setup.

pipx Distribution

pipx install git+https://github.com/bri-stevenski/oracle-test-ai-agent@v2.0.0

Package renamed to oracle-test-ai on PyPI. The oracle binary on PATH is unchanged.

Action Fixes

• --no-setup flag now correctly placed before the subcommand (oracle --no-setup generate) — fixes empty PR comments on every previous run
• Commit ledger if changed step skipped on cross-fork PRs to prevent push-back failures

Other improvements

• Orchestrator scanner isolation: project_root threaded through run() so tests scan an isolated temp dir instead of Oracle's own source tree
• export const regex fix in DomainScanner — dotted TS type annotations (React.FC<Props>, z.ZodObject<...>) now captured correctly
• Pre-commit hook auto-refreshes security ledger and runs markdownlint on staged markdown files

Upgrading the GitHub Action

- uses: bri-stevenski/oracle-test-ai-agent@v2

or pin to the exact version:

- uses: bri-stevenski/oracle-test-ai-agent@v2.0.0

v1.0.1 — Fix output directory when installed as a package

Bug fix

oracle generate produced empty PR comments when run via the GitHub Action.

When Oracle is pip-installed (as the action does with pip install "${GITHUB_ACTION_PATH}"), __file__ in orchestrator.py resolves to the site-packages directory. The mkdir call on that path raised PermissionError, crashing oracle generate before any JSON was written to stdout. The action silently swallowed the error (|| RESULT=""), resulting in a PR comment with blank test type, framework, and output fields.

Fix: switched output_dir from Path(__file__).resolve().parents[2] to Path.cwd(), so generated tests are always written relative to the invoking project directory.

Upgrade

Users pinned to @v1 receive this fix automatically. To pin explicitly:

- uses: bri-stevenski/oracle-test-ai-agent@v1.0.1

v1.0.0 — Initial Release

Oracle Test Generator — v1.0.0

First stable release of the Oracle GitHub Action. Auto-generates tests for changed files on pull requests using AI (Claude, OpenAI, or Gemini).

Known issue fixed in v1.0.1: PR comments show empty outputs (blank test type, framework, and file). Caused by oracle generate crashing on startup when pip-installed due to a bad output directory path. Upgrade to @v1 or @v1.0.1 to get the fix.

Usage

- uses: bri-stevenski/oracle-test-ai-agent@v1
  with:
    api-key: ${{ secrets.ANTHROPIC_API_KEY }}
    provider: claude        # claude | openai | gemini | mock
    run-tests: 'false'      # execute generated tests after generation
    comment: 'true'         # post a PR comment with the test preview

Inputs

Input	Required	Default	Description
api-key	yes	—	API key for your LLM provider
provider	no	claude	LLM provider: claude, openai, gemini, mock
prompt	no	auto-derived	Custom generation prompt
run-tests	no	false	Execute generated tests after generation
comment	no	true	Post a PR comment with the generation summary
python-version	no	3.11	Python version for the Oracle CLI

Outputs

Output	Description
output-file	Path to the generated test file
test-type	Classified test type (e2e_ui, api, frontend_unit, performance, python_unit)
framework	Recommended test framework (playwright, pytest, vitest, k6)

What's included

• Composite GitHub Action wrapping the Oracle CLI
• Auto-detection of changed source files from PR diff
• AI-powered test generation with framework classification
• PR comment with generated test preview
• Support for Claude, OpenAI, and Gemini providers