v0.2.2

🐦 PromptCanary v0.2.2

Detect silent behavioral drift in LLM providers — before it breaks production.

This is a stabilization release. No new features — every change here exists
to make the project's own CI, packaging, and public-facing links trustworthy.
If you're evaluating PromptCanary for the first time, this is the version to start with.

What's Fixed

Two CI-only mypy failures, both caused by the same root issue: local
development environments had accumulated packages over time that were never
declared in pyproject.toml, so checks passed locally but failed in CI's
genuinely clean install.

• types-PyYAML was missing from [dev] extras — without it, yaml.dump()
  returns Any instead of str, which mypy --strict correctly flags
  against CanarySuite.to_yaml_template()'s declared return type.
• pandas was declared in [viz] extras but never actually used anywhere
  in the codebase — its only effect was pulling in numpy transitively.
  numpy 2.5's type stubs require Python 3.12+ to parse, conflicting with
  this project's deliberate python_version = "3.10" mypy target (chosen
  because we support 3.10+). Removed the unused dependency rather than
  working around the symptom. Full investigation in DECISION_LOG.md ADR-011.

Both were caught by adopting clean-room verification as standard
practice: installing only what pyproject.toml declares into a fresh,
disposable virtualenv before every release, rather than trusting a
long-lived dev environment. This is now documented as a permanent step in
CONTRIBUTING.md's new Release Checklist.

What's Corrected

• Fixed the placeholder GitHub org across the entire project. Every
  badge, URL, and cross-reference — 32 occurrences across the README, CI
  workflows, issue templates, pyproject.toml, mkdocs.yml, docstrings,
  and documentation pages — pointed to a placeholder org. All now correctly
  point to the real repository.
• Fixed a related inconsistency: the documentation URL pointed to a
  ReadTheDocs domain that was never configured; the actual deploy target
  (GitHub Pages, via the existing docs.yml workflow) is now referenced
  correctly everywhere.
• New badge row on the README: CI status, PyPI version, Python version,
  license, Ruff, mypy, a live view counter, and a one-click Colab launch
  badge for the notebooks.
• New notebooks/README.md with direct "Open in Colab" links for all four
  notebooks — anyone can now run the interactive examples with zero local
  setup.

PromptCanary v0.2.0

PromptCanary v0.2.0

Detect silent behavioral drift in LLM providers — before it breaks production.

This release responds directly to community feedback on v0.1.0: current-generation
model examples throughout, first-class support for free local models, a new
Tool Use probe category for agent workflows, trend visualization, property-based
tests, and a full documentation site.

✨ Highlights

Tool Use probes (new category) — silent drift in agent/function-calling
behavior is one of the most damaging and hardest-to-debug regressions. Four new
probes (ToolCallPresenceProbe, ToolCallNameProbe, ToolCallArgsProbe,
ToolCallSchemaProbe) parse OpenAI, Anthropic, and generic JSON tool-call
formats natively — no per-provider configuration needed.

Trend visualization — promptcanary.utils.visualization adds score-history
charts, per-probe heatmaps, and drift timelines. Zero-dependency ASCII rendering
works everywhere; pip install promptcanary[viz] upgrades to interactive Plotly
HTML with no loss of functionality either way.

Updated to current-generation models — README, CLI help text, examples, and
notebooks now reference GPT-5.x, Claude 4.x, and Gemini 3.x instead of legacy
2024-era model strings.

Free local models are now first-class — the Supported Providers table leads
with cost and API-key requirements per provider, and explicitly documents Ollama
models (qwen3.6:27b, deepseek-r1:14b, gpt-oss:20b) as zero-cost canaries
suitable for high-frequency CI checks, with a full cost-aware scheduling guide.

Full documentation site — 23-page MkDocs Material site covering getting
started, core concepts, every probe with examples, per-provider guides,
CI/CD patterns, LangChain/FastAPI integration guides, and auto-generated API
reference. Builds clean in --strict mode (zero broken links).

Property-based testing — Hypothesis now proves core invariants hold across
arbitrary inputs: scores always in [0.0, 1.0], baseline round-trips are
lossless, identical runs never produce false-positive drift.

📊 By the Numbers

• 19 built-in probes across 5 categories (was 15)
• 235 tests passing, 89% coverage (was 141 tests, 74%)
• 23-page documentation site (new)
• 4 Jupyter notebooks (was 2)
• Full CLI test coverage via typer.testing.CliRunner (new)

🔧 Fixed

• @probe decorator now correctly satisfies the BaseProbe ABC contract under
  all conditions (rewritten using type() construction)
• ToolCallArgsProbe now correctly parses OpenAI's JSON-encoded arguments
  string format inside tool_calls[]

📦 Install

pip install --upgrade promptcanary

🚀 Try It

promptcanary init my-suite
cd my-suite
promptcanary run --provider openai/gpt-5.4 --save-baseline
# ...or free and local:
ollama pull qwen3.6:27b
promptcanary run --provider ollama/qwen3.6:27b --save-baseline

Full changelog: CHANGELOG.md
Architecture rationale: DECISION_LOG.md
Docs: https://promptcanary.readthedocs.io

---

As always — PromptCanary is community-driven. New probes, canary suites, and
storage backends are the highest-value contributions. See CONTRIBUTING.md.