Canary v2.0.0 — Repository-Aware Semantic Regression Analysis

Canary v2.0.0 🐤

Canary v2 introduces repository-aware semantic regression analysis for Python pull requests.

Instead of stopping at “this API changed,” Canary can now trace that change across the repository, find affected call sites, and determine when an existing call is no longer compatible with the new interface.

API change
    ↓
Python AST analysis
    ↓
semantic compatibility detection
    ↓
repository-wide call-site discovery
    ↓
impact analysis
    ↓
argument-aware validation
    ↓
confirmed breakages

What's new

Semantic API compatibility analysis

Canary now parses BASE and HEAD versions of changed Python files using the Python AST and detects:

• removed public APIs
• newly added required parameters
• removed parameters
• reordered parameters
• removed parameter defaults
• return annotation changes
• sync ↔ async behavior changes

Example:

-def authenticate(token):
+def authenticate(token, strict):
     ...

Canary reports:

REQUIRED_PARAMETER_ADDED

and records the old and new callable signatures as technical evidence.

---

Repository-wide call-site analysis

Canary now builds a lightweight repository symbol and call index.

When an API changes, Canary can locate callers across other Python files rather than analyzing only the changed diff.

For example:

from auth import authenticate

def login():
    return authenticate("demo-token")

If authenticate() gains a new required argument, Canary connects the API change to this existing caller.

---

Argument-aware compatibility validation

Discovered call sites are validated against the changed function signature.

Canary accounts for:

• positional arguments
• keyword arguments
• required and optional parameters
• positional-only parameters
• keyword-only parameters
• excess positional arguments
• unexpected keywords
• duplicate argument binding
• *args
• **kwargs
• awaited and non-awaited calls

Each call site is classified as:

Status	Meaning
BREAKS	The call can be statically confirmed incompatible
UNAFFECTED	The call remains compatible
UNKNOWN	Static analysis cannot safely determine compatibility

---

Repository impact and blast-radius analysis

Semantic findings now carry repository impact information.

Canary reports:

• call sites analyzed
• confirmed breaking calls
• compatible calls
• calls requiring manual review
• exact affected file and line locations

This separates a potentially risky API change from a confirmed repository breakage.

---

Richer GitHub Checks

GitHub Checks now include repository-aware impact information alongside the original risk analysis.

A breaking change can produce output such as:

🔴 HIGH RISK

REQUIRED_PARAMETER_ADDED

Required parameter `strict` was added to `authenticate`.

Repository impact
1 call site analyzed
1 confirmed breaking

Confirmed breakages
caller.py:5

Canary also provides:

• HIGH / MEDIUM / LOW risk summaries
• inline annotations
• technical old → new signature evidence
• repository-impact tables
• merge-review recommendations

High-risk findings fail the Canary Check so regressions are visible before merge.

---

Richer CLI analysis

The terminal now exposes the same semantic impact information as the GitHub App:

canary inspect https://github.com/owner/repository/pull/123

GitHub Checks and the CLI share the same analysis engine.

---

Architecture

Canary v2 expands the original diff-based pipeline into:

GitHub PR
    ↓
BASE / HEAD source snapshots
    ↓
Python AST
    ↓
Compatibility Engine
    ↓
Repository Symbol / Call Index
    ↓
Impact Analysis
    ↓
Argument-Aware Call Validation
    ↓
AnalysisResult
    ↓
GitHub Check / CLI

The analysis remains deterministic and explainable and does not rely on an LLM.

---

Live validation

v2 was smoke-tested end-to-end against a real GitHub pull request containing an intentional breaking API change.

Canary successfully:

• detected the newly required parameter
• classified the finding as HIGH risk
• found the affected cross-file call site
• confirmed the call was incompatible
• reported the exact caller.py:5 location
• produced an inline GitHub annotation
• failed the GitHub Check as expected

---

Test suite

Canary v2.0.0 ships with:

88 passing tests

covering:

• AST extraction
• semantic compatibility rules
• repository symbol analysis
• cross-file call-site discovery
• impact analysis
• argument-aware validation
• pull-request orchestration
• GitHub integration
• GitHub Check presentation
• CLI behavior
• terminal presentation
• webhook behavior

Canary v1.0.0 — Initial Stable Release

Canary v1.0.0 🐤

Catch risky API changes before they merge.

Canary is a deterministic behavioral regression detector for GitHub pull requests. It analyzes Python code changes, identifies potentially breaking function-interface changes, and surfaces the results directly through GitHub Checks or from the terminal.

This release establishes Canary's first stable analysis pipeline and the foundation for deeper repository-aware regression analysis in v2.0.

---

✦ Highlights

• GitHub App integration — automatically analyzes pull request events
• Python diff analysis — inspects changed Python code
• Function signature regression detection
• HIGH / MEDIUM / LOW risk classification
• GitHub Check output with inline annotations
• Terminal PR inspection
• Installable canary CLI
• Unit and integration test coverage

---

Deterministic regression detection

Canary v1 focuses on changes to Python callable interfaces, including:

• Function signature changes
• Parameter changes
• Default-value changes
• Type annotation changes
• Return annotation changes
• Sync / async declaration changes

Public API changes receive higher severity than private implementation changes.

Example

Before

def create_user(name: str) -> User:

After

def create_user(name: str, organization_id: int) -> User:

Canary identifies changes like this as potentially breaking public API changes and surfaces them during code review, before merge.

---

CLI

Inspect a pull request directly from the terminal:

canary inspect https://github.com/owner/repository/pull/123

The CLI uses the same analysis pipeline as the GitHub App.

---

Design philosophy

Canary v1.0 deliberately uses deterministic, explainable regression rules rather than opaque scoring or LLM-based review.

What behavior or interface might this change break?

The goal is not simply to describe a diff. It is to surface changes that may alter the contract other code depends on.

---

→ What's next: Canary v2.0

Canary v2.0 expands the engine into repository-aware semantic regression analysis, including:

• AST-based compatibility detection
• Repository-wide symbol analysis
• Cross-file call-site resolution
• Dependency and blast-radius analysis
• Argument-aware call validation
• Richer GitHub Check explanations
• Richer terminal analysis

---

v1.0 establishes the baseline: catch risky API changes before they merge.