Conformance

Conformance

Conformance testing provides a quality gate for persona upgrades. A persona pack can declare a minimum test score; newer versions must meet or exceed that score to install.

Conformance baseline

In pack.toml:

[conformance_baseline]
score = 0.92
bundle_hash = "sha256:..."

• score -- Minimum acceptable test score (0.0 to 1.0).
• bundle_hash -- SHA-256 hash of the test bundle that produced the baseline.

Test bundles

A test bundle is a set of test cases that exercise the persona's behavioral properties:

• Input: A task description or scenario.
• Expected behavior: What the persona should do (or not do) in response.
• Timeout: Maximum time for evaluation.

The frameshift-conformance crate provides the schema, a runner trait, and scoring logic.

Regression gate

When upgrading a persona, the regression gate:

1. Runs the test bundle against the new version.
2. Computes a score.
3. Compares against the declared baseline.
4. Blocks the upgrade if the score drops below the floor.

This prevents persona regressions -- a newer version that lost behavioral properties won't silently replace a working one.

Verification

frameshift verify <persona>

Runs conformance checks against an installed persona and reports the score.