Skip to content

AgentCheck 0.5.5 — scenario-aware confirmation and authority-bound evidence

Choose a tag to compare

View all tags
@WaseemGhanem98 WaseemGhanem98 released this 05 Sep 03:21
v0.5.5
41ebab5
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

An evidence-correctness patch: scenario-aware confirmation, authority-bound
coverage, and exact-wheel release qualification. The compatibility restrictions
below intentionally refuse evidence that cannot support its recorded claims.
Generation, framework support, containment and existing baselines are unchanged.

Fixed

  • Confirmation-before-tool evaluation now distinguishes scenario-supplied,
    tool-scoped consent, deliberate withholding, and absent context. Compliant
    completed withholding and optional action refusal can pass without claiming
    consent was exercised; absent context remains inconclusive with or without
    a call. Positive consent must bind to a delivered user turn before every
    consistently recorded call. Prose, run-only flags, ambiguous scope and
    incomplete no-call evidence cannot produce a pass. This intentionally tightens
    acceptance of historical unbound consent evidence; see docs/behavioral-policies.md.

    Suite identity: unchanged; no generated cases, obligations or denominators removed.

  • Stored-run comparison now reports spec_changed when a tool's risk changes
    between inferred and authoritative, even if its resolved risk values and
    recorded coverage digest stay the same. Each run's recorded specification
    supplies the authority; existing artifacts, scenario classifications, and
    comparison exit codes are unchanged.

    Suite identity: unchanged.

  • Coverage now binds effective per-axis tool-risk authority using a versioned
    specification digest. Existing checksums, scenario denominators and source
    checks are retained. Unknown digest algorithms fail closed; mixed-algorithm
    comparisons do not report a spec change merely because the algorithm differs.

    Compatibility restriction: older selected-run coverage with an omitted
    reference set cannot prove its recorded authority and is now rejected by
    stored-run consumers. Fully rederivable legacy coverage and summaries that
    originally lacked coverage remain readable. No stored file is rewritten;
    see docs/behavioral-coverage.md for recovery and the remaining reference limits.

    Suite identity: unchanged. New coverage digests/checksums intentionally change.

Release qualification

  • Publishing now qualifies the already-built wheel before uploading release
    artifacts. Clean base, OpenAI Agents and PydanticAI extra environments check
    direct-wheel hashes, installed import/version identity, missing-extra refusal
    and small offline confirmation-contract probes. Artifact bytes are checked
    again afterward; publishing still downloads those same artifacts without a
    rebuild. These are release gates, not a substitute for source review or proof
    of arbitrary target behavior.

Source commit: 41ebab5dc5cdb9607269baf2685fe6f554a93c86.

Full changelog: https://github.com/WaseemGhanem98/AgentCheck/blob/v0.5.5/CHANGELOG.md

Assets 2
Loading