Skip to content

v2.5.0

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 10 Oct 04:14
v2.5.0
75bdac0
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

v2.5: Local JSON reports and evidence calibration

API Relay Audit v2.5 adds machine-readable local reports and incorporates the
changes from the unpublished v2.4.1 draft. The previous public release is
v2.4.0. This release is based on commit 75bdac000051871c9f088a3be58e6f17a8854ffb.

What Changed

  • #106 adds opt-in
    --format json to both standalone audit.py and modular scripts/audit.py.
    Markdown remains the default. JSON schema version 1 carries run provenance,
    the existing final risk rating, ordered flags, coverage, and full Markdown
    evidence. Stdout contains one JSON document; progress goes to stderr.
    --output writes UTF-8 to a local file. Connectivity checks retain their
    0/1 exit codes and return risk_level: null and coverage: null.
  • #108 keeps weak
    prompt-related keyword matches INCONCLUSIVE. Cat Test reports preserve
    exact caller payloads and describe instruction non-adherence without
    asserting the mechanism or attributing it to a provider.
  • #104, included in
    the earlier draft, supports exact DSH peers 0.1.0-rc.6, 0.1.7-rc.2, and
    0.2.0-rc.2. Curl fallback passes credential headers through config stdin
    and holds request bodies in private temporary files.
  • #97,
    #80, and
    #102 add research
    references, evidence boundaries, and signal-interpretation guidance.

The 14 audit steps, general / web3 / full profiles, and existing six
dimension LOW / MEDIUM / HIGH risk matrix are unchanged. For example, Step 4
can remain inconclusive while the final rating is LOW: read individual
findings alongside the rating.

Quick Start

Tagged downloads below use the immutable v2.5.0 release tag.

AUDIT_SCRIPT_REF=v2.5.0
curl -fsSL "https://raw.githubusercontent.com/toby-bridges/api-relay-audit/${AUDIT_SCRIPT_REF}/audit.py" -o audit.py
export API_RELAY_AUDIT_KEY=your-relay-key
python3 audit.py --key-env API_RELAY_AUDIT_KEY --url https://relay.example.com/v1 --format json --output report.json

See the JSON schema and coverage boundaries.

Verification

  • Release gates: 885 Python tests, 19 DSH adapter tests, version sync,
    standalone drift, metrics drift, and zero-dependency standalone startup.
  • Eight real CLI loopback checks cover both distributions, successful and failed
    connectivity, stdout and UTF-8 file output, exit codes, and credential
    exclusion using synthetic credentials.
  • Required PR checks, master CI, and the draft-release workflow passed for the
    release commit. Source, generated standalone, downloaded release asset, and
    SHA-256 file were verified before publication.
  • Standalone audit.py SHA-256: fd99409bf745090a26bb4616fd33fbe0d873de08cfe077df413b2b93845f151d
  • Publication verification checks /releases/latest and
    /releases/tags/v2.5.0, the tag target, and attached assets. Local test
    success alone does not prove those external states.

Limits

JSON is an unredacted local execution report, separate from the draft public
report-artifact schema. It may contain private targets and model responses;
review and redact before sharing. The DSH integration retains its existing
Markdown workflow and Claude model baseline. This release adds no connector,
new detector, model baseline, or safety certification. Interactive DSH host
submission and real-provider attribution are outside this release's
verification.

Assets 4
Loading