Skip to content

Releases: roy-tong/Verovo

v0.9.0 — Verovo

Choose a tag to compare

@github-actions github-actions released this 30 Oct 08:07

v0.9.0 — AgentMeasure is now Verovo

CLI release 0.9.0; Provider SDK 0.3.0 (@verovo/mcp). Same measure, new name,
new logo.

The project is renamed from AgentMeasure to Verovo. Nothing about what the
tool claims to do changed in this release: it still re-counts supported billing
exports locally, keeps findings and gaps separate, and reports unknown as unknown.
This release is the rename, plus the compatibility work the rename required.

What changes for you

Area Before Now
Install and CLI pip install agentmeasure, agentmeasure … pip install verovo, verovo …
Provider SDK @agentmeasure/mcp @verovo/mcp (0.3.0)
Environment variables AGENTMEASURE_*, AM_* VEROVO_* (e.g. VEROVO_EVENTS_DIR, VEROVO_LAB_DEBUG)
Local data directory ~/.agentmeasure/ ~/.verovo/
Schema namespaces agentmeasure.lab/…, agentmeasure.commercial/… verovo.lab/…, verovo.commercial/…
Settlement standard AMS-1 VSS-1 (Verovo Settlement Statement)
Repository and site roy-tong/AgentMeasure roy-tong/verovo, roy-tong.github.io/verovo

If you scripted against the old names, this is a breaking rename; pin 0.8.0 if you
need the previous surface for a migration window.

What does not change

  • Recorded evidence stays valid. External conformance vectors
    (conformance/external, conformance/vectors/external) and recorded run logs
    are kept byte-for-byte under the old namespace. The schema accepts
    agentmeasure.lab/funnel-event and agentmeasure.lab/report as permanent legacy
    aliases, so old fixtures still validate and still reproduce their expected
    metrics.
  • Measurements, verdicts and boundaries are unchanged. PASS / FAIL /
    UNPROVABLE still mean what they meant; missing evidence still lowers a posture
    instead of filling the gap.
  • No new claim. A rename is not adoption, not a customer, not a validation of
    the commercial thesis.

Compatibility notes

  • JSON exports keep the tool tag verovo-healthcheck (was
    agentmeasure-healthcheck); schema version and field names are unchanged.
  • The settlement standard keeps its numeric suffix: AMS-1 became VSS-1, so
    references to "VSS-1 S-9" in conformance runners line up again.
  • All 511 healthcheck tests, 91 lab tests, 106 buyer-ledger tests, 21 SDK tests and
    the conformance runners pass after the rename.

Brand

The new wordmark and owl mark live in assets/ and website/assets/; usage rules
are in docs/brand.md. The mark is the two eyes that spell the "vo" of the
wordmark — the part that reads evidence, not the part that advertises it.

v0.8.0 — AgentMeasure

Choose a tag to compare

@github-actions github-actions released this 15 Oct 15:36

v0.8.0 — Evidence-qualified recounts and repeatable reviews

CLI release 0.8.0; Provider SDK version remains 0.2.0.

agentmeasure recount --sample runs a bundled five-row synthetic export. Three
showcase cases cover findings in both directions, missing evidence and a separate
buyer contract lane. They are demonstrations, not customer recovery results.

The report headline distinguishes published-price estimates from verified invoice
totals, displays unsettled rows, and labels signed net variance without implying
refunds. Zero identified variance does not clear an incomplete bill.

The optional monthly workflow requires an independent private repository and
uploads data to GitHub. It installs the reviewed source revision, checks the
billing period and input hash, and stops stale scheduled input. Optional issues
track unresolved or offsetting findings per vendor and period. Each new period
still needs a reviewed export; see RECONCILE.md. Local CLI handling remains available.

The website routes developers to local checks and $10/$20 bounded help, and buyers
to separate free feasibility review. A PDF total alone cannot support recount.
Vendor rule sources remain inspectable; no payment or PMF is established by tests.

Validation: 511 Healthcheck tests, including 21 new regressions for report evidence
boundaries, stale input, path confinement and unresolved/offsetting findings.
Upgrade: python -m pip install --upgrade agentmeasure.

agentmeasure response-audit FILE.jsonl [FILE.jsonl ...] adds a read-only Claude
cross-file evidence view: response identity reuse, usage conflicts and missing
fields. It never rewrites logs, selects first/last/max usage or adds cache subsets
to totals. Session and invoice totals remain unknown. Ten regressions cover
missing IDs, conflicting snapshots, bad input and output overwrite protection.

v0.7.2 — AgentMeasure

Choose a tag to compare

@github-actions github-actions released this 15 Oct 09:56

v0.7.2 — Route user inputs to the right checker

Canonical provider observations and FMT-002 are different contracts. A valid provider submission previously produced unrelated FMT-002 missing-field errors when passed to conformance.

agentmeasure inspect-input sample.jsonl --json routing.json now identifies the input shape and explains the next step. Recognition never claims schema validation or a measurement verdict: both are explicitly NOT-RUN. Mixed, invalid, ambiguous, empty and unsupported inputs return exit 2. No source rows are included in the report, and no fields are invented or converted.

Conformance now intercepts recognized incompatible inputs with a targeted recovery path. Existing FMT-002 validation remains mandatory. Canonical semantic analysis still uses the provider workflow; this release does not claim universal input support, token/cache cost verification or live quota monitoring.

Upgrade: python -m pip install --upgrade agentmeasure.

Validation: 487 Healthcheck tests; installed-wheel smoke with input routing; required repository checks. SDK remains 0.2.0. Research feedback and test results do not establish payment or PMF.

v0.7.1 — AgentMeasure

Choose a tag to compare

@github-actions github-actions released this 14 Oct 22:02

v0.7.1 — Fix installed conformance and first-run friction

A clean PyPI install of 0.7.0 could run Healthcheck but failed on conformance
with No module named am_healthcheck.lab.analysis. This patch includes that
subpackage in the wheel and exercises it in the installed-artifact smoke test.

  • conformance --new-vector defaults to the caller's working directory, not the Python installation.
  • conformance without a fixture returns a useful input error and a sample-generation command.
  • share --out says the file was saved locally and should be inspected; it no longer claims the user already reviewed it.
  • Zero-cost fixtures retain their conservative UNPROVABLE verdict, with an accurate reason rather than claiming that cost fields are absent.
  • Website first-run guidance, illustrative reports and email templates cover no-mail-client, missing-input and checkout-pending paths.

Upgrade

python -m pip install --upgrade agentmeasure
agentmeasure conformance --new-vector my-sample --out ./vectors
agentmeasure conformance --fixture ./vectors/my-sample-001/agentmeasure_my_sample_fixture_001.events.jsonl

The generated fixture is synthetic: 4 PASS, 1 UNPROVABLE for zero-cost preservation;
token/cache rules remain NOT-SUPPORTED. Replace placeholders with real evidence
before interpreting your own data. SDK version remains 0.2.0.

Validation: 479 Healthcheck unit tests, installed-wheel smoke including sample
generation and conformance, and the repository's required checks. User-journey
simulations are not customer adoption or payment evidence. Online checkout is
still being configured; no purchase or refund was simulated as a real transaction.

v0.7.0 — claims index, community wave, SDK 0.2.0, paid diagnosis

Choose a tag to compare

@github-actions github-actions released this 14 Oct 01:47

v0.7.0 — The claims index, a growing community, and paid diagnosis

Outcome-based pricing won the pricing page. Who verifies the billed outcome?

What's new

  • Metering claims index — every finding from the 110-tool audit is now a linkable entry: one status (Verified Mismatch / Fixed · Re-verified / Conformance Pass / UNPROVABLE / Not a bug), one date, one public link. No composite scores, no ranking, no paying for placement. https://roy-tong.github.io/AgentMeasure/claims.html
  • The community is growing — this cycle merged three external PRs (an SSE conformance vector #28, a Freshdesk field mapping #30, an Intercom walkthrough #35) plus the first availability-certification vector contributed via PR #32 (@gunjanjaswal); good-first-issues #23/#24 were closed by contributors. agentmeasure conformance --new-vector <name> now scaffolds a CI-valid external vector in one command.
  • Paid diagnosis, live — two fixed-price, human-delivered diagnostics: a $10 healthcheck anomaly diagnosis and a $20 metering QA single-sample check. Apply by email, 24h turnaround, refund terms on the page; the free CLI stays free. https://roy-tong.github.io/AgentMeasure/services.html
  • SDK 0.2.0 on npm — tool annotations (readOnlyHint/destructiveHint/idempotentHint/openWorldHint) declared in the metering standard's own MCP: npm i @agentmeasure/mcp.
  • Windows — console output degrades unencodable glyphs instead of crashing on legacy code pages (cp1252/cp936), and CI now runs the full unit suite on windows-latest.
  • validate accepts the standalone share summary — agentmeasure share report.json --out summary.json then agentmeasure validate summary.json prints valid share-summary; tampered files are named, never waved through.

Install

pip install agentmeasure      # 0.7.0 on PyPI
npm i @agentmeasure/mcp       # 0.2.0 on npm
agentmeasure demo             # synthetic example, 30 seconds

Numbers as of 2026-10-14: 477 healthcheck tests green (macOS + Ubuntu + Windows CI); 110 tools audited, 45+ reproduced metering bugs, 25 upstream fixes merged — every claim links from the index above.

v0.6.0 — AgentMeasure

Choose a tag to compare

@github-actions github-actions released this 11 Oct 00:11

v0.6.0 — distribution assets, drill-down, and the three value books

Who this is for

  • CX & support operations reviewing per-resolution AI support bills (Intercom Fin, Zendesk AI Agents…) — before paying, disputing or renewing.
  • Finance & procurement who need the evidence behind a number, one finding at a time, in a form they can hand to the vendor.
  • Brands running agent channels measuring agent-driven orders, margin and guardrails from the merchant side.
  • Tool builders & maintainers shipping conformance vectors, fixtures and corrections into the open standard.

What you can do with this release

  • Re-check any single finding with agentmeasure drilldown — the row, the rule steps in plain language, the verdict, the repro command.
  • Keep three separate value books: recovered cash (recovery), hours saved (delivery), decisions made differently (decisions — new).
  • Send a link, not a PDF: public vendor rule cards (versioned, sourced) and AMS-1 badges (a four-stage evidence ladder — no composite scores).
  • Install from PyPI, npm or GitHub; CI smoke-tests the packaged wheel in a fresh environment on every push.

Public release-notes page (kept current with the site): https://roy-tong.github.io/AgentMeasure/releases.html

Since v0.5.0: the verification system gained its public distribution layer
and closed the two product gaps BP r32 named. Both package channels are now
live — this release ships to GitHub, PyPI, and npm in one push.

Re-check one finding

agentmeasure drilldown replays a single conversation on screen: the row
exactly as the buyer exported it, every vendor rule step that fired in plain
language, the verdict with its three-state, the amount at issue, and the
exact command to reproduce it. The explanation walks the same branches as
the recount in the same order — a test replays the whole fixture through
both engines and asserts the verdicts agree, so the explanation and the
judgement cannot drift apart. This is the "ask the customer to re-check one
finding" promise, as an artifact.

The third value book

Cash (recovery ledger) and hours (delivery journal) are joined by
agentmeasure decisions: an append-only journal of how the buyer used the
verification in a business decision — payment approved or held, discount
requested, renewal signed, vendor review. Types are enum-validated, entries
link back to the finding they rest on, outcomes are stated, and amounts are
context that is never summed with recovered cash. Three kinds of value,
three separate books.

Distribution layer

  • Vendor rule cards (website/rules/) — one public, linkable card per
    vendor: price, billing trigger, reopen handling, silence behaviour,
    closure timers, source link, registry version. Generated from the same
    registry the CLI and browser recount read, with a CI check so the page
    can never drift.
  • AMS-1 badges (website/badges.html) — the four-stage adoption ladder
    as embeddable SVGs with markdown snippets, an adopters snapshot listing
    external conformance authors with links, and an explicit statement of
    what we refuse: no composite score, no A–F grade.
  • The website now mirrors to gh-pages on every green CI run, so new pages
    ship themselves.

Also in this release

Three new external conformance vectors/cards (iwasinnam-003 Metrecept
session-aggregate receipts; rksharma-001 contribution card; chenhz01-001 —
the first availability-certification vector, contributed via PR #32), a
release-smoke CI job that installs the built wheel into a fresh venv and
runs the first-user paths (--version / demo / selftest), and an
idempotent release workflow (PyPI and npm steps skip already-published
versions, so re-runs never fail).

466 tests. Zero runtime dependencies; no network code; judgement stays
100% rule-based; no message text is ever read.

v0.5.0 — AgentMeasure

Choose a tag to compare

@github-actions github-actions released this 03 Oct 05:20

v0.5.0 — the buyer-side verification system, complete

Everything since v0.4.1 lands in one release: the buyer-side verification
pipeline taken from "tools" to "a system a normal user can operate", plus a
full agent-commerce measurement family, plus the console that ties both
together. 450 tests hold the line (248 at v0.4.1). Zero runtime
dependencies; no network code (an import guard covers every new module);
judgement stays 100% rule-based; no message text is ever read.

The buyer-side review pipeline

prepare maps a native Intercom/Zendesk export onto the canonical columns —
judgement columns stay empty for human review, native signals travel as
hint_* leads, every value carries prov_* provenance, and --inspect
aligns before any number exists. recount applies the vendor's own
published rules line by line, findings in both directions, UNPROVABLE
never guessed — with an optional --contract overlay whose outcome-standard
findings are a separate lane, never netted into the claim. dispute builds
the negotiable pack; recovery records what the vendor actually conceded;
crosscheck puts the buyer's billing ledger beside the export and names
every disagreement without resolving it; crm-join fills empty judgement
cells from the buyer's own business records and names conflicts. verify
composes all lanes into one Verified Ledger; rules-diff gates material
vendor-rule changes for the monthly ritual.

Agent Commerce Measurement

The chain where the click disappears, measured from the merchant side:
commerce-ledger produces the measurement receipt (metric + value +
evidence, materiality vs the naive dashboard in currency AND percent), with
live/replay/synthetic segregated at the schema level and a canonical retry
vector pinned in CI (1 Operation / 2 Attempts / 1 verified order /
net GMV = 0). The operating layer on top: pnl-trees (Organic and Paid,
never blended), cm-ledger (the seven-line contribution margin),
demand-audit (channel potential vs operator performance), intent-taxonomy
(Operating Cell = cluster × surface × proposition), decision-audit
(auto/approval/forbidden with the policy citation on every execution), and
benchmark-export (cross-brand aggregates only — customer data cannot
structurally enter the shared pool).

The console

agentmeasure console renders the verification documents as one offline
app: sidebar navigation over two user journeys (vendor bill review; agent
channel operations), an overview workbench whose stage states are derived
from the documents, hash-routed views, a print mode that flattens every
view, and a test-enforced copy discipline that refuses draft or internal
language. Single file, no network, escaped values. Design checked with the
impeccable Operate-mode playbook and its detector (zero findings).

Also in this release

outcomes — the outcome-unit verification engine (fail-closed: counted +
criteria_unproven is an inconsistency, refused); outcome-unit schema v0.2
covering qualified_lead, completed_case, action (allowance-aware), and
usage_aggregate — one evidence plane across the billing stages;
period-compare for the monthly delta; template for appendix-E reuse;
certify grading artifacts on the Stage 0..3 adoption ladder with a
deterministic digest; incrementality producing treatment-vs-holdout
statistics that never masquerade as a verdict.

Channels

  • GitHub Release (this page) — ships on tag push.
  • npm @agentmeasure/mcp — published by the same workflow when the token
    is present.
  • PyPI agentmeasure — the workflow attempts trusted publishing; the
    PyPI-side publisher configuration is a project-owner setting. If this
    release's publish job fails the same way v0.4.1's did, configure the
    pending publisher on pypi.org and re-run the failed job — no rebuild
    needed.

v0.4.1 — AgentMeasure

Choose a tag to compare

@github-actions github-actions released this 22 Sep 13:14

v0.4.1 — CLI regression fix

Fixes the PyPI 0.4.0 first-command crash reported by a real user (morpheus): _run_check read an unbound args for audit_mode / sdk_events_path (NameError on demo / check / compare). Main already passed the values as parameters; this release ships that fix to PyPI.

  • fix(cli): 717c2d3 — release 0.4.1 (regression shipped in 0.4.0)
  • docs: e4d287d — README warns PyPI served crashing 0.4.0; git install as the working door

Install: pip install agentmeasure==0.4.1 (or pipx upgrade agentmeasure).

v0.4.0 — AgentMeasure

Choose a tag to compare

@github-actions github-actions released this 19 Sep 08:07

v0.4.0 — PyPI, Claude Code adapter, conformance pack, settlement bundle

First release published to PyPI — pipx run agentmeasure works with no
repo checkout. Healthcheck remains local-only: no network calls at runtime,
no upload, no account.

Highlights

  • PyPI distribution (agentmeasure): pipx run agentmeasure demo /
    pipx run agentmeasure check — Claude Code adapter v1 joins the Codex
    rollout reader (HC-01..06 audit checks: duplicate records, retry chains,
    consecutive tool failures, operation-resolution coverage, cache accounting,
    token stability). Missing evidence is UNPROVABLE, never silently zero.
  • Embedded conformance pack (agentmeasure conformance, also a
    GitHub Action): run the fixture suite against your own
    parser — plain JSONL + expected totals, no runtime install.
  • OTel / Prometheus exports and run trends (agentmeasure trend).
  • Settlement bundle drafts for outcome-based billing
    (agentmeasure settle) — see
    proposals/2026-09-18-settlement-bundle-design.md.

Ecosystem

  • AMS-1 settlement standard (Draft 0.1):
    verdict taxonomy, clauses S-1..S-8 each enforced by a named test,
    machine-readable clause manifest,
    and a one-pager statement generator (agentmeasure settle --format md)
    with input-sha256 provenance and a third-party reproduction block.
  • 21 verified metering fixes merged upstream across the ~110-tool audit —
    including langfuse, litellm, and codeburn. Full report:
    The Token-Accounting Bug Report.

Checks

190 healthcheck unit tests · 21/21 metric vectors · 15/15 outcome-audit
vectors · 9/9 delegation vectors · core gate, external fixtures, and pack
selftest all green at this tag.

v0.3.0 — Ecosystem Audit edition: 33 verified findings, method as demo

Choose a tag to compare

@roy-tong roy-tong released this 09 Sep 05:56
991e4e0

v0.3.0 — Healthcheck: installable CLI + versioned export schemas

Released automatically from docs/RELEASE-v0.3.0.md by
.github/workflows/release.yml (tag v0.3.0).
First PyPI release of the agentmeasure command.


What's in this release

AgentMeasure Healthcheck — a local check-up report for coding agent runs.
Reads the logs your agent runtime already writes (Codex rollout first), checks
them for duplicate records, retry amplification, and tool-error runs, and
writes a terminal summary plus a local HTML report. Same honesty rules as the
conformance pack: OK / FINDING / UNPROVABLE, and UNPROVABLE is a first-class
result. Python 3.9+ standard library only; zero runtime dependencies; no
network code (tests enforce both).

Install

pipx install agentmeasure
# or: python3 -m pip install --user agentmeasure
agentmeasure demo    # synthetic session, no data needed
agentmeasure check   # your Codex logs, last 7 days

Headlines

  • Checks with evidence. HC-01 duplicate records (including the format's
    by-design dual-stream recording, disclosed as a ratio), HC-02 retry
    amplification (attempts vs logical operations), HC-03 tool-error runs — each
    finding carries file, line, and exit codes, plus a concrete next step.
  • Honest numbers. Token subsets never summed into totals; token aggregates
    UNPROVABLE unless every logical session contributes a valid cumulative
    snapshot; corrupt/truncated input reported as lower bounds, never dropped.
  • Snapshots & compare. check --save-snapshot + compare show what
    actually changed between two runs (verdict transitions, metric deltas,
    caveats on version/window mismatches).
  • Project breakdown & filters. cwd-basename project aggregates;
    --project, --since/--until, --days, --all.
  • Versioned export schemas (report-v1 / snapshot-v1 / compare-v1).
    Fields are only added within a version; agentmeasure validate checks any
    export; examples/track-weekly.py is a complete external consumer that
    imports no package internals.
  • Share-safe by construction, preview-then-export. agentmeasure share run.json previews the sanitized summary on the terminal and writes nothing
    until an explicit --out; the summary is whitelist-only (no prompts, paths,
    commands, repo names, session ids) and re-checked before either step —
    planted-string tests enforce it.

Verification

  • 101 unit tests + selftest (fixture verdicts, redaction, compare, schemas)
  • Clean-venv install smoke in CI (Python 3.9/3.11): build → install →
    selftest → demo → validate → external example, all from the packaged
    artifact
  • Verified against 55 local Codex rollout files (cli 0.142.3–0.153.0);
    snapshot-vs-fresh compare is fully deterministic

Boundaries (stated, not hidden)

  • Supported runtime: Codex rollout (Desktop). Claude Code is detected but not
    yet supported — its adapter ships after validation against real samples.
  • Engineering preview: quiet publish, no launch marketing yet. The GitHub
    entry points and naming follow the v4 launch plan.