How to measure Capsule visibility and token-efficiency results

[hakiyaka]

What Capsule measures

Capsule is a local-first Codex plugin and MCP server. It stores the original tool result locally, then emits a smaller model-visible evidence view when the workload is a safe candidate. The complete result remains recoverable through the Capsule lease or exact expansion path.

The useful number is workload-specific model-visible reduction. It is not automatically a reduction in hidden provider reasoning, provider cache usage, billing, or subscription quota. A meaningful report includes the workload, Capsule version, command, sample size, fallback rate, and recovery check.

Reproduce a measurement

npm run verify
npm run benchmark:web:zero-copy
npm run benchmark:get-content:history
npm run benchmark:project
npm run audit:github-visibility:write -- visibility.json
npm run audit:github-visibility:search:write -- visibility-search.json
npm run audit:github-visibility:search:baseline -- visibility-search.json

The explicit visibility helpers avoid npm's leading-option ambiguity on Windows. For a paired visibility snapshot, save a later report and compare it with the search baseline helper. The audit separates cumulative metadata (stars, forks, releases, topics) from the rolling 14-day views, clones, referral, and popular-path signals.

• Methodology: https://hakiyaka.github.io/capsule/guide/benchmarks-and-methodology.html
• Benchmark source: https://github.com/hakiyaka/capsule/blob/main/BENCHMARK.md
• Installation: https://hakiyaka.github.io/capsule/guide/install-capsule.html
• Repository: https://github.com/hakiyaka/capsule

If your result differs, include the exact workload and a sanitized reproduction. Do not include credentials, private prompts, or raw session logs.

[hakiyaka]

Updated for Capsule 1.0.2 and the current visibility-audit CLI:

npm run verify
npm run audit:github-visibility:write -- visibility.json
npm run audit:github-visibility:search:write -- visibility-search.json

# At a later measurement point, compare the same fixed queries:
npm run audit:github-visibility:search:baseline -- visibility-search.json

For a fork or another repository, set CAPSULE_GITHUB_REPO=owner/name or run node scripts/github-visibility-audit.cjs --repo owner/name. Do not rely on npm run ... -- --repo option forwarding on Windows.

The comparison reports null-safe rank_improvement (positive means a smaller, better rank), present_in_first_100_delta, and total_count_delta. A null rank means the repository was outside the first 100 or the result was unavailable; it is not evidence of a loss. Traffic and referrer fields are separate rolling 14-day GitHub signals. These snapshots do not prove Google ranking, provider billing, quota, or universal token savings.