Skip to content

ANF v0.2.0 — Frontier Benchmark & Paper

Choose a tag to compare

View all tags
@shreyanshjain7174 shreyanshjain7174 released this 15 May 07:02
· 13 commits to main since this release
v0.2.0-paper
073862f

ACP: Intent-Resolved Execution Manifests for Token-Efficient Agent Tool Use

Paper PDF attached below. Benchmarked on Claude Opus 4.7, Sonnet 4.6, GPT-5.5, GPT-5.4.

Highlights

  • 120 API calls across 4 frontier models × 5 scenarios × 2 paths (MCP/ACP)
  • Provider-reported input token reduction: 26–93% depending on model/scenario
  • tiktoken/cl100k_base control column for deterministic normalization
  • Tool-call correctness preserved or improved (GPT-5.4 S4: 0% → 100%)
  • Competitive landscape: positioned against TSCG (arXiv:2605.04107) and MCP-Zero (arXiv:2506.01056)
  • BFCL (ICML 2025) selected as open benchmark standard for next track

What's in this release

  • paper/acp.md — Markdown paper source (canonical)
  • paper/acp.tex + paper/references.bib — LaTeX source for preprint
  • benchmark/frontier/run_frontier.py — Frontier benchmark harness
  • results/frontier/ — Raw JSON + summary for all 120 runs
  • ACP-Paper-2026.pdf — Compiled paper (attached)

Install

go install github.com/Clawdlinux/ninevigil-acp/cmd/acp-server@v0.2.0-paper
Assets 3
Loading