🌱 AgentCC

Drop-in middleware that cuts your agent's token bill on autopilot. Wrap your LLM client in one line — then forget it. AgentCC meters every call and acts on the waste for you: routes easy calls to cheaper models, replays repeat responses from cache, and auto-stops runaway agents before they torch your budget. You ship; it saves.

Stop building your own cost-optimization layer. This is the control plane you'd otherwise hand-roll — budgets, hard caps, kill switch, loop/bloat detection, model routing, response caching (your DB or ours), and cost-per-success analytics — every lever flipped from a dashboard, never from your code. Integrate once; tune it for life without a redeploy.

Never sees your API keys, prompts, or completions. Never adds latency to a call.

See the site → · Open the live dashboard →

---

🌿 Why it matters

Agent token cost is an operating discipline, not a prompt-trimming chore. The goal isn't "fewer tokens" — it's the lowest cost per successful completion with reliability, governance, and an audit trail intact. AgentCC turns the practical levers (routing, caching, enforcement) into a board-operable model.

Benefit	What you get
🎯 Optimize cost per successful completion	The board metric, not raw tokens: spend ÷ successful completions, tracked by workflow. reportOutcome("success"|"failure"|"rework") feeds it; the board dashboard turns it into unit economics. Cheap answers that cause rework/escalation don't count as savings.
🔀 Routing & cascading	Send low-difficulty work to cheaper models, keep strong models for the hard calls — auto-heuristic or explicit policy, set from the dashboard and pushed to the SDK live.
♻️ Response caching — built-in + BYODB	Replay identical requests for $0 from memory, the hosted store (storage-capped), or your own Redis/Upstash. Keyed on a content-free fingerprint, zero added latency, fails open.
🦷 Enforcement, not just dashboards	Budgets + auto-stop, a hard call cap, and a one-click kill switch actually stop a runaway agent mid-loop (AgentKilledError or a graceful fallback). Visibility tools only watch.
🤖 Auto-remediation control plane	Policies act on waste signals automatically — downshift or kill on threshold, with cooldowns and a full audit log of every action.
📊 Board dashboard	A monthly Cost / Quality / Danger view: cost per success, cache hit rate, escalation & rework rates, policy blocks, and incidents.
🔒 Private & governed by design	No prompts, completions, or API keys ever leave your process — only the usage object + content-free metadata (fingerprint, tool names, output hash). Per-owner isolation + audit evidence.
⚡ One line, any framework, zero latency	withCostControl(client, opts) — same methods, types, and return values. OpenAI · Vercel AI SDK · Mastra · LangChain / LangGraph · OpenAI Agents, one shared pipeline. Telemetry is post-response, fire-and-forget.

Scope & honesty: routing and caching currently take effect on the OpenAI adapter (withCostControl(client, …)); the other adapters record telemetry + honor the kill switch. Today's cache is exact-match (same prompt + model) — embedding-based semantic caching, prefix-segment caching, and lazy tool-loading are on the roadmap, not claimed as shipped. Routing is heuristic/policy-based, not yet risk-tiered.

🪴 Install

npm install agentcc
# plus whichever framework you already use:
#   openai · ai · @langchain/core · @openai/agents

Requires Node 18+. Every framework is an optional peer dependency — install only the one you use and import from the matching subpath:

Framework	Import	Wrapper
OpenAI	agentcc	withCostControl(client, opts)
Vercel AI SDK / Mastra	agentcc/ai	withCostControl(model, opts)
LangChain.js / LangGraph.js	agentcc/langchain	wrapModel(model, opts) / CostControlHandler
OpenAI Agents SDK	agentcc/agents	wrapAgentsModel(model, opts)

All adapters feed the same privacy-safe pipeline, so the guarantees above hold everywhere.

🌍 Frameworks

The same agentId / accKey / kill-switch options apply to every adapter below.

OpenAI

import { withCostControl } from "agentcc";
import OpenAI from "openai";

const client = withCostControl(new OpenAI({ apiKey: process.env.OPENAI_API_KEY }), {
  agentId: "support-bot",
  accKey: "acc_abc123",
});

// Everything else is unchanged — same methods, same types, same return values.
const res = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Hello" }],
});

Streaming works too — the SDK auto-requests usage stats and reads them off the final chunk:

const stream = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Hello" }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

Vercel AI SDK

Wrap any model and hand it to generateText / streamText:

import { withCostControl } from "agentcc/ai";
import { openai } from "@ai-sdk/openai";
import { generateText } from "ai";

const model = withCostControl(openai("gpt-4o"), { agentId: "support-bot", accKey: "acc_…" });
await generateText({ model, prompt: "Hello" });

Mastra

Mastra is built on the Vercel AI SDK, so the same wrapper works — pass the wrapped model to your agent:

import { Agent } from "@mastra/core/agent";
import { withCostControl } from "agentcc/ai";
import { openai } from "@ai-sdk/openai";

const agent = new Agent({
  name: "support-bot",
  instructions: "Help customers.",
  model: withCostControl(openai("gpt-4o"), { agentId: "support-bot", accKey: "acc_…" }),
});

LangChain.js / LangGraph.js

wrapModel enforces the kill switch and records telemetry. (For telemetry only, add new CostControlHandler(opts) to any run's callbacks.)

import { wrapModel } from "agentcc/langchain";
import { ChatOpenAI } from "@langchain/openai";

const model = wrapModel(new ChatOpenAI({ model: "gpt-4o" }), { agentId: "support-bot", accKey: "acc_…" });
await model.invoke("Hello"); // use anywhere, including LangGraph nodes

OpenAI Agents SDK

import { wrapAgentsModel } from "agentcc/agents";
import { Agent, run, OpenAIResponsesModel } from "@openai/agents";
import OpenAI from "openai";

const model = wrapAgentsModel(new OpenAIResponsesModel(new OpenAI(), "gpt-4o"), {
  agentId: "support-bot",
  accKey: "acc_…",
});
const agent = new Agent({ name: "Support", model });
await run(agent, "Hello");

⚙️ Options

Option	Required	Default	Description
agentId	yes	—	Identifies the agent for this client.
accKey	yes	—	Bearer token for the telemetry endpoint.
endpoint	no	https://agentcc.ca/v1/events	Telemetry ingest URL.
flushInterval	no	5000	Max ms before a buffered batch is sent.
batchSize	no	50	Send early once this many events queue.
killCheck	no	true	Check kill status before each call; throw AgentKilledError if killed. Inert until a budget + auto-stop is set in the dashboard.
onKilled	no	throw	Run instead of throwing when a killed agent's call is blocked; its return value becomes the response (graceful containment).
onError	no	swallow	Called on telemetry/pricing failures.
router	no	off	Model routing: "auto" (downshift cheap, tool-free calls) or a RoutePolicy. OpenAI adapter only.
cache	no	off	Exact-match response cache ({ provider: "memory" | "upstash" | "redis" | "managed", … }). OpenAI adapter only.

💸 Cost reduction (routing + cache)

Two opt-in, zero-latency ways to spend less — both on the OpenAI adapter (withCostControl(client, …)). They run before the call, fail open, and keep the response shape identical.

Model routing — send cheap, simple calls to a cheaper model:

import { withCostControl } from "agentcc";

// "auto": tool-free calls under ~2k tokens downshift (e.g. gpt-4 → gpt-4o-mini).
const client = withCostControl(new OpenAI(), { agentId: "bot", accKey: "acc_…", router: "auto" });

// …or an explicit policy (priority-sorted; first match wins):
const client2 = withCostControl(new OpenAI(), {
  agentId: "bot",
  accKey: "acc_…",
  router: { routes: [{ name: "tiny", condition: { type: "token_estimate", max: 500 }, targetModel: "gpt-4o-mini" }] },
});

The model actually used is what's recorded, so cost telemetry reflects routing. A routed call that errors automatically retries on the original model (routing.fallback: true). The dashboard can also push a routing policy down via the status endpoint — that's how auto-remediation's "downshift" action takes effect with no code change.

Response cache — replay an identical request instead of paying for it again:

const client = withCostControl(new OpenAI(), {
  agentId: "bot",
  accKey: "acc_…",
  cache: { provider: "memory" }, // or "upstash" / "redis" with { url, token? } — BYODB
});

Keyed on the content-free prompt fingerprint + model (one keyed read, no embedding call). A hit skips the LLM entirely and is recorded with cache.hit: true and $0 cost. The cache stores the raw provider response, which lives only in your store (memory / your Redis / Upstash) — it is never sent to the telemetry endpoint.

Prefer not to manage a store? Set provider: "managed" (or just flip on the cache from the dashboard) — the SDK proxies through the hosted ACC cache using your accKey; our storage credentials never reach your process and a per-account storage cap is enforced server-side:

const client = withCostControl(new OpenAI(), { agentId: "bot", accKey: "acc_…", cache: { provider: "managed" } });

Dashboard-driven config. The cache backend (off / built-in managed / bring-your-own-DB) can be set per agent in the dashboard and is pushed to the SDK via the status endpoint — so you can turn caching on, or swap your BYODB credentials, without a code change. A pushed config takes precedence over the local cache option.

📈 Cost per successful completion

Token savings only count if reliability holds — a cheap-but-wrong answer that needs rework isn't cheap. reportOutcome lets you mark how a finished workflow turned out, so the dashboard can divide spend by successful completions and show failure / rework rates.

import { reportOutcome } from "agentcc";

const res = await client.chat.completions.create({ model: "gpt-4o", messages });

// Judge success however you already do (schema validated? user accepted? eval passed?):
await reportOutcome("success", { agentId: "support-bot", accKey: "acc_…" });
// …or "failure" / "rework" — plus an optional `workflow` label for by-workflow rollups.

Mechanics. It's a standalone, stateless function, not something hung off the wrapped client — the wrapper must keep your framework's exact return type (so there's no per-call handle), and the outcome is judged after the response anyway. So it works identically across every adapter. It POSTs to a derived …/outcomes endpoint with your accKey and fails open (a transport error never throws into your code). Privacy holds: only the enum and an optional workflow label leave the process — never prompt or response content.

🦷 The chomp (budgets & kill switch)

Set a budget or hard call cap on an agent in the dashboard and turn on auto-stop; once spend or call count crosses the limit the backend marks the agent killed. With killCheck on (the default), the SDK checks the agent's status before each call and throws AgentKilledError instead of hitting the LLM — stopping a runaway loop before it spends more. Status is cached briefly and fails open, so a status-endpoint outage never blocks your calls.

Catching the bite

import { withCostControl, AgentKilledError } from "agentcc";

const client = withCostControl(new OpenAI(), { agentId: "support-bot", accKey: "acc_…", killCheck: true });

try {
  await client.chat.completions.create({ model: "gpt-4o", messages });
} catch (err) {
  if (err instanceof AgentKilledError) {
    // agent was killed from the dashboard — stop looping
  }
}

Prefer graceful degradation over a thrown error? Pass onKilled and its return value becomes the call's response, so one killed sub-agent doesn't crash the whole run.

🔬 Diagnosing token waste

Every event carries a prompt fingerprint — enough to deduce why an agent burns tokens, without shipping a single character of prompt content:

Prompt fingerprint shape

"prompt": {
  "message_count": 14,
  "total_chars": 21840,
  "roles": {
    "system": { "count": 1, "chars": 1800 },
    "user":   { "count": 7, "chars": 9200 },
    "assistant": { "count": 6, "chars": 10840 }
  },
  "hash": "9f2c…"   // SHA-256 of the message array (one-way)
}

• Prompt bloat — message_count / total_chars climbing across calls = history re-sent and growing.
• Loops — the same hash recurring = the agent re-issuing a near-identical prompt.
• Fat system prompt — a large roles.system.chars relative to the rest.

The hash is one-way: identical prompts collide so you can correlate them, but the original text is never recoverable or transmitted.

🧬 How it works

Each adapter hooks its framework at the model boundary (withCostControl() proxies chat.completions.create; the AI SDK uses LanguageModelMiddleware; LangChain uses a callback handler + model wrap; the Agents SDK wraps the Model). All of them normalize the finished call into one CallRecord and feed a single shared core: it reads the usage object after the response returns, computes cost from a static price table, fingerprints the prompt, and pushes the record onto an in-memory queue that flushes in batches via fetch. Telemetry is fire-and-forget: it never adds latency and never throws into your request path.

your code ──▶ withCostControl(client) ──▶ looks identical ──▶ create() as normal
                       │
                       └─▶ CallRecord ──▶ core ──▶ queue ──▶ fetch ──▶ dashboard

For a deeper walkthrough — file-by-file architecture and a step-by-step trace of a call — see docs/.

🌳 Development

npm install
npm test          # vitest suite
npm run typecheck  # tsc --noEmit
npm run build      # tsup → dist/{index,ai,langchain,agents}.{mjs,cjs,d.ts}

📜 License

MIT — feed it whatever you like.