bundle

Frozen research artifact from our study of orchestration models — published as-is, not maintained.

A transparent mixture-of-agents orchestrator. bundle answers a prompt by fanning it out to a swappable pool of models, having a judge compare their answers, and a synthesizer write the final reply — and reports the honest, itemized cost of every answer. You consume it as a single model: it exposes an OpenAI-compatible endpoint, so anything that speaks OpenAI can point at http://localhost:8787/v1 with model name bundle.

We built it to answer one question independently: do mixtures of agents beat one good model? The answer, measured on hard benchmarks with real billed costs, is the reason this repo exists:

	HLE (hard reasoning)	LiveCodeBench (hard coding)
bundle	50.0% · $0.04/q	82% · $0.58/problem
Claude Opus 4.8 (single)	62.5% · $0.006/q	74% · $0.27/problem
OpenRouter Fusion	51.3% · $0.61/q	78% · $1.58/problem

Orchestration loses on reasoning and wins on coding — the same machine, opposite outcomes. The full study (methodology, five findings, all caveats): An Independent Study of Orchestration Models.

The commercial orchestrators are sealed boxes. bundle is the same machine with the lid off: every stage swappable by config, every call metered.

Quickstart

Requires Node ≥ 24 (runs TypeScript natively). No npm install — the runtime is dependency-free.

Path A — metered API (recommended, ~2 minutes):

git clone https://github.com/nbitslabs/bundle-oss && cd bundle-oss
export OPENROUTER_API_KEY=sk-or-...      # https://openrouter.ai/keys
BUNDLE_CONFIG=./bundle.config.metered.json node src/cli.ts "why is the sky blue?"

Path B — subscription CLIs ($0 dev loop): if claude and/or codex are installed and signed in, the default config uses them directly:

node src/cli.ts "why is the sky blue?"

Costs on this path are shadow-estimated, never billed — and never valid for cost claims. Either way you get the answer, the routing path, and an itemized per-stage cost table. If nothing is configured yet, the CLI prints setup guidance instead of a stack trace.

Use it as a model (the point)

node src/gateway.ts        # http://localhost:8787/v1   (model: "bundle")

curl http://localhost:8787/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"model":"bundle","messages":[{"role":"user","content":"..."}]}'

The response is a normal OpenAI chat completion plus a bundle field carrying the cost breakdown and routing path:

{
  "choices": [{ "message": { "role": "assistant", "content": "..." } }],
  "usage": { "prompt_tokens": 86, "completion_tokens": 26, "total_tokens": 112 },
  "bundle": {
    "path": "panel",          // or "tier0"
    "mode": "judge",          // or "aggregator"
    "cost_usd": 0.0164,
    "cost_estimated": false,  // true => shadow cost (sub/CLI), not billed
    "num_calls": 5,
    "by_stage": [ /* per-model tokens + cost + latency */ ]
  }
}

stream: true is supported at the wire level (the full answer is chunked into SSE deltas; true token streaming is on the roadmap). To plug into a harness (opencode, Cursor, any OpenAI client): custom provider, baseURL http://localhost:8787/v1, any API key string, model bundle.

Configure the pool

Everything is JSON config, not code. Copy bundle.config.example.json, edit, point BUNDLE_CONFIG at it:

{
  "pool": [
    { "provider": "openrouter", "model": "google/gemini-3.5-flash" },
    { "provider": "openrouter", "model": "deepseek/deepseek-v4-flash" },
    { "provider": "openrouter", "model": "anthropic/claude-opus-4.8" }
  ],
  "judge":       { "provider": "openrouter", "model": "google/gemini-3.5-flash" },
  "synthesizer": { "provider": "openrouter", "model": "anthropic/claude-opus-4.8" },
  "tier0": { "enabled": false, "provider": "openrouter", "confidenceThreshold": 0.85 },
  "mode": "judge"            // "aggregator" disables the judge (ablation)
}

The config above is bundle.config.strongsynth.json — the exact pool that scored 82% on LiveCodeBench in the study. Provider names come from src/providers/registry.ts; adding a provider type is one entry there.

Reproduce the study

The study ran on the code at tag v0-baseline — check that out for byte-exact reproduction:

git checkout v0-baseline

• scripts/rerun_hle.sh — the HLE rig (4 arms, checkpointed, resumable).
• scripts/run_all_bench.sh, scripts/lcb/ — the LiveCodeBench rig: generation via gen_arm.py (checkpointed per problem), local grading via prep_grade.py + grade_host.py.
• src/bench/ — the harness: arms, letter grading, per-item checkpoints, CIs.

What you must bring yourself (not redistributable here): the HLE questions (HuggingFace, cais/hle, gated) and LiveCodeBench (release_v6 via the official repo, which also provides the grading harness the scripts call). The scripts' headers document the expected local paths. Fair warning from our own bill: the bundle arms alone cost ≈ $31 to run in June 2026; reproducing every arm including Fusion cost us ≈ $180.

Honest limitations

• Reasoning is not this architecture's game. Our own study: the single frontier model beat every mixture on hard reasoning while being cheaper and 27x faster. bundle earns its keep on code and other checkable work.
• Latency: the panel path is several sequential model calls — minutes, not seconds, on hard prompts. bundle is a deliberation machine, not autocomplete.
• Tier-0 is a known anti-pattern. The self-rated-confidence gate ships for ablation completeness; verbalized confidence is miscalibrated (our default configs keep it off).
• Text-only. No tool calls, no images, no structured-output passthrough.
• Sub/CLI costs are estimates. Real cost numbers require the metered path.
• src/bench/data/mmlu_pro_stem.jsonl is a small harness-validation sample from MMLU-Pro (MIT, TIGER-Lab).

What's next

The study's findings are a design brief, and we've been executing it: routing by task shape, an execution-gated verification loop for code, and compressed inter-stage prompts. The second write-up — how bundle's coding cost went from $0.58 to $0.04 per problem, and exactly what that trade bought and cost — is coming soon.

License

MIT © 2026 nBits Labs