Skip to content

rishbjain1/fusion

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Fusion

A Mixture-of-Agents (MoA) gateway that fans one prompt across a frontier-model panel and synthesizes a single answer — matching the best solo model's quality at ~28% lower cost per task, with the failure case measured and published too.

What it is

Fusion sends your prompt in parallel to a panel of models, then uses an aggregator model to critique and merge their answers into one response. It reports per-model cost against the most-expensive-baseline. OpenAI-compatible — point any OpenAI client at the deployed Cloudflare Worker as a drop-in base-URL swap.

The honest headline (see Eval): on a frontier panel, the fusion tied the best solo model at the quality ceiling while costing 28.5% less ($0.1385 vs $0.1938/task). On a cheaper-model panel it underperformed the solo premium by 2 points — that result is shipped too, because an eval you only publish when it wins isn't an eval.

How it works

  1. Fan-out: Your prompt is sent in parallel to all panel models (Claude Opus 4.8, GPT-5.5, Gemini 3.1 Pro).
  2. Candidates: Each model returns its answer independently.
  3. Aggregator: Claude Opus 4.8 silently critiques all answers, merges the best elements, and returns one unified response.
  4. Final: You get the synthesized answer plus cost breakdown.

Install & Setup

Requires Python 3.11+. No pip dependencies—uses only urllib and concurrent.futures from stdlib.

  1. Get an OpenRouter API key from openrouter.ai.
  2. Set the key in one of these locations:
    • Environment variable: export OPENROUTER_API_KEY=sk-or-...
    • File: ~/.config/fusion/.env (reads OPENROUTER_API_KEY=)
  3. Run: python fuse.py "Your prompt here"

Usage

CLI

python fuse.py "Explain quantum entanglement briefly"
python fuse.py "What are the top 3 risks in startups?" --quiet
python fuse.py "Analyze this code" --json
python fuse.py "Your prompt" --temp 0.7

Flags:

  • --quiet: Suppress per-model output, show only fused answer
  • --json: Return output as JSON
  • --temp <float>: Set temperature (default: 1.0)

Slash command in Claude Code

/fuse <prompt>

Eval

Run the evaluation harness on 6 hard tasks with gold-standard answers:

python eval.py --tasks 6
python eval.py --tasks 6 --json

Frontier panel (6 tasks, Opus 4.8 / GPT-5.5 / Gemini 3.1 Pro; Haiku-4.5 judge):

  • All solo models + Fused: 10.0/10 — everyone hits the quality ceiling, so fusion matches, it doesn't beat.
  • Cost: $0.1385 (fusion) vs $0.1938 (all-Opus baseline) = 28.5% cheaper. Same answer quality, less spend.

Cheap panel (8 tasks, Haiku / GPT-5.4-mini / Gemini-Flash) — the honest failure:

  • Best cheap solo: 10.0 · cheap fusion: 8.0 (−2.0) · and fusion cost more ($0.097 vs $0.056 opus-solo).
  • Takeaway: fusion buys cost efficiency at the frontier, not a quality floor for weak panels. Shipped so the limits are visible.

Config

Edit fuse.py:

PANEL = [
    "anthropic/claude-opus-4.8",
    "openai/gpt-5.5",
    "google/gemini-3.1-pro-preview",
]
AGGREGATOR = "anthropic/claude-opus-4.8"

To replicate the "cheap panel trades quality with solo premium" story, swap the panel to cheaper models (e.g., Claude Haiku, GPT-4 Mini, Gemini Flash).

Files

  • fuse.py: Engine + CLI. Handles fan-out, aggregation, cost reporting.
  • eval.py: Evaluation harness. Runs on benchmark tasks with a judge model.
  • results.json: Latest eval output (scores, costs, per-task breakdown).
  • dashboard.html: Web UI. Renders charts and cost panel from results.json.
  • README.md: This file.

About

Mixture-of-agents LLM gateway — OpenAI-compatible Cloudflare Worker that fans one prompt across a model panel and synthesizes; includes the honest benchmark where MoA lost to a solo model

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages