Rewrite BitRouter vs LiteLLM around claims that hold up - #63
Merged
Conversation
LiteLLM now ships a Rust core ("Rust core with Python SDK") and publishes
benchmarks at ~0.05ms added overhead. Our page claimed they were a Python
gateway at ~20ms+ against our ~5ms p50 — a comparison we lose on their own
numbers, on a page whose whole job is credibility.
Reframed the argument onto ground that survives their December roadmap:
- Lead with all-in-one vs. focused-and-extensible around the act → observe
→ evaluate → learn loop. It's the one difference they can't ship past.
- Promote self-serve cloud vs. quote-only Enterprise (verified against
litellm.ai/pricing).
- Demote Rust to "Rust-accelerated vs. Rust-native": their recommended mode
is a PyO3 bridge in the Python wheel, the standalone server is beta with
fewer routes, and their own endgame keeps Python in a sidecar for plugins
and SSO. Argue dependency surface, not language speed.
- Concede properly: LiteLLM's in-process library and its embeddings/rerank/
audio coverage are real advantages we don't match. Both now appear in the
table and the migration guide, not just the prose.
The security paragraph describes the 2026 RCE by incident class rather than
CVE number, and explicitly disclaims immunity — our MCP gateway executes
tools too, and Rust prevents none of that bug class.
Also:
- Trim the compare table 11 rows → 5; drop the routing-overhead row.
- Correct platform fee to 0% markup on the OpenRouter and Portkey tables.
- Rewrite the LiteLLM terminal animations, which asserted ~85ms p99 / GIL
and "LiteLLM: none of the above" (false — they ship MCP). No numbers.
- Wire two terminals into the page and whitelist CompareTerminal, which was
registered in mdx-components but missing from the docs lint whitelist.
- migrate-from-litellm claimed LiteLLM was "self-hosted only". It isn't.
Numbers left as TK- placeholders pending real benchmarks.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…page The page can't ship with TK- markers rendering literally, so the three numbers are now qualitative: - LiteLLM's overhead is "sub-millisecond" — their published 0.05ms, stated as their result. Conceding it plainly is stronger than hedging, since the paragraph's point is that the distinction is where the Rust stops, not speed. - BitRouter "cold-starting in well under a second and holding tens of megabytes resident" — conservative bands, still worth replacing with measurements before this is load-bearing anywhere. Also removes both <CompareTerminal> blocks from the page. The rewritten terminal programs stay in compare-programs.tsx (correct now, just unused again), as does the CompareTerminal whitelist entry — the component is registered in mdx-components.tsx, so the whitelist was out of sync regardless of whether a page uses it today. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PR #62 landed the OpenRouter rewrite and its own platform-fee fix, hitting the same registry file. Resolution: - LiteLLM rows: ours (the 5-row trim is what this PR is for) - OpenRouter rows: main's — #62 rewrote that comparison deliberately, and it is not this PR's business - Portkey fee row: main's "none · 0% markup" phrasing, now the house style, superseding the equivalent change made here Net effect: this branch's diff against main is now only the LiteLLM entry. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
LiteLLM's GitHub tagline is now "The fastest, litest AI Gateway. Rust core with Python SDK" and they publish benchmarks at ~0.05ms added overhead / 31.7MB memory. Our comparison page claimed they were a Python gateway at
~20ms+versus our~5ms p50.That's a fight we lose by two orders of magnitude, using their published numbers, on the one page whose entire job is credibility. This reframes the argument onto ground that survives their roadmap.
The new argument, in order
Concessions added
The page previously read as a strawman. It now concedes, in the prose and the table and the migration guide:
from litellm import completion) — we have no answer, and the page says soOn the security paragraph
Describes the 2026 unauthenticated RCE by incident class — command injection in MCP endpoints chained with a host-header bypass upstream — without CVE numbers or the CVSS score, notes both were disclosed and patched responsibly, and explicitly disclaims immunity: our MCP gateway executes tools too, and Rust prevents none of that bug class. The claim is smaller blast radius, not safety.
Deliberate call. Citing a competitor's CVSS 10.0 reads as FUD and ages badly the first time we ship our own advisory. Easy to harden if you disagree.
Also fixed
2% stablecoin / 5% card→0% markupon the OpenRouter and Portkey tables~85ms p99 · Python GILandLiteLLM: none of the above(false — they ship MCP). Rewritten with no numbers; marks changed red ✗ → amber ⚠CompareTerminalmdx-components.tsxbut missing fromCOMPONENT_WHITELIST, so any page using it failedlint:docs. AddedThe terminal programs are corrected but not rendered on the page — no
<CompareTerminal>blocks, same as before this PR. The whitelist entry stays because the registry/whitelist desync is real either way.Numbers
No hard performance numbers are asserted for BitRouter. The Rust section reads "cold-starting in well under a second and holding tens of megabytes resident" — conservative qualitative bands, chosen because the section argues dependency surface rather than speed. Worth replacing with real measurements if this page ever needs to carry a benchmark claim.
Known follow-ups (pre-existing, not touched here)
127.0.0.1:4356while the rest of the site uses:8787— and it's in a copy-paste path.0% markupfix may collide with a parallel branch that already rendersnone · 0% markup.Verification
pnpm buildclean ·pnpm test83/83 ·pnpm lint:docs49 docs pass · page rendered and checked locally🤖 Generated with Claude Code