Repository navigation
Campfire as a single self-contained binary — no Redis, no worker process #263
Replies: 4 comments
|
An update, one week on. The short version: the native target went from Since the post above:
Still zero changes to Campfire's source, still no annotations, and The performance claims at the top of this thread remain intentions, |
|
Benchmarks: https://rubys.github.io/roundhouse/bench/campfire/ |
|
The binary now clearly outperforms Rails: https://intertwingly.net/blog/2026/09/09/Campfire-Buffered.html Try it yourself: Status: https://intertwingly.net/blog/2026/09/11/Campfire-Dockerized.html |
|
Where the suite stands, and the first — and so far only — pull request, opened under the rule from the top of this thread. Your test suite now runs against the native binary, one compiled program per test file, and it is at 299 of 300 on that lane (54 of 55 files green, none failing to link): https://rubys.github.io/roundhouse/bench/campfire-suite-spinel/ — and the interpreted lane (https://rubys.github.io/roundhouse/bench/campfire-suite/) reads the same. The denominator grew with #267/#268/#276; the direct-upload guard, the sign-out disconnect and the link-preview hardening all pass as written. The one remaining test is |
Uh oh!
There was an error while loading. Please reload this page.
This is a heads-up rather than a request — nothing here needs a response. If it belongs in
a different category, or you'd rather it weren't here at all, say the word and I'll move or
delete it.
The picture
Campfire's
Procfileis three processes today:I'm working toward compiling Campfire — unmodified — into one native binary. No Redis:
Action Cable fan-out, the cache, and the job queue all move in-process. No separate worker
process. No Ruby runtime to install, no gems to bundle. For something people self-host, that
seems like it might be worth having: the install stops being an orchestration problem and
becomes a file you copy.
The target is to beat the current deployment on every measure I can think to take: cold
start, p50 and p99 latency, throughput, resident memory, and — the one that matters most for
a chat app — how many concurrent WebSocket connections a given box will hold.
Two structural differences are behind that last one. The emitted server is
fiber-per-connection rather than a thread pool, so WebSocket concurrency is bounded by
roughly 64 KB of stack per connection rather than by the size of a worker pool. And Spinel
has no GVL — it runs a true M:N threading model, N OS workers scheduling green threads
across however many cores the box has, the JRuby/TruffleRuby model rather than CRuby's.
That second one is what makes "one process" a serious proposition rather than merely a
smaller one. A good part of why a Rails deployment runs a pool of worker processes at all is
that the GVL stops threads from executing Ruby in parallel — so you buy parallelism by
forking, and every one of those processes carries its own copy of the application. Take the
GVL away and a single process can use the whole machine, which is the difference between
consolidating three processes into one and merely renaming them.
None of that is proven on Campfire yet — and I'd rather say so here, next to the claim,
than leave it to a footnote at the bottom. What follows is where it actually stands.
How
Two pieces. Roundhouse is a transpiler I maintain: it
reads a Rails application and emits a standalone project in another language, so the
deployment target becomes a build flag rather than a rewrite.
The target I care about for Campfire is Spinel — Matz's
ahead-of-time Ruby compiler. It's fairly new and not widely known yet, so briefly: it reads
Ruby source, performs whole-program type inference, emits C, and hands that to the system
compiler. The output is a standalone native executable with no Ruby runtime and no gems.
Same language, no VM.
What's published so far is that pipeline on a smaller app, where the emitted framework runs
on both CRuby and Spinel and the runtime is the only difference between the two cells:
https://rubys.github.io/roundhouse/bench/
Roughly 2.8× the throughput on about a tenth of the memory, from the same Ruby source.
Whether that holds at Campfire's size is exactly the open question, and it's why I'm working
through Campfire rather than something I invented.
Where it stands
Correctness first, since a fast binary that renders the wrong page is worthless. Campfire
turns out to be an unusually good thing to check against — a complete, idiomatic Rails 8 app
where Hotwire, Action Cable, Active Storage and Action Text are all load-bearing, and, the
part that matters most, it ships its own test suite. So I don't have to invent a
definition of "correct."
The current run transpiles Campfire and executes Campfire's own test files against the
emitted tree, targeting CRuby — the waypoint on the way to the native target:
219 of 240 tests passing — 40 of 52 files green (as of today; the page regenerates on
every push, so the number moves).
https://rubys.github.io/roundhouse/bench/campfire-suite/
Of the 21 unpassed: 11 are my compiler's own modeling debt (Active Storage attachment
scopes,
ActionText::Attachment.from_node, fixture accessors, theallow_browsergate,Global ID minting, a view helper in a module), 2 are gems I shim rather than compile, and 8
are untriaged.
Zero are attributed to Campfire. That's the part I most want to state plainly, because
the obvious worry about a project like mine is that it will eventually turn up asking you to
contort the app to suit a transpiler. The rule I work under is that the port never asks for
app-side divergence: if I ever open anything here, it will be a plain-Rails change that
stands on its own merits under stock Rails, or it doesn't get opened. So far there have been
none, and I expect that to remain the normal case.
The Spinel target doesn't build for Campfire yet. I'm walking the compile errors one at a
time, and so far most have been my own modeling gaps rather than the compiler's.
On the benchmark
There's also a performance comparison, and it seems better to mention it than to have you
find it. Real Rails against the emitted tree, one seeded SQLite database, real HTTP, each
lane signed in through its own form:
https://rubys.github.io/roundhouse/bench/campfire/
The honest summary, and it does not currently favour me: with Campfire's fragment cache on
— the lane that says what a deployment actually costs — Rails is 1.7–3.2× faster than what I
emit. Comparing renderers directly with the cache off, the emitted tree is about 5× faster,
which mostly says I don't have a fragment cache yet rather than anything flattering about my
renderer. Memory is the one place I'm ahead today: 230 MB against 453 MB.
That comparison is CRuby-to-CRuby. The native target is where the claims at the top are
supposed to be settled, and until it builds they're intentions rather than results.
cached: trueon the message list is doing a lot of work for you, and it took me a while tonotice I was benchmarking Redis rather than Rails.
Things that would be genuinely useful — only if you have a moment
Is publishing a pass rate under Campfire's name something you're comfortable with? I've
tried to make every page say that failures are mine and not Campfire's, but you should get
to judge whether the framing reads that way.
Is there anywhere in Campfire you'd consider incidental rather than load-bearing? Right
now I treat every assertion in the suite as equally binding, which is probably stricter
than you intended in places — and knowing where you'd shrug would help me spend effort
where it matters.
If the port ever surfaces something in Campfire itself, is a discussion here the right
place? The likeliest shape is a silent divergence — my tree and CRuby disagreeing
somewhere the tests don't happen to look — which is usually my bug but occasionally
isn't.
Caveats, so none of this is oversold
The suite run splices five stubs covering known gaps in my compiler; each is documented and
gets deleted as the gap closes. Three of the eight untriaged failures are silent divergences
rather than clean errors, which are the ones I trust least. The in-process job queue
described at the top is a design, not an implementation. And Roundhouse is nowhere near
production-ready — nobody should deploy anything it emits today. The single-binary Campfire
is a goal I think is reachable, not a thing I have.
On authorship
One disclosure worth making up front rather than leaving to be discovered. Roundhouse,
Spinel, and this post are all written with Claude Code. Every recent commit in Roundhouse
carries a
Co-Authored-By: Claudetrailer, and nearly all of Spinel's do too.I mention it for two reasons. The volume of work involved otherwise invites a fair question
about how it got done. And if engaging with a project built that way isn't something you
want to do, you should get to know that now rather than several exchanges in.
Thanks for building it, and for keeping it plain Rails. That's exactly what makes it usable
this way.
All reactions