Proposal: a llama-rpc-worker worker hive
Summary
A worker hive whose bees raise llama.cpp RPC workers as cells. Uses stock upstream llama.cpp tools/rpc/rpc-server.cpp for layer-sliced multi-machine inference, with a coordinator llama-server as the compute front.
Motivation
ds4.c gives us deep layer-sliced distributed inference, but only for the DwarfStar families (V4 Flash, GLM-5.2). For models ds4 does not cover (e.g. GLM-4 / GLM-4.7 Flash), upstream llama.cpp RPC is the vanilla path to multi-machine inference. Nesting it as a worker hive adds a generic, stock-llama.cpp data-plane cell type.
Hive stencil
- Typology: worker (accepts
chi:"prompt", emits chunk then finish).
- Bee binary:
llama-rpc-worker, launches ggml-rpc-server for a layer slice as its cell; a coordinator cell runs llama-server with RPC support.
- Propensity:
StatelessPerCall.
- Richness: medium (system, content, tools).
- Wire: as a worker it speaks thrum; the RPC transport between coordinator and workers is private.
Egg spec (candidate)
- model path/quant, layer slice A:B, ctx, host/port for the RPC listener
- coordinator address, hidden-state width
Cell shape
One rpc-server process. feed is the prompt JSON (the coordinator translates to RPC), mmm is the chunk stream, emerged is the worker exit, silence cancels. Standard WorkerBee::raise(egg) -> Cell, tended by lifecycle.rs.
Integration with humd / Nest
Workers handshake a humd and announce bee:["worker"]. humd tracks layer coverage across rpc-workers so the coordinator knows the route is complete (mirrors ds4's coordinator route check).
Risks
- Upstream
SECURITY.md explicitly flags the RPC backend as unsafe on untrusted networks. Only suitable on a private cluster.
- Hidden-state streaming between coordinator and workers consumes network bandwidth.
- Coordinator coordination complexity.
Open questions
- Is the coordinator a separate hive or the same hive in a different role (like ds4's
--role worker vs --role coordinator)?
- How does humd track and report layer coverage across rpc-workers?
- How much does this overlap
ds4-worker? Keep both, or make llama-rpc-worker the generic fallback and ds4-worker the specialized fast path?
Proposal: a
llama-rpc-workerworker hiveSummary
A worker hive whose bees raise llama.cpp RPC workers as cells. Uses stock upstream llama.cpp
tools/rpc/rpc-server.cppfor layer-sliced multi-machine inference, with a coordinator llama-server as the compute front.Motivation
ds4.c gives us deep layer-sliced distributed inference, but only for the DwarfStar families (V4 Flash, GLM-5.2). For models ds4 does not cover (e.g. GLM-4 / GLM-4.7 Flash), upstream llama.cpp RPC is the vanilla path to multi-machine inference. Nesting it as a worker hive adds a generic, stock-llama.cpp data-plane cell type.
Hive stencil
chi:"prompt", emitschunkthenfinish).llama-rpc-worker, launchesggml-rpc-serverfor a layer slice as its cell; a coordinator cell runsllama-serverwith RPC support.StatelessPerCall.Egg spec (candidate)
Cell shape
One rpc-server process.
feedis the prompt JSON (the coordinator translates to RPC),mmmis the chunk stream,emergedis the worker exit,silencecancels. StandardWorkerBee::raise(egg) -> Cell, tended bylifecycle.rs.Integration with humd / Nest
Workers handshake a humd and announce
bee:["worker"]. humd tracks layer coverage across rpc-workers so the coordinator knows the route is complete (mirrors ds4's coordinator route check).Risks
SECURITY.mdexplicitly flags the RPC backend as unsafe on untrusted networks. Only suitable on a private cluster.Open questions
--role workervs--role coordinator)?ds4-worker? Keep both, or makellama-rpc-workerthe generic fallback andds4-workerthe specialized fast path?