Skip to content

v0.3.271-alpha

Choose a tag to compare

@enapt enapt released this 04 Nov 11:10
· 29 commits to main since this release

A computer that only takes stops being served, the peer list shows what each computer does for
yours, and long chats fit on a nearly full graphics card.

The peer list shows what each computer has done for yours, and yours for it. Two new columns —
"Did for you" and "You did for it" — count the requests each side ran for the other, counted by your
own computer from what it saw itself, so nobody else's word enters them and nobody can be blamed by
someone else. A computer your computer has run 10 or more requests for, with none back, is tagged
"Takes only" (it may still be helping other computers; only what passed between the two of you is
counted). swarmllm peers shows the same two counts. They are kept across restarts and change
nothing: no computer is served or refused by them, and the old self-counted credit balance stays
hidden, because every computer counted its own and a computer that only takes could show as high a
balance as anyone.

Other computers run your chats only if your computer shares, and keeps up — and a computer that
stops updating loses the swarm.
A computer must hold at least 2 model parts and run a release at
most one behind the newest for the swarm to run its requests. One that shares too little still sees
the swarm, still fetches parts (which is how it starts to share), still runs the models it holds
whole, and its own linked devices still serve it. One that is two or more releases behind is
disconnected by every updated computer and let back ten minutes later only if it has updated —
updates come from GitHub, so it can still update, and its dashboard shows "Out of date" with the fix
even when no computer is left connected. One release behind keeps everything: that is where every
computer is for the minutes an update takes, so a new release never splits the swarm. Releases are
counted as published, so a version number that was never released does not count, and a computer
that has not yet read the list of releases (just started, or updates turned off) disconnects nobody.
A release that is withdrawn stops being the newest at the next update check. The dashboard and
swarmllm status say why chats stay on that computer and what lifts it (turn on Auto-Download
Models, or update). The peer list marks computers that share too little, are out of date, or have
automatic downloads or updates turned off. Nothing changes for a computer that shares and updates,
and linked devices of your own are never disconnected. "Newest" is only a release whose signature
checks out, and the first release of a new series counts as one release, so a new version number
never cuts anyone off at once; an old announcement played back by another computer cannot get yours
refused or disconnected. A modified program can still claim parts or a version it does not have —
this stops the released program from taking without giving.

The computer new installs join through is never disconnected for its version, and it now updates
within about five minutes of a release.
A computer is never cut off by one that bootstraps from it
(the built-in anchor, or any address in your own bootstrap list), so an anchor that falls behind can
never leave new installs with nobody to join. The anchor's own updater checks every 4 minutes
instead of every hour; an anchor already running picks the new timer up with the one-time refresh in
deploy/anchor/README.md.

A chat this computer has no memory left for goes to the other computers instead of failing.
A model held whole here used to answer "not enough free memory for this conversation" even when
other connected computers held the same model — a long prompt to a 14B on a 12 GB card did, in
0.2 s. It is now planned again with them before any of the reply is written, and the reply's route
names both the attempt here and the computers that answered. With nobody else to ask, the message
is unchanged.

A model that would fill your graphics card is placed partly on the processor before it loads, so
long chats fit.
On a 12 GB card a 14B loaded whole and kept about 250 MB — some 1,300 tokens — for
conversations, so every longer prompt was refused, even on a computer with nobody else to ask. Before
the load the computer now reads its card as the load will, and where the model would leave less than
a 4,096-token conversation it moves the fewest layers to the processor that make that room (the
order llama.cpp uses: the conversation's room first, then layers). Replies are a little slower and
long prompts are answered. A model that fits still loads whole, and one moved this way goes back to
the card whole only once other programs have given back the memory it was short of.

The model browser recommends only a file your computer can run. Searching for a model could
recommend — and download — a file no computer on the swarm can run: an "IQ" format this version
cannot read yet, one piece of a model split across several files, or a file kept in a folder of the
model's page. The recommendation now skips those and picks the usual 4-bit file where there is one.
In the size list each such file is greyed out with the reason, and a model with no usable file says
"None of these files can run yet" instead of offering Download. A type of model this version cannot
run at all (Qwen3.8-Flash-Next, for one) now says so in the results, instead of offering a download
that is refused once it starts. The daily list of popular models follows the same rule. Support for
those formats and layouts is planned.

A full computer that frees no room for a part the swarm is short of says why. Its log names
every rule that kept a part it holds, and how many parts each kept, at most every 10 minutes; the
hourly "storage is over the budget" warning carries the same list instead of blaming the number of
copies the network needs, which nothing had checked. Log only — no part is kept or given back
differently.


Full commit list: v0.3.270-alpha...v0.3.271-alpha