Repository navigation
Berth v0.4.0-rc.1
Pre-release
Pre-release
·
168 commits
to main
since this release
A pre-release for testing. Production installs stay on the latest release; berth update never offers a pre-release.
Fixed
- A custom park hook (
park_url,resume_url) sends{}withContent-Type: application/json. It sent an
empty body, which Strata's/v1/unloadand/v1/loadrefuse with 415, so a Strata entry could not park through
them. The built-in vLLM and SGLang calls and the Berth park protocol are unchanged (#96). - A resumed lease that is still waiting for the card counts against it only for the residual it holds, not for the
whole card it asked for. Its weights are in RAM until its grant lands. Aberth holdbehind a resumed ComfyUI
waited 1m44s with nothing to drain, because berthd still charged it ComfyUI's whole card. Such a waiter is now
granted beside the residual, or the parked engine gives way first, exactly as beside any parked engine. - The harbor library shows SERVED BY and EST. VRAM for a llama-swap model whose ID differs from its GGUF file's
stem, such asqwen-3.8-27bforQwen3.8-27B-UD-IQ4_XS. Berth reads the model file from thecmdin
llama-swap's/runningand matches the library row on that file before it tries the name (#197). - The AMD probe no longer charges a process for purgeable VRAM, the buffers the kernel may reclaim. A GNOME
desktop on the same card reported more resident VRAM than the card used, so berthd refused every reading and
exited at startup. Purgeable bytes now stay in the card's baseline beside shared ones (#196). berth service install,uninstallandstatusprint the error when they fail. They panicked instead, for
example under a shell with no user-session D-Bus, wheresystemctl --usercannot answer (#196).- Park admission counts the RAM a parked engine holds: the anon and shmem of its berth scope's cgroup, not its VRAM
footprint. A parked ComfyUI was counted at 17.5 GiB while it held about 2.5 GiB, because its weights sit in page
cache, which the kernel counts as available. Admission also subtracted parked RAM a second time from a reading that
had already dropped by it. Both refused parks that fit. The next park of an engine now costs what its last park
added: the RAM it grew by while it parked, since what it held hot was already out of the reading. A park still in
flight now counts against the next one, so two engines parking at once cannot claim the same free RAM. A run with
no berth scope still counts its footprint (#193). - A lease that parks a second time gets the whole release grace for its own park call. It ran from the previous
park's call, so a park that takes longer than one tick (a 23 GBcuda-checkpointtakes 5 s) was given up on at
once, and the engine was left checkpointed while berthd waited for it to drain. - A drain that arrives while the run is still parking the engine (
berth stop, or berthd giving up on the park) is
kept and acted on when the park ends, instead of dropped. berth stopstops a parked or parking engine. It answered "no running engine" before.
Added
- Pre-releases for testers. Work for the next release lands on the
nextbranch and is taggedvX.Y.Z-rc.N, which
publishes as a GitHub pre-release. The README's download link,go install ...@latestandberth updatestay on
the latest release. Testing a pre-release covers installing one and going back. berth config checkwarns when[park]turns eviction off:floor_gib = 0means Berth never stops a parked
engine when RAM runs low, and apressureof 100 or more is past anything/proc/pressure/memoryreports.
--strictfails on both (#193).- Engines with no offload call of their own (llama.cpp, NInfer, any llama-swap model) park in RAM through NVIDIA's
cuda-checkpoint:berth run --park checkpoint --front 127.0.0.1:${PORT}with--port {{BERTH_PORT}}in the
engine's command. The front takes the requests llama-swap sends, waits for the ones in flight before the
checkpoint, holds new ones while the engine sleeps, and asks berthd for the card on the first, so no request
hangs on a checkpointed engine. A 23.1 GB NInfer comes back in 3.3 s instead of a 14 s reload.park = "checkpoint"on an engine entry is refused: a front is one run's address. docs/design.md is the model this and
the coming lends, multi-card leases and networked nodes follow. - Experimental: an engine that can shrink a cache lends part of its card to a sized waiter instead of being drained for it:
lend = "hook"on the entry, withlend_urlfor an engine that serves the lend routes
(contrib/LEND-PROTOCOL.md), and Strata's/v1/vrambuilt in. The holder keeps
serving while the borrower builds beside it. Berth grants the borrower only once a card read taken after the
engine answered shows the memory free, and the lend stops counting against the holder at that read, never at
the engine's word. The holder is asked for its memory back once the borrower's is measured gone, and only when
the card has room for it again. A lend the engine refuses, or one the card has not confirmed within
--release-grace(30 s), falls through to the park or drain that would have run anyway. Nothing is asked of a
run that did not declare lend calls, of a pinned holder or a hold, or on an APU. This is the broker's decision
and the wrapper's calls, and the wrapper keeps the ledger of what its run has lent. A lend the engine did not
answer in time (its503) is treated as refused for that waiter, and the wrapper reads the engine's ledger
(GET /vram/state) for 30 s in case it lands, then asks for that memory back at once. After a
berthd restart the run re-attaches naming its open lend, so the new daemon counts the holder's budget less the
lend and asks for the take-back once the card has room, instead of blocking waiters behind memory that is gone.
lease.lending,lease.lent,lease.returnedandlease.lend_failedname it on the event stream, and
docs/gateway.md's Lending section states the rules. - A Docker Compose project can run as one engine:
compose = "/abs/path/compose.yml"on an entry, or
berth run --compose PATH. The lease covers every container in the project, found with
docker compose -f PATH ps -q --allon every poll (podman when docker does not answer), so a container that
restarts under a new ID comes back to the lease. A drain runscompose stop -t <grace>, neverdown, which would
delete containers and networks you own. The start command stays yours, anddocker compose -f PATH upfits.
berth config checkrefusescomposetogether withcontainer, and a relative path.
Container engines now work on Windows too, where--containerwas accepted and ignored. WSL2 reports no
per-container VRAM. So while one container engine is the only one on the card, Berth charges it the WSL VM's
measured GPU row whenever that row is larger than itsneeds_mib. With two container engines Berth cannot split
the row, so each counts itsneeds_miband the row stays a stowaway, counted once. Sizeneeds_mibfrom your
heaviest run. One operator's photo tagger measured 2.1 GiB on a small scan and 6.7 GiB on its biggest (#172).
Inberth status --format jsonand the harbor, a lease'scontainerandcontainer_nameare replaced by
containers, a list of full container IDs. Aberth runstarted before an upgrade still attaches its container.