Skip to content

Berth v0.4.0-rc.1

Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 26 Oct 17:28
· 168 commits to main since this release
9462729

A pre-release for testing. Production installs stay on the latest release; berth update never offers a pre-release.

Fixed

  • A custom park hook (park_url, resume_url) sends {} with Content-Type: application/json. It sent an
    empty body, which Strata's /v1/unload and /v1/load refuse with 415, so a Strata entry could not park through
    them. The built-in vLLM and SGLang calls and the Berth park protocol are unchanged (#96).
  • A resumed lease that is still waiting for the card counts against it only for the residual it holds, not for the
    whole card it asked for. Its weights are in RAM until its grant lands. A berth hold behind a resumed ComfyUI
    waited 1m44s with nothing to drain, because berthd still charged it ComfyUI's whole card. Such a waiter is now
    granted beside the residual, or the parked engine gives way first, exactly as beside any parked engine.
  • The harbor library shows SERVED BY and EST. VRAM for a llama-swap model whose ID differs from its GGUF file's
    stem, such as qwen-3.8-27b for Qwen3.8-27B-UD-IQ4_XS. Berth reads the model file from the cmd in
    llama-swap's /running and matches the library row on that file before it tries the name (#197).
  • The AMD probe no longer charges a process for purgeable VRAM, the buffers the kernel may reclaim. A GNOME
    desktop on the same card reported more resident VRAM than the card used, so berthd refused every reading and
    exited at startup. Purgeable bytes now stay in the card's baseline beside shared ones (#196).
  • berth service install, uninstall and status print the error when they fail. They panicked instead, for
    example under a shell with no user-session D-Bus, where systemctl --user cannot answer (#196).
  • Park admission counts the RAM a parked engine holds: the anon and shmem of its berth scope's cgroup, not its VRAM
    footprint. A parked ComfyUI was counted at 17.5 GiB while it held about 2.5 GiB, because its weights sit in page
    cache, which the kernel counts as available. Admission also subtracted parked RAM a second time from a reading that
    had already dropped by it. Both refused parks that fit. The next park of an engine now costs what its last park
    added: the RAM it grew by while it parked, since what it held hot was already out of the reading. A park still in
    flight now counts against the next one, so two engines parking at once cannot claim the same free RAM. A run with
    no berth scope still counts its footprint (#193).
  • A lease that parks a second time gets the whole release grace for its own park call. It ran from the previous
    park's call, so a park that takes longer than one tick (a 23 GB cuda-checkpoint takes 5 s) was given up on at
    once, and the engine was left checkpointed while berthd waited for it to drain.
  • A drain that arrives while the run is still parking the engine (berth stop, or berthd giving up on the park) is
    kept and acted on when the park ends, instead of dropped.
  • berth stop stops a parked or parking engine. It answered "no running engine" before.

Added

  • Pre-releases for testers. Work for the next release lands on the next branch and is tagged vX.Y.Z-rc.N, which
    publishes as a GitHub pre-release. The README's download link, go install ...@latest and berth update stay on
    the latest release. Testing a pre-release covers installing one and going back.
  • berth config check warns when [park] turns eviction off: floor_gib = 0 means Berth never stops a parked
    engine when RAM runs low, and a pressure of 100 or more is past anything /proc/pressure/memory reports.
    --strict fails on both (#193).
  • Engines with no offload call of their own (llama.cpp, NInfer, any llama-swap model) park in RAM through NVIDIA's
    cuda-checkpoint: berth run --park checkpoint --front 127.0.0.1:${PORT} with --port {{BERTH_PORT}} in the
    engine's command. The front takes the requests llama-swap sends, waits for the ones in flight before the
    checkpoint, holds new ones while the engine sleeps, and asks berthd for the card on the first, so no request
    hangs on a checkpointed engine. A 23.1 GB NInfer comes back in 3.3 s instead of a 14 s reload. park = "checkpoint" on an engine entry is refused: a front is one run's address. docs/design.md is the model this and
    the coming lends, multi-card leases and networked nodes follow.
  • Experimental: an engine that can shrink a cache lends part of its card to a sized waiter instead of being drained for it:
    lend = "hook" on the entry, with lend_url for an engine that serves the lend routes
    (contrib/LEND-PROTOCOL.md), and Strata's /v1/vram built in. The holder keeps
    serving while the borrower builds beside it. Berth grants the borrower only once a card read taken after the
    engine answered shows the memory free, and the lend stops counting against the holder at that read, never at
    the engine's word. The holder is asked for its memory back once the borrower's is measured gone, and only when
    the card has room for it again. A lend the engine refuses, or one the card has not confirmed within
    --release-grace (30 s), falls through to the park or drain that would have run anyway. Nothing is asked of a
    run that did not declare lend calls, of a pinned holder or a hold, or on an APU. This is the broker's decision
    and the wrapper's calls, and the wrapper keeps the ledger of what its run has lent. A lend the engine did not
    answer in time (its 503) is treated as refused for that waiter, and the wrapper reads the engine's ledger
    (GET /vram/state) for 30 s in case it lands, then asks for that memory back at once. After a
    berthd restart the run re-attaches naming its open lend, so the new daemon counts the holder's budget less the
    lend and asks for the take-back once the card has room, instead of blocking waiters behind memory that is gone.
    lease.lending, lease.lent, lease.returned and lease.lend_failed name it on the event stream, and
    docs/gateway.md's Lending section states the rules.
  • A Docker Compose project can run as one engine: compose = "/abs/path/compose.yml" on an entry, or
    berth run --compose PATH. The lease covers every container in the project, found with
    docker compose -f PATH ps -q --all on every poll (podman when docker does not answer), so a container that
    restarts under a new ID comes back to the lease. A drain runs compose stop -t <grace>, never down, which would
    delete containers and networks you own. The start command stays yours, and docker compose -f PATH up fits.
    berth config check refuses compose together with container, and a relative path.
    Container engines now work on Windows too, where --container was accepted and ignored. WSL2 reports no
    per-container VRAM. So while one container engine is the only one on the card, Berth charges it the WSL VM's
    measured GPU row whenever that row is larger than its needs_mib. With two container engines Berth cannot split
    the row, so each counts its needs_mib and the row stays a stowaway, counted once. Size needs_mib from your
    heaviest run. One operator's photo tagger measured 2.1 GiB on a small scan and 6.7 GiB on its biggest (#172).
    In berth status --format json and the harbor, a lease's container and container_name are replaced by
    containers, a list of full container IDs. A berth run started before an upgrade still attaches its container.