A RAM release. Every agent's MCP server used to carry its own copy of the encoder, and the encoder never gave memory back. Now the model runs once per machine, and the encoder stays small whatever it is asked to embed.
What was measured
On an 8 GB M1 laptop running Claude Desktop, four Claude Code sessions and the menu bar app, six compartment serve processes held between 160 and 390 MB each, about 1.3 GB together, for one 61 MB vault. Two things caused it.
The first is onnxruntime's memory arena. It keeps every byte it grows to and grows again on the next batch: one embed_passages of 64 full 448-token windows took a process from 118 MB to 1.5 GB resident, a second identical call to 3 GB, and neither was ever returned. A long memory_store_many did that to whichever server ran it.
The second is duplication. Every server loaded the runtime, the tokenizer and the model, about 50 MB, and every server ran its own arena.
The encoder keeps its RAM
The arena is off, and batches are sized in padded tokens (4096) rather than in texts, with texts sorted by length so a batch never pads sixty short memories out to the length of one long one. The same 64 long windows now peak at 250 MB while they are in flight and leave the process under 90 MB when they are done, in the same time. This applies to every process that embeds, on every platform.
Opening a vault no longer builds the encoder to check the model pin. The pin is a SHA-256 of a file, so the file is hashed and nothing is loaded; the model is loaded by the first store or search that needs it. compartment status, the app's poll and an MCP server nobody has searched from yet cost no runtime at all.
One embedding process per machine
On macOS and Linux the model now runs in a shared daemon. The first compartment serve, command or app that needs a vector starts it; the rest connect to it over a Unix domain socket in the session directory, beside the unlock credential. It binds within a fifth of a second and loads a model on the first request for it, holds about 70 MB with the model loaded, and exits five minutes after its last client hangs up. A server that asks it for vectors instead of loading its own runs at about 60 MB plus its vault and index.
Only text goes in and only unit vectors come out: no key, nothing from the vault at rest. The daemon accepts connections from the same uid only, a client connects only to a socket its own uid owns, and both agree on the protocol, the package version and the model's SHA-256 before a vector crosses. A daemon left behind by an older install is told to exit by the first newer client. If the daemon is killed or unavailable the client reconnects, restarts it, or falls back to a local model; a vault never notices. --assert-offline allows AF_UNIX and forbids everything else, exactly as before. SECURITY.md has the full account.
compartment embed-daemon status shows the process, its clients, the models it holds and its size; stop ends it, and the next search starts a fresh one. compartment update stops it so the next search runs the new build. COMPARTMENT_EMBED_DAEMON=0, or embed_daemon: false in the vault's settings, keeps the model in each process; Windows, which has no AF_UNIX in Python, always does. memory_status reports embedder: where this process's model lives, and the daemon's pid.
Upgrading
pip install --upgrade compartment, or compartment update. Existing vaults open unchanged and no vector is recomputed: the model, its pin and the stored vectors are the same. Servers already running keep their old copy of the model until they are restarted. The app bundle is updated by the installer below.
Downloads
| Artifact | For |
|---|---|
Compartment-4.9.6.pkg |
macOS, one click, with the optional menu bar utility |
Compartment-4.9.6.dmg |
macOS, drag to Applications |
compartment-4.9.6-py3-none-any.whl |
pip / offline installs, any OS |
compartment-4.9.6.tar.gz |
source distribution, packagers, air-gapped builds |
pip install compartment