Launchers

Launchers

A launcher is one model file together with its settings. The list on the left shows them; ▶ marks the one that is loaded, ⇣ one that is still downloading.

The form

• A ticked box means the option is used. Unticked means llama-server's own default.
• On the GPU: all puts the whole model on the graphics card (it has to fit), auto puts on it what fits and runs the rest from normal RAM (slower), or give a number of layers. Not set means the default from the Settings tab.
• Experts in normal RAM: for mixture-of-experts models (Qwen3.6-35B-A3B and the like). The experts, most of the file, stay in normal RAM and only the small part that is always used goes on the graphics card. That lets a model several times the size of the card run at a good speed.
• Memory estimates what the launcher needs against what the graphics card has. Orange means tight or only partly on the graphics card, red means too big.
• Writing style buttons (Model defaults, Factual, Creative, Story) set several options at once. They are only defaults; a persona can set its own.
• Extra options: anything the form doesn't know, typed as on a command line.
• What will run shows the full command, updated while you change things.

Buttons

• Save / Revert: changes only count after Save.
• Save & Start saves, stops the model that was loaded and loads this one.
• Stop llama-server stops only the model.
• Add model… brings in a new model: from a Hugging Face link (it lists the versions and marks the one that fits this machine) or from a file that is already on the disk. Downloads run on their own and continue where they stopped, also after closing AIcove.
• New from this one… copies the form into a new launcher.
• New persona on this one… makes a persona with its own copy of this launcher.
• From another machine… adds the launchers of another computer of yours (see below).
• Update from other machines asks those computers what they have now.
• Rename…, Delete: Delete removes the launcher (a copy is kept). If no other launcher uses its model file, the box Also delete the model file from the disk offers to delete the file too, with its eyes file and a half-finished download. Not while that model is loaded: stop it first.

Useful options: Context size is how much of the chat the model remembers (bigger needs more memory). Parallel chats set to 1 gives the whole context to one chat. Compress context (K) and (V) roughly halve the context's memory for a small loss in quality. Max answer length stops an answer that would otherwise run on.

Thinking budget caps how much a thinking model thinks before it answers (in tokens; it counts against the answer's length too). When the budget is used up is a sentence the model is told at that moment, so it moves on to answering cleanly. Both apply to every persona on that launcher; they take effect when the model is started again.

Models on another machine

A second computer with AIcove (say one with a faster graphics card) can run the model while the chat stays here: personas, chats, files, pictures and voice remain on this machine, only the chat text goes back and forth. You need an ssh key that logs in to that machine without a password, and a name for it in ~/.ssh/config (a Host line). From another machine… asks for that name and adds all its launchers, as machine-name. On this tab they are listed under a heading with the machine's name, below this machine's own; elsewhere they carry 🖥 and the machine's name, and the status says on that machine. They follow that machine: AIcove asks it when the window opens, when one of its launchers is started, and on Update from other machines. Then a new launcher there appears here, one deleted there goes here too, and a changed name or model shows. One you delete here stays away until you add the machine again. Start starts the model over there with that machine's own settings and opens an ssh tunnel, so to the chat it looks like a model here. Stop llama-server stops it there and closes the tunnel. The settings of such a launcher are changed on the machine it belongs to. Its model stays private there: nothing is opened to the network.

By hand, a launcher file with two lines does the same: host: <name from ~/.ssh/config> and launcher: <its launcher there>.

---

Made from the guide inside AIcove (cove/data/guide.md); it opens with the ? button.