Skip to content

v.0.5.4-alpha

Choose a tag to compare

@omgitsbase omgitsbase released this 07 Oct 11:07
· 33 commits to main since this release

Install command, run in a user level powershell.

irm https://raw.githubusercontent.com/omgitsbase/llmash/main/install.ps1 | iex

On Linux:

curl -fsSL https://raw.githubusercontent.com/omgitsbase/llmash/main/install.sh | sh

Already installed: llmash update.

Changed

  • A drafter pulldraft fetches for a model in an Ollama store your user can't write to (on Linux the store usually belongs to the ollama account) is used again. pulldraft saved it to the loose-GGUF folder, but the server only looked beside the model's blob, so the model kept drafting by n-gram lookup. The server, pulldraft and run now look in both places, for an MTP head, DSpark, DFlash, EAGLE-3 or a draft model alike. Reported and pinned down by @ibilbrey.
  • llmash serve reads LLMASH_MODELS, as the help, the installers and llmash models already said it did.

Checked before release:

  • The client program and its tests on Windows and Linux.

The runtime is still build 146, so an update from 0.5.3 does not download it again.