Skip to content

v0.3.6

Choose a tag to compare

@github-actions github-actions released this 28 Jun 12:30
· 52 commits to main since this release
82ed0fa
Add --server-args: one whitespace-delimited string of llama-server flags

Speculative decoding has many flags; passing them one --server-arg at a
time is tedious. --server-args takes a single string that's split on
whitespace and appended to llama-server's args, e.g.
  --server-args "--spec-type draft-mtp --spec-draft-n-max 2"

Both forms are supported and combined (repeatable --server-arg first, then
the split --server-args). Added allow_hyphen_values so flag-like values
("--foo") are accepted as values — the repeatable --server-arg silently
rejected those before. Test covers the combine.