Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions cmd/spinloop/serve_daemon_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1002,7 +1002,7 @@ port = 8000
[qwen]
model = Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed
context-window = 32768
scheduler-mode = parallel
scheduler-mode = ar_batch
max-active-requests = 4
model-id = preset-alias
`
Expand Down Expand Up @@ -1051,7 +1051,7 @@ func TestNodeDeployConfigMtplxFromPreset(t *testing.T) {
}
// The operator's own settings survive: the bind, and the scheduling mode,
// which spinloop does not compute.
for _, want := range []string{"--host 0.0.0.0", "--port 8000", "--scheduler-mode parallel"} {
for _, want := range []string{"--host 0.0.0.0", "--port 8000", "--scheduler-mode ar_batch"} {
if !strings.Contains(args, want) {
t.Errorf("a node's serve args should keep %q, got: %s", want, args)
}
Expand Down
4 changes: 2 additions & 2 deletions cmd/spinloop/serve_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ port = 8000
[qwen]
model = Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed
context-window = 32768
scheduler-mode = parallel
scheduler-mode = ar_batch
max-active-requests = 4
c = should-stay-literal
`
Expand Down Expand Up @@ -810,7 +810,7 @@ func TestCmdServe_MTPPLXPresetKeysPassThrough(t *testing.T) {
for _, want := range []string{
"--model Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed",
"--context-window 32768",
"--scheduler-mode parallel",
"--scheduler-mode ar_batch",
"--max-active-requests 4",
"-c should-stay-literal",
"--download",
Expand Down
10 changes: 6 additions & 4 deletions docs/commands/serve.md
Original file line number Diff line number Diff line change
Expand Up @@ -230,10 +230,12 @@ BASEURL http://127.0.0.1:8000/v1 # mtplx serve --host/--p
- `BASEURL` sets the bind address. With none, no bind flag is emitted and
MTPLX's own defaults stand.

The scheduling mode (`--scheduler-mode`: `serial`, `parallel`, `concurrent`) is
per-deployment tuning, not a Spinloop field — set it in a `PRESET`, written in
MTPLX's own long-form flags. `serve` passes every other preset key through
unchanged, so a preset is portable only to MTPLX, as with every engine.
The scheduling mode (`--scheduler-mode`) is per-deployment tuning, not a
Spinloop field — set it in a `PRESET`, written in MTPLX's own long-form flags.
`serve` passes the value through without checking it, so it must be valid for
the installed `mtplx` (`mtplx serve --help` lists the current modes). Every
preset key is passed through unchanged, so a preset is portable only to MTPLX,
as with every engine.

`serve` never passes `--api-key`, for the same reason as [oMLX](#omlx): it
prints the command it runs, and a key on the line would be in your screen and
Expand Down
8 changes: 5 additions & 3 deletions examples/mtplx/qwen3.8-27b/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ mtplx serve \
--model-id qwen3.8-27b \
--context-window 32768 \
--max-active-requests 4 \
--scheduler-mode parallel \
--scheduler-mode ar_batch \
--download \
--host 127.0.0.1 --port 8000
```
Expand All @@ -44,8 +44,10 @@ What the flags do:
- `--context-window` — the context a single request gets. Never scaled by
`PARALLEL`.
- `--max-active-requests` — an admission cap on how many requests run at once.
- `--scheduler-mode` — `serial`, `parallel`, or `concurrent`. This is
per-deployment tuning, not a Spinloop field, so it lives in the
- `--scheduler-mode` — how admitted requests execute. `spinloop` passes the
value through without checking it, so it must be valid for your `mtplx`
build (`mtplx serve --help` lists the current modes). This is per-deployment
tuning, not a Spinloop field, so it lives in the
[`preset.ini`](preset.ini).
- `--host`/`--port` — the OpenAI-compatible API is served at
`http://127.0.0.1:8000/v1`.
Expand Down
2 changes: 1 addition & 1 deletion examples/mtplx/qwen3.8-27b/preset.ini
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,4 @@ port = 8000
model = Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed
context-window = 32768 # keep in step with the Spinloop's CONTEXT
max-active-requests = 4 # the Spinloop's PARALLEL, when it states one
scheduler-mode = parallel # serial | parallel | concurrent — per-deployment
scheduler-mode = ar_batch # per-deployment; 'mtplx serve --help' lists the current modes