Skip to content

v1.1.2

@Mapika Mapika tagged this 22 Sep 16:58
The server picks its device as decider.infer.Decider does (CUDA, else MPS, else CPU; float16 on MPS, bfloat16 elsewhere);
DECIDER_DEVICE overrides it. Off CUDA the graph warm-up is skipped and requests run eager; MPS gets the library's patch.
An unavailable device, or FP8/compile off CUDA, stops start-up with a message naming the requirement. /health reports the
device. Checked on CPU with no GPU visible: same probabilities as Decider(device='cpu') on the README example. 117 tests.

Fixes #5

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Assets 2
Loading