Examples Python Quick tests

Quick tests

Smoke test LLM

import xlocllm

with xlocllm.runtime([xlocllm.unit("LLM", "Qwen-3.5-0.8b")]) as rt:
    rt.run()
    print(rt.chat("Say hello in one sentence.", temperature=0))

Smoke test embeddings

import xlocllm

emb = xlocllm.unit("embedding", "multilingual-e5-small")
with xlocllm.runtime([emb]) as rt:
    rt.run()
    print(rt.embed(["hello", "world"])[:1])

Show recommended models

import xlocllm

print(xlocllm.benchmark("LLM"))
print([m.model_id for m in xlocllm.models(unit="LLM", max_vram_mb=1500)])