Python Unit Syntax

Unit Syntax

Catalog model by type and alias

llm = xlocllm.unit("LLM", "Qwen-3.5-0.8b")
emb = xlocllm.unit("embedding", "multilingual-e5-small")
unit = xlocllm.unit("chat", "qwen-0.8b")  # alias type -> LLM

Keyword form

llm = xlocllm.unit(type="LLM", model="Qwen-3.5-0.8b", quant="q8", reasoning=False)

From ModelInfo

info = xlocllm.model("Qwen-3.5-0.8b", unit="LLM")
llm = xlocllm.unit(info)

Context manager

with xlocllm.unit("LLM", "Qwen-3.5-0.8b") as llm:
    ...

With RAG attached to LLM

emb = xlocllm.unit("embedding", "multilingual-e5-small")
rag = xlocllm.rag(emb=emb, name="kb")
llm = xlocllm.unit("LLM", "Qwen-3.5-0.8b-fp32", rag=rag)

Custom ONNX / sklearn / torch

onnx_unit = xlocllm.unit("model.onnx", type="regression", name="reg", input_name="float_input")
sk_unit = xlocllm.unit(sklearn_model, type="text-classification", name="clf", labels=["no", "yes"])
torch_unit = xlocllm.unit(torch_model, type="regression", name="torch-reg", input_shape=[None, 8])

Quantization

Для native GGUF LLM quant принимает q2, q3, q4, q5, q6, q8, fp16, fp32. Если quant не задан, SDK запрашивает q4 и использует fallback order из каталога.