Python Model Examples

Model Examples

Pick a fast local LLM

llm = xlocllm.models(unit="LLM", mode="native", max_vram_mb=1500)[0]
print(llm.label, llm.vram_mb)

Pick RAG models

emb = xlocllm.models(unit="embedding", mode="native", use_case="rag", limit_per_unit=3)
rerank = xlocllm.models(unit="reranker", mode="native", use_case="search-ranking", limit_per_unit=3)

Resolve quantized GGUF metadata

q8 = xlocllm.model("Qwen-3.5-0.8b", unit="LLM", mode="native", quant="q8")
print(q8["files"])

Pass ModelInfo into unit()

info = xlocllm.model("multilingual-e5-small", unit="embedding")
emb = xlocllm.unit(info)