Examples Python Production

Production

Stable local OpenAI-compatible URL

import xlocllm

llm = xlocllm.unit("LLM", "Qwen-3.5-0.8b", quant="q4", reasoning=False)
runtime = xlocllm.runtime([llm], port=1146, mode="native")
runtime.run()
print(runtime.url)

Explicit cache/state directory

$env:XLOCLLM_HOME = "D:\xlocllm-state"
python app.py

No auto-install native dependencies

$env:XLOCLLM_NATIVE_DISABLE_INSTALL = "1"
python app.py