Repository navigation
v0.2.0: LM Studio engine
Adds a second local engine: any chat model served by LM Studio.
- How it answers: each option is shown as a letter and the model replies with one token. The confidence is the probability the model gave each letter (
logprobs), not a score it writes about itself. - No thinking: reasoning is turned off (
reasoning_effort: none) - Where to choose it: engine selector in the web UI;
--engine lmstudio,--modeland--lmstudio-urlin the CLI - Optional API key: read from
LMSTUDIO_API_KEY - New example: severity (
score) example
Results on the bundled examples (Qwen 3.5 4B, laptop CPU):
| Laya | LM Studio | |
|---|---|---|
| choice / noul | 86–100% | 100% |
| score (severity) | 10/14 | 11/14 |
| Time per row (3 questions) | ~0.6 s | ~14 s |
score remains experimental on both engines.