Repository navigation
v0.3.1: evaluation on public data
LUAR was tested on 300 texts from each of two public datasets labeled by people. The manual now uses these numbers instead of figures from our short hand-written examples.
| Laya | LM Studio (Qwen 3.5 4B) | |
|---|---|---|
| News topic (English, AG News) | 93% | 89% |
| Would recommend? (Portuguese reviews, B2W) | 70% | not run |
| Sentiment pos/neutral/neg (B2W) | 68% (neutral: 5%) | not run |
Star rating 1–5, score (B2W) |
26% (60% within one star) | not run |
| Time per row | ~1 s | ~8 s per question |
Method, script and full results: bench/.
Changes
- Score questions: low-confidence
scoreanswers no longer mark rows for review. On real reviews they marked 97% of rows; a missing answer still counts. - Manual:
- Speeds and accuracy: real figures, with Laya recommended as the default engine.
- New tip: on "middle" options such as neutral.
- New note: what the confidence is worth on different kinds of data.
- Evaluation script: saves results after each dataset.