v1.1.0 - Performance & Inference Pipeline Improvements
⚡Performance & Inference Pipeline Improvements (v1.1.0)
This release significantly improves both accuracy and latency, making the system faster, more reliable, and better suited for real-world Roblox trade parsing.
📊 Performance
Accuracy (Exact-Match)
| Model Version | Correct | Total | Accuracy |
|---|---|---|---|
| v1.0.0 | 442 | 500 | 88.4% |
| v1.1.0 | 486 | 500 | 97.2% |
Δ +8.8% absolute improvement in accuracy evaluated on the same dataset.
~76% reduction in errors compared to the previous model.
Inference Latency (Batch = 1, End-to-End)
| Model Version | Median (ms) | p95 (ms) |
|---|---|---|
| v1.0.0 | 139.4 | 233.3 |
| v1.1.0 | 36.8 | 73.4 |
~3.8× reduction in median latency after pipeline refactoring.
Latency measured after warm-up over 50 iterations on full test set.
🛠️ Internal Improvements
| Area | Improvement |
|---|---|
| Inference Pipeline | CLIP-first architecture: CLIP predicts items first, OCR only triggered if CLIP is uncertain; the "judge" then selects the most confident source or skips if both are unconfident |
| CLIP Model | Custom-trained on 185,000+ image thumbnails across 2,500+ classes for highly accurate image-based identification; handles visually identical items with different names |
| OCR | Batched for faster processing; only runs when CLIP is uncertain, reducing expensive OCR calls |
| Pretrained Data | Added 6 previously missing items (now 2,553 total classes) to improve coverage |
🔍 Notes
- Accuracy is measured using exact-match evaluation (all fields must be correct).
- Evaluation performed on real trade screenshots representative of production usage.