Repository navigation
MeteorLLM v0.2.0
MeteorLLM v0.2.0 adds rolling conversations, configurable reply length, and faster local Qwen3-0.6B inference on PS Vita.
Vita runtime
The same 67-token formatted Tavi prompt returned Tavi. with token IDs 51, 6190, 13 and natural EOS.
| Build | First token | Submit to EOS |
|---|---|---|
| Earlier optimized build | 92.58 s | 136.38 s |
| Vocabulary shortlist, no template cache | 60.63 s | 69.57 s |
| Previous tiled default, template ready | 32.71 s | 41.69 s |
| v0.2.0, template ready | 30.07 s | 39.00 s |
The newest matrix kernel processes eight prompt tokens across SIMD lanes. The build also uses int8 activations, a Q4 output projection, and a vocabulary shortlist. Preparing the shared 32-token template took 22.72 s at startup. Preparation plus the measured response took 61.72 s, excluding model load. The question and generated answer were processed during the timed run.
This is a single short prompt on physical hardware. The shortlist and quantized math can change output on other prompts. See the README for the benchmark settings, clock measurement caveat, and full-vocabulary and cold-mode overrides.
Install
- Install
MeteorLLM.vpk. - Keep or prepare the original
weights.bin,embedding.bin, andtokenizer.binusing the repository instructions. - Extract
output-q4-test.binfromMeteorLLM-Q4-output.zipand put it inux0:data/VitaDesk/MeteorLLM/with the model files. This enables the faster output path for the tested model.
The output archive includes the upstream model license and a modification notice. The measured Vita run is attached as Tavi-Vita.log.