Skip to content

MeteorLLM v0.2.0

Choose a tag to compare

@daivarsi daivarsi released this 05 Nov 02:33

MeteorLLM v0.2.0 adds rolling conversations, configurable reply length, and faster local Qwen3-0.6B inference on PS Vita.

Vita runtime

The same 67-token formatted Tavi prompt returned Tavi. with token IDs 51, 6190, 13 and natural EOS.

Build First token Submit to EOS
Earlier optimized build 92.58 s 136.38 s
Vocabulary shortlist, no template cache 60.63 s 69.57 s
Previous tiled default, template ready 32.71 s 41.69 s
v0.2.0, template ready 30.07 s 39.00 s

The newest matrix kernel processes eight prompt tokens across SIMD lanes. The build also uses int8 activations, a Q4 output projection, and a vocabulary shortlist. Preparing the shared 32-token template took 22.72 s at startup. Preparation plus the measured response took 61.72 s, excluding model load. The question and generated answer were processed during the timed run.

This is a single short prompt on physical hardware. The shortlist and quantized math can change output on other prompts. See the README for the benchmark settings, clock measurement caveat, and full-vocabulary and cold-mode overrides.

Install

  1. Install MeteorLLM.vpk.
  2. Keep or prepare the original weights.bin, embedding.bin, and tokenizer.bin using the repository instructions.
  3. Extract output-q4-test.bin from MeteorLLM-Q4-output.zip and put it in ux0:data/VitaDesk/MeteorLLM/ with the model files. This enables the faster output path for the tested model.

The output archive includes the upstream model license and a modification notice. The measured Vita run is attached as Tavi-Vita.log.