Skip to content

Project Maya v1.0.6

Choose a tag to compare

View all tags
@mw00 mw00 released this 08 Oct 01:29
· 611 commits to main since this release
v1.0.6
2b0f1a3

A standard speed test: ./maya.sh --bench (Windows: START-MAYA.bat --bench) - decode and prefill on your machine in a few minutes, comparable everywhere. Update with git pull.

What's new

  • --bench measures the installed model on your machine in a few minutes, with Maya stopped: decode on the same
    three questions everywhere (after a warm-up answer) and prefill at 2k and 8k tokens, through the engine the way the
    dashboard starts it. It writes maya-bench.txt with the results and the engine's per-token breakdown; --report
    includes it - so speed reports from different machines can be compared.
  • README: single-GPU speeds (1x Tesla V100 32 GB, 64 GB RAM: decode up to 19 tokens/s, prefill up to 370 tokens/s).
Assets 2
Loading