Project Maya v1.0.2

Faster prefill (how fast Maya reads your prompt): up to 46% on two GPUs and up to 2x on one. Update with git pull, then ./setup.sh (Windows: START-MAYA.bat); it recompiles only what changed.

What's new

• Prefill speed: the experts a prompt needs that are neither in VRAM nor in RAM are read from the SSD ahead of
  time into a deeper buffer (the reader was keeping the NVMe at about a quarter of its speed), the next layer's
  experts are read while the current one computes, and the prompt is cut into bigger pieces on bigger cards (fewer
  times every expert is fetched). Prefill measured on 2x Tesla V100 32 GB with 30 GB RAM, prompts of 2k / 8k / 16k /
  30k tokens: 240 / 331 / 345 / - -> 273 / 428 / 487 / 506 tokens/s. On one of those GPUs: prefill of an 8k-token
  prompt 115 -> 226 tokens/s. Answers are unchanged in quality (the same text, up to rounding).
• Support Project Maya: buymeacoffee.com/peasantsmith (README, and the
  Sponsor button on GitHub).
• README: how Maya grew out of Strata; exported chats are named maya-chat-*.md.