Straydog v0.1.6

Straydog 0.1.6. It is still a pre-release. What Straydog does and how fast it runs are in the
v0.1.0 notes.

This release is for machines whose RAM holds only part of a model's experts, such as a 64 GB Windows PC with an RTX 4060.

Changes

Fixes

• An engine that stopped for memory says why. A worker that stopped between requests was logged with a bare exit code, and
  the lines it wrote last, which name the cause, were kept out of the log. The error line now carries its last six lines (an
  allocation that failed, a GPU error) and its exit code, on Windows in hex with the memory codes named (STATUS_NO_MEMORY,
  STATUS_COMMITMENT_LIMIT).
• A prompt's CPU share no longer stops the engine when pinned memory is refused. It takes pageable memory then, and where even
  that is refused, the batch streams its experts as before. A batch gives the CPU at most 4096 routes.
• The engine no longer stops when the conversation outgrows the memory left. After a request, Straydog copied the conversation's
  state to host memory in one ordinary allocation of hundreds of MB to a few GB; where the system's commit charge was used up (a 64
  GB Windows machine with most of a model's banks page-locked), the allocation failed and ended the process (0xC0000017). That copy
  now asks first, gives back page-locked layers to make room, and else goes on without it. A refused growth of the K/V likewise
  takes back locked layers, newest first, and fails only the request when none is left; the locking behind ready leaves room for
  the K/V of the first 64K tokens. On an RTX 5070 Ti machine with its commit charge reduced, a conversation that failed at 30K
  tokens went on to 63K.

Faster prompts on machines short of RAM

• Layers that are not page-locked give the CPU only the experts it computes for less than their gather. There the CPU's
  workers gather each streamed expert into pinned memory first, and the CPU's own routes waited behind those gathers. On a Radeon
  AI PRO R9700 at 5.5 GiB in a 40 GiB memory scope (27 of 48 layers locked): prompts of 2.1K, 4.2K and 10.6K tokens 7-15% faster.
• Windows: a prompt's PLE rows are read at the drive's speed. NTFS ran unbuffered reads of a mapped file one at a time; the
  table's file is now opened apart. On an RTX 5070 Ti at 5.5 GiB the wait for the rows fell from 0.9-1.9 s to 10-26 ms, a 1.9K-token
  prompt from 4.5-5.0 s to 2.9 s.