Skip to content

mac_linuxgpu 0.1.162

Choose a tag to compare

@Geramy Geramy released this 07 Oct 14:38
· 18 commits to main since this release
  1. Code objects are sub-allocated from a code heap. The HSA runtime carves each program's code objects out of shared 4 MiB VRAM chunks, stages them in host memory and uploads them in one SDMA copy per batch, followed by one code-cache sync, instead of one allocation, one copy and several driver calls per object.
    • Loading a code object: 0.437 ms and 6.2 driver calls → 0.009 ms and 0.02 calls (98% time saved).
    • Destroying 1,000 code objects: 438 ms → 1.2 ms.
    • LemonSeed Engine's 262 code objects load in 2.3–3.4 ms instead of 130 ms, and its live driver buffers drop from 624 to 366.
  2. Code objects no longer take entries in the driver's 8,192-entry buffer table, which every program on the GPU shares. 20,000 code objects load in one process.

LemonSeed Engine's time to first token and decode speed are unchanged.

Upgrading from an earlier build: the old build does not yet leave on its own. Turn the display off in the menu bar, then use Disconnect GPU (or power-cycle the GPU), and the new build takes over.