Skip to content

v0.5.13

Choose a tag to compare

@Andyyyy64 Andyyyy64 released this 25 Jun 04:57

Added

  • HF_ENDPOINT support for Hugging Face model metadata fetches, so users behind a mirror can point whichllm at a compatible Hub endpoint. (#128, #131)
  • Manual detected-GPU overrides for usable VRAM and bandwidth, useful for iGPU and unified-memory systems where automatic detection is too conservative. (#132, #133)
  • README guidance for safer first-run flags when users want full-GPU, usable-speed recommendations with extra VRAM headroom.

Fixed

  • Search terms such as 7B, 0.5B, and 500M now match model parameter size instead of plain substrings, so qwen 7b no longer returns 1.7B or 30B-A3B by accident. (#107, #126)
  • GGUF sizing now treats FP16 and ternary TQ1_0 / TQ2_0 quant types correctly, avoiding underestimates for those files. (#125)