Skip to content

v1.0.0

Latest

Choose a tag to compare

@huangzhengxiang huangzhengxiang released this 20 Aug 05:34
· 16 commits to master since this release

MNN-LLM v1.0.0
New Features:

  1. Individual Sampler (greedy, temperature, top K, top P, min P )
  2. StateCacheManager (Paged Attention, Fused CPU Attention op)

qwen1.5-4B-chat model in MNN format with fused attention operator is appended in the Release binary.