llama.cpp v0.8.0

[yunguangli]

Some AI models , for example, LiquidAI LFM3 model card says that LFM3's unique tri-hybrid architecture is only supported in llama.cpp v0.8.0 (build b11960) and newer.

The pre-compiled wheel hosted on Flet's binary package registry is locked at version 0.3.3. Could we possibly has a version for 0.80+ ?

Thanks!

[ndonkoHenri]

llama-cpp-python 0.3.36 is now available on pypi.flet.dev (recipe update), but it won't load LFM3 yet.

The v0.8.0 on the model card is a llama.cpp version, not a llama-cpp-python one, and llama-cpp-python has no 0.8 release. Its latest release, 0.3.36, bundles an older llama.cpp commit, from before LFM3 support landed in b11960, so the lfm3 architecture isn't in it.

Our wheels build whatever llama.cpp upstream bundles, so LFM3 support depends on a llama-cpp-python release that includes b11960 or newer. We'll update the package once that's out. Keep in mind the smallest LFM3 files are a bit over 1 GB, so check that your target devices have enough memory.

[yunguangli]

Yes, I successfully built an 1.6 GB chatbot apk with the LFM3 and installed it on my 3 year old Android phone (12G memory). 16 tokens/ second. Not bad. I used onnx instead of llama. Repository is in https://github.com/yunguangli/edgeAI_chatbot. llama version shall run faster.