Could Google's Turboquant get incorporated into llamafile?

[kreely]

Hi,
I'm wondering if there's a plan to incorporate Turboquant into llamafile....

I think at least one person has built turboquant into llama, so in theory (?) it should be possible to build it into llamafile...

It may be harder than I realise, but thought I'd ask.

Cheers

[aittalam]

Hi @kreely, many thanks for your interest and for the link, and sorry for the late reply!

While I like to occasionally experiment with new models / ideas before llama.cpp, for turboquant I'd feel more comfortable standing on their shoulders. I am following the conversation here and the currently open PRs and I'll try to sync soon after the new feature is in.

And of course you did well asking! Feel free to do so anytime, this is also a good way for us to get a sense of people's interest in llamafile 🙏