Local LLM support?

[bjarkiella]

Ive been playing around with local llm's and trying to find a suitible user case in the coding realm for it. Ive looked into the autocomplete one (llama.nvim), but ive never really liked that "copilot" autocomplete feel. Then I found this one and got curious and tried it out.

I understand that this tool makes use of agentic workflow when searching, vibing and visualizing. A local model might struggle with that, since local agentic workflow is no where near as good as the "cloud" models, but it could perhaps be useful in visual mode since local agentic models perform ok with smaller tasks. I think this could be possible, with some modifications.

Im only rocking a whooping 12gb of vram, so im not expecting anything mindblowing but i think it would be fun to give it a try. Has someone tried implementing it before or is there an impossible hurdle im not seeing?

[JustusBraun]

Maybe you could configure opencode with a local model and tell 99 to use the local model to try how well local models can work with small tasks using the existing flow 🤔

[bjarkiella]

Of course, forgot that it is possible and it actually works :)

Thanks for the pointer!