Looking at AI Hosting Local/Cloud

[Stressed-Out09]

Hi guys,
I'm getting into odysseus and I'm really finding out how bad local AI modals are at tool calling (biggest modal i'm using is qwen3.6:27b)
I'm running a rx7900xtx (sharing gaming pc as AI pc) and because of this I've tried out Google's free tier API and man is it so much better and faster which got me thinking of should I spend the money building a dedicated AI computer, go cloud based, or just stick with what I've got and hope it gets better soon?

I'm really struggling here as I love odysseus and my privacy but my local AI models really only do basic tool handling reliably. (gemma4, Qwen3.5, Qwen3.6)

[SirThinksAl0t]

I'm curious about this too, and I have a very similar setup with a RTX4090 and the same models as you.
So far I didn't ran into a limitation but I'm trying to improve the setup.

[Stressed-Out09]

I'm running cachyOS and docker and I'm hosting Odysseus/ollama on the same pc so running a modal like qwen3.6:27b runs me out of vram whenever I utilise it so I usually run gemma4:12b as its a nice middle ground but I'll have fun parts where it won't use the documents tool and try to bash or pythong the tool instead of actually just using the tool. :/

[j-hunter]

While I voted keep as is, I'm in a similar boat. It feels like local models need more hand holding in these cases. (Also running Gemma4-12B on my windows PC to serve a debian VM running the odysseus container.) The main point for me about using local in the first place is the privacy concern, so using a cloud is still ruled out except in specific curated use cases. That said, I feel like V100s (And other data center hardware) are hitting the used market at 'reasonable' prices, and when I can stomach the much overdue upgrade to my main computer, a second dedicated build to run AI might come out of the leftover parts. (So a vote for build a dedicated PC, but only if it makes sense and after the tools prove their worth to you to justify the expense.)