Fantastic project! Feedback

[jxs714]

Hey guys,

Just wanted to chime in and say this is a fantastic project!! I was finally able to replace other inference GUI platforms I've used on linux. Keep up the great work. The only changes I would make is better model management and scanning. I had to create a script to pick up my local existing models and mmproj vision tools in order for llamaforge to add it to the list with full features. If you guys can add vllm on linux, that would be awesome as well. Otherwise, this software is fantastic and I've already fully replaced it with my productivity tools for using it on things like openwebui, and others.

I hope you guys keep working on the project and improving it. I am excited to see the growth!

[dadwritestech]

Hey @jxs714 sorry for the slow reply. one-person project and it took me a while to get here.

glad it's working on linux and playing nice with openwebui. that's what i hoped for.

Model scanning got updated since you posted. you can now point it at specific folders instead of whole drives. the scanner also links MTP draft models to their base model when it finds them next to each other.

Your script was working around a real bug with mmproj. the scanner only attached mmproj files if the model's architecture matched a hardcoded list, and newer vision models like gemma 4 and qwen 3.5 weren't on it. so it found your mmproj file but skipped it. i tested this on my machine and it skipped 12 out of 12. that's fixed now, it'll be in the next release. projectors get matched to a model by their actual size in the file header instead of a name list, so several quants of the same model in one folder can share one mmproj, and it won't hand a projector to a model it doesn't fit.

vLLM on linux is still windows/WSL2-only. I opened an issue to track native support: #27. the main work is getting vllm to run directly on linux instead of through WSL. if you're up for contributing, that'd be appreciated.

If you still have your scanning script around, could you share what folder layouts it handled? i'd like to test the fix against those same cases before releasing.