I started using Unsloth Studio and I can't help but compare it with LM Studio. There are some features that I miss, one of them is the ability to tweak how the model is loaded. Currently it seems that it tries to fill the VRAM and offload everything else to RAM. I use Unsloth Studio directly on my main PC.
I would like more fine control so I can use models while I play games or watch videos. I like the control over how many experts are offloaded to RAM.
If you decided to add this feature I recommend that you also add Profiles. For example:
Profile 1 loads more layers into the GPU and decreases the context window.
Profile 2 has quantized KV cache and a larger context window.
This could be managed just before loading the model as a pop-up window with some tabs on top to choose the profile.
In regard to what sliders and toggles would have, I think it would be nice to cover LM Studio features. More users would feel familiar with it and have an easier adoption.
I did see the sidebar that lets you tweak the model, I just feel that there are things missing for me. this stuff could be implement directly there.
Beside that the app is pretty good, running it in docker is convenient.
I apologize if this isn't as technical as it needs to be — I am just a user that saw that something could improve.
I started using Unsloth Studio and I can't help but compare it with LM Studio. There are some features that I miss, one of them is the ability to tweak how the model is loaded. Currently it seems that it tries to fill the VRAM and offload everything else to RAM. I use Unsloth Studio directly on my main PC.
I would like more fine control so I can use models while I play games or watch videos. I like the control over how many experts are offloaded to RAM.
If you decided to add this feature I recommend that you also add Profiles. For example:
Profile 1 loads more layers into the GPU and decreases the context window.
Profile 2 has quantized KV cache and a larger context window.
This could be managed just before loading the model as a pop-up window with some tabs on top to choose the profile.
In regard to what sliders and toggles would have, I think it would be nice to cover LM Studio features. More users would feel familiar with it and have an easier adoption.
I did see the sidebar that lets you tweak the model, I just feel that there are things missing for me. this stuff could be implement directly there.
Beside that the app is pretty good, running it in docker is convenient.
I apologize if this isn't as technical as it needs to be — I am just a user that saw that something could improve.