Thank you for developing LM Studio and sharing it with us.
I'm using LMS on my MBP M2 Max with 96Gb memory and 1Tb storage. With so many flavors of LLMs to choose from, it doesn't take long to fill up all the space I have. I also have a large external SSD and even a larger NAS. But moving files manually isn't much fun. Most of the time I just delete stuff, but I also find myself re-downloading the same models to try out different configurations. I could use the external SSD as the only storage directory, but that's not only slower, I'd also lose access to LLMs when I'm on the road. I can imagine two similar solutions to this problem;
a) LM Studio can use multiple (or at least two) directories as caches to load the local models, but still save the newly downloaded models to the current cache location. In this case, the offloading of less used models to secondary storage is done by the user. So hopefully this will be easier to implement.
b) LM Studio allows users to define a 'cold storage' location and a maximum usage limit for the internal cache. So when the total size of the internal cache reaches its limit, LM Studio will automatically offload the least used models to the cold storage location. And when user requests to load a model from cold storage, it would be moved (or just copied) to internal cache directory.
If you know another solution to this issue that I can apply myself, I'd be grateful to hear it.
Best,
Thank you for developing LM Studio and sharing it with us.
I'm using LMS on my MBP M2 Max with 96Gb memory and 1Tb storage. With so many flavors of LLMs to choose from, it doesn't take long to fill up all the space I have. I also have a large external SSD and even a larger NAS. But moving files manually isn't much fun. Most of the time I just delete stuff, but I also find myself re-downloading the same models to try out different configurations. I could use the external SSD as the only storage directory, but that's not only slower, I'd also lose access to LLMs when I'm on the road. I can imagine two similar solutions to this problem;
a) LM Studio can use multiple (or at least two) directories as caches to load the local models, but still save the newly downloaded models to the current cache location. In this case, the offloading of less used models to secondary storage is done by the user. So hopefully this will be easier to implement.
b) LM Studio allows users to define a 'cold storage' location and a maximum usage limit for the internal cache. So when the total size of the internal cache reaches its limit, LM Studio will automatically offload the least used models to the cold storage location. And when user requests to load a model from cold storage, it would be moved (or just copied) to internal cache directory.
If you know another solution to this issue that I can apply myself, I'd be grateful to hear it.
Best,