Which version of LM Studio?
LM Studio 0.3.15 build 11
Which operating system?
Windows 11 pro
What is the bug?
I think I have found a bug in lm studio 0.3.15 build 11: if I use it as inference server and I enable just in time model loading and auto unload unused jit loaded models , after ttl expire and I make a new request, I see the same model loaded again but the ram occupied is doubled. In the developer section I see two llm with the same name loaded , with the possibility to eject also the old one
LLM used for test was mistral 7B instructed.
This is not what I expect because i was using the auto unload to be able to load different llm for different task with one llm inference server and being able to minimize the vram needed in the gpu
To Reproduce
Steps to reproduce the behavior:
- enable server - enable just in time model loading - enable auto unload unused jit loaded models
- use any type of client with openai compatible api , select the correct model and ask a question
- in the lm studio log see the model loaded and wait for the reply
- wait the TTL expire (default 60 minutes) and do another request
- see in the lm studio log the model loading again but in the upper part of the screen you will see the same llm model loaded twice with the eject button on both and the ram used is double
Which version of LM Studio?
LM Studio 0.3.15 build 11
Which operating system?
Windows 11 pro
What is the bug?
I think I have found a bug in lm studio 0.3.15 build 11: if I use it as inference server and I enable just in time model loading and auto unload unused jit loaded models , after ttl expire and I make a new request, I see the same model loaded again but the ram occupied is doubled. In the developer section I see two llm with the same name loaded , with the possibility to eject also the old one
LLM used for test was mistral 7B instructed.
This is not what I expect because i was using the auto unload to be able to load different llm for different task with one llm inference server and being able to minimize the vram needed in the gpu
To Reproduce
Steps to reproduce the behavior: