Jan
Manually unload a model #9433
|
Just wondering if there is a way to manually unload a model to free RAM. I'm using Jan and a model to help tweak AI generation prompts. I then try the prompt in ComfyUI and go back to Jan to adjust more if needed. I have an RTX 3060 12GB so VRAM is at a premium, hence was wondering if there is a way to unload the model from RAM to free it up before running the generation. |
Replies: 2 comments 1 reply
|
Yes, there is a per-model Stop button. Open Settings > Model Providers and pick the llama.cpp provider: each model in the list shows a Start button while it is idle and a red Stop button while it is loaded. In the Jan 0.8.7 source the button calls The same provider's settings also have "Max Concurrently Loaded Models" (default 1, 0 = unlimited), which limits how many models the router keeps loaded at once. Older builds had an auto-unload option, and the extension migrates it into this setting. |
|
If the Stop button does what you need, could you mark the answer as accepted so it is easy for others to find? |
|
Thank you. I was looking in the chat window up where the model is listed and didn't think to look in Settings. |
Yes, there is a per-model Stop button. Open Settings > Model Providers and pick the llama.cpp provider: each model in the list shows a Start button while it is idle and a red Stop button while it is loaded. In the Jan 0.8.7 source the button calls
handleStopModel, which goes through the models servicestopModelto the engine'sunloadfor that model. So you can free the memory before switching to ComfyUI and press Start again when you come back to Jan. The button is only shown for the local llama.cpp and MLX providers.The same provider's settings also have "Max Concurrently Loaded Models" (default 1, 0 = unlimited), which limits how many models the router keeps loaded at once. Older build…