Models aren't unloaded from GPU memory (windows)

[cchance27]

I was testing i loaded a 2B model then wanted to change the system prompt, so i changed it and hit start, and the new gen was much slower, when i checked i managed to verify the issue, it seems that the app is not unloading the GPU memory before loading the new model so it's getting loaded into shared memory instead of the native gpu memory.

[clarkmcc]

Interesting, I haven't noticed this on metal yet so I'm wondering if this is a rustformers issue. I'll do some more testing to pin it down.