[Feature Request] ControlNet lowvram

[jn-jairo]

The Mikubill/sd-webui-controlnet has a nice lowvram optimization, I can run it on stable-diffusion-webui in my 2GB GPU, but I cannot run the ControlNet from here because I get an out of memory error.

I tried to understand how it works to make that optimization in this repo, but I guess I'm not smart enough for that, I know it keeps things on cpu and load to gpu only when needed and send back to cpu after used, but I couldn't figure out how to make it in this repo.

It would be nice if someone smart like you could add this optimization here 😁

[teldon]

Having a lowvram mode for controlnet would be amazing as I am about 250MB short at worst when trying to run controlnet.

[jn-jairo]

I tried the t2i-adapter with the models from https://huggingface.co/webui/ControlNet-modules-safetensors and it works on my GPU, so I'll use t2i-adapter from now on, it looks like the better option for my lack of vram.

[comfyanonymous]

I fixed some controlnet vram related issue a few days ago so your OOM error might be fixed.