[FEATURE REQUEST] - switch GPU to render headless

[rethink-studios]

I have a multi-GPU system and would love to offload rendering to GPU2:
On Facebook, I started a thread in hopes that someone there could help, however since this looks like the head of the snake, I'll submit here,

This lad is running a version of SD that uses dream.py. Inside that file, he tweaks a command to render on another GPU:
os.environ["CUDA_VISIBLE_DEVICES"] = "1"
I can not find dream.py for this generic CompVis build, so is there another file that I can tweak to designate which GPU to render to? I'm on Windows 10.

Is this what we're looking for??
editing txt2img.py and switch this line from:
device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu") model = model.to(device)
TO
device = torch.device("cuda:1,3" if torch.cuda.is_available() else "cpu") ## specify the GPU id's, GPU id's start from 0.
model = CreateModel()
model= nn.DataParallel(model,device_ids = [1, 3])
model.to(device)

Would love your thoughts on the subject.

Cheers!

[hlky]

--gpu is already an option
if the gpu id you want to use is 1 then you would use --gpu 1

it uses os.environ["CUDA_VISIBLE_DEVICES"] = str(opt.gpu)

if the gpu id you want to use is 1 then you could add
os.environ["CUDA_VISIBLE_DEVICES"] = "1" to the top of any other script too

explanation of how to set cli arguments

close if this solves your issue, thank you

[rethink-studios]

Apologies! That msg was to go to another Git, however after reading more about your build, I have switched over to your webui. Great stuff thus far!

When submitting a prompt, I'm adding --gpu (x) at the end, however I'm not seeing SD engage different GPUs.

Example prompt:
"The cat in the hat" --gpu 1

Am I missing something?

Is there a way to have the webGUI query the number of GPUs in the system, and add a check mark next to which GPU to render on?

[rethink-studios]

GPU 1 being my main display, 0 and 2 are used for rendering CG

[hlky]

--gpu (x) at the end, however I'm not seeing SD engage different GPUs.

It doesn't go in the prompt, it's a command line option

[rethink-studios]

Gotchu.
Inside of webUI, I switched
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID" # see issue #152
#os.environ["CUDA_VISIBLE_DEVICES"] = str(opt.gpu)

to

os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID" # see issue #152
os.environ["CUDA_VISIBLE_DEVICES"] = "0"

and after restarting the server, the webUI still rendered on device 1

[hlky]

webUI still rendered on device 1

You mean the models, correct? This option is to set which gpu the models load on, and is working for other people as far as I know, also, you should really have just set the cli option because opt.gpu is used in more than one place

Can you show me the output from nvidia-smi? just to confirm device id's are as you intend them to be

[hlky]

Probably easier to just duplicate relauncher.py and the .cmd, one for each, so you change relauncher0.py to use add --gpu 0, relauncher1 to add --gpu 1 etc
for .cmd file change relauncher.py to relauncher0.py etc

[rethink-studios]

You are correct. I duplicated webuildm, relauncher and webui. Works like a charm!
Thank you so much!

[hlky]

You might be interested in #264

[rethink-studios]

What's the absolute best way to download and update my local branch?

[hlky]

Probably best to just wait until someone else tests it then it will be merged to main repo when someone does, but in this case it's only webui.py that's changed so get it from that branch and copy it to your main repo folder, to clarify, to scripts in that main repo folder, for that you can either download the zip, just the one file, or checkout the branch. This is one of the reasons it's advised to not use the dev repo unless you know what you're doing because the dev repo's structure is set up to be easy for developers, the files are moved to the correct location on sync, but this isn't something I need to explain to developers.