Running on Multiple GPU?

[Joematpal]

I have two gtx 1070s, non-sli. Has anyone experienced running the aidungeon on two gpus? What were some of the issues or challenges? The program suggests a beefy amount of gpu ram. Would a configuration with two gpus work well?

[ben-bay]

That's a good point. Definitely something we should support.

[Joematpal]

Should as in currently do support? Or as in we should get to supporting it?

[WAUthethird]

Should as in currently do support? Or as in we should get to supporting it?

I would assume he means that it’ll be supported later. There is no multi-GPU support at the moment.

[Akababa]

I don't know if GPT-2 can be split across GPUs. Probably the best bet is to get quantization working or train a smaller model that uses less VRAM.

[bryanlyon]

Multi GPU cannot split a model like that without a LOT of work, instead multiGPU would be used to handle multiple inputs at once (not something that this project would benefit from).

Essentially using model-parallelism is much more difficult to implement than data-parallelism and neither would work terribly well with GPT-2 without some significant changes.