Training on Cluster of CPUs

[kanuarj]

I experienced quite less training speed with GPU in Synthesizer phase of the model even though VRAM consumption was on point. I was wondering if there is any way to train on a cluster of CPUs which are not being utilized instead of GPUs.

[ghost]

This is not something that we support or plan to implement, but you may find this helpful: pytorch/pytorch#2733

[kanuarj]

@blue-fish Thanks