May I run any one of the example of deepseed on one 1080ti, P40 and P100

[SeekPoint]

I want try and learn deepspeed, but I only got 3 cards: 1080ti, P40 and P100

[minjiaz]

Hi @SeekPoint,

We have not fully tested DeepSpeed on the Pascal architecture, but since they are all NVidia GPUs, we expect it to work with DeepSpeed.

One example of a DeepSpeed application is DeepSpeed Chat, a ChatGPT-like model that we recently released. I recommend trying it out by following the instructions here:
https://github.com/microsoft/DeepSpeedExamples/tree/master/applications/DeepSpeed-Chat

Meanwhile, we are interested in your experience of running DeepSpeed on those devices, so if you run into any specific issues, we are happy to help.

Best,
Minjia

[zy-sunshine]

P40 and P100 do not have tensor kernel, so It will be slow to training. and 1080ti's memory is too small.
I do it with rtx3090 24GB, but the model of step3 is not well to use (I change the batch_size to 4)

[minjiaz]

@zy-sunshine, thank you for sharing your experience. You're right. P40 and P100 are going to be much slower because they lack TensorCore. Reducing the batch size will reduce the memory consumption but may or may not lead to model quality differences. We are interested in knowing if you can still get good generation results with reduced batch size.