Can zero++ be combined with cpu_offload?

[henryxiao1997]

I don't have too many cards in practices, so I have to leverage cpu_offload to train a larger scale of model. I wonder can I use zero++ together with cpu_offload at the same time?

Thanks!