[BUG]not a bug, ask for solution：ppo zero3+offload generate too slow 

Hi, when I ran ppo with llama-13b (prompt_len=256, answer_len = 512) with zero stage 3 and offload (8*V100-32G), it seems the generation time is too slow (average about 20s-300s). Currently, I consider the reason of generating slowly is using zero stage 3 with open synced_gpus, but it consumes a lot of resource if I do not use this. Is there any solution for solving this problem? Thanks a lot!