step3 rlhf traning: RuntimeError: torch.cat(): expected a non-empty list of Tensors

[JackHenry1992]

Train RLHF- phase， encountered error：

Traceback (most recent call last):
  File "applications/DeepSpeed-Chat/training/step3_rlhf_finetuning/main.py", line 586, in <module>
    main()
  File "applications/DeepSpeed-Chat/training/step3_rlhf_finetuning/main.py", line 469, in main
    out = trainer.generate_experience(batch_prompt['prompt'],
  File "/applications/DeepSpeed-Chat/training/step3_rlhf_finetuning/ppo_trainer.py", line 119, in generate_experience
    seq = self._generate_sequence(prompts, mask, step)
  File "applications/DeepSpeed-Chat/training/step3_rlhf_finetuning/ppo_trainer.py", line 113, in _generate_sequence
    out_seq = torch.cat(out_seq, dim=0)  # concate output in the batch dim
RuntimeError: torch.cat(): expected a non-empty list of Tensors

Using A100 cuda=11.8, torch=2.0.0， python=3.10.12, deepspeed=0.10.0, transformers=4.30.2
training script config:

ACTOR_ZERO_STAGE=2
CRITIC_ZERO_STAGE=3
ACTOR_MODEL_PATH=../step1_supervised_finetuning/output_opt6.7b
CRITIC_MODEL_PATH=../step2_reward_model_finetuning/output_opt125m

[iamsile]

it seems out_seq is None, you can check out_seq

[koking0]

@JackHenry1992 me too, how to solve this error?