ValueError: RLHF do not support sequence parallel #3673
Description
When I set the --sequence_parallel_size parameter for grpo training, an ValueError occurs：
[rank0]: ValueError: RLHF do not support sequence parallel
[rank1]: Traceback (most recent call last):
[rank1]: File "/home/a/zmb_workspace/vscode_workspace/ms-swift/swift/cli/rlhf.py", line 5, in
[rank1]: rlhf_main()
[rank1]: File "/home/a/zmb_workspace/vscode_workspace/ms-swift/swift/llm/train/rlhf.py", line 98, in rlhf_main
[rank1]: return SwiftRLHF(args).main()
[rank1]: ^^^^^^^^^^^^^^^
[rank1]: File "/home/a/zmb_workspace/vscode_workspace/ms-swift/swift/llm/train/sft.py", line 28, in init
[rank1]: super().init(args)
[rank1]: File "/home/a/zmb_workspace/vscode_workspace/ms-swift/swift/llm/base.py", line 18, in init
[rank1]: self.args = self._parse_args(args)
[rank1]: ^^^^^^^^^^^^^^^^^^^^^^
[rank1]: File "/home/a/zmb_workspace/vscode_workspace/ms-swift/swift/llm/base.py", line 30, in _parse_args
[rank1]: args, remaining_argv = parse_args(self.args_class, args)
[rank1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]: File "/home/a/zmb_workspace/vscode_workspace/ms-swift/swift/utils/utils.py", line 147, in parse_args
[rank1]: args, remaining_args = parser.parse_args_into_dataclasses(argv, return_remaining_strings=True)
[rank1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]: File "/home/a/anaconda3/envs/swift/lib/python3.11/site-packages/transformers/hf_argparser.py", line 357, in parse_args_into_dataclasses
[rank1]: obj = dtype(**inputs)
[rank1]: ^^^^^^^^^^^^^^^
[rank1]: File "", line 367, in init
[rank1]: File "/home/a/zmb_workspace/vscode_workspace/ms-swift/swift/llm/argument/rlhf_args.py", line 112, in post_init
[rank1]: self._check_rlhf()
[rank1]: File "/home/a/zmb_workspace/vscode_workspace/ms-swift/swift/llm/argument/rlhf_args.py", line 193, in _check_rlhf
[rank1]: raise ValueError('RLHF do not support sequence parallel')
[rank1]: ValueError: RLHF do not support sequence parallel
[rank0]:[W326 11:57:08.683877236 ProcessGroupNCCL.cpp:1250] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present, but this warning has only been added since PyTorch 2.4 (function operator())