Support Ulysses in Swift

[zhoumengbo]

Describe the feature
Support Ulysses in Swift

Paste any useful information
In ms-swift, RLHF does not support SP. I saw that the Verl framework supports Ulysses for long context training. I want to know whether Ulysses is supported in Swift? Now I need to train GRPO on a long context. I hope ms-swift can support using ulysses from deepspeed to help me with training.

Reference:
https://github.com/deepspeedai/DeepSpeed/tree/master/blogs/deepspeed-ulysses
https://verl.readthedocs.io/en/latest/perf/perf_tuning.html#ulysses-sequence-parallel-for-long-context-training

[wind91725]

same issue

[stormchasingg]

same issue

[tastelikefeet]

Ulysses+Ring-Attention has been supported inPT/SFT/DPO/GRPO
Try the main branch with --sequence_parallel_size N

[tastelikefeet]

check here:
https://huggingface.co/blog/exploding-gradients/ulysses-ring-attention

[stormchasingg]

@tastelikefeet
What's the difference between --context_parallel_size and --sequence_parallel_size?

[tastelikefeet]

context_parallel_size used in megatron training
sequence_parallel_size used in transformers training

[stormchasingg]

@tastelikefeet
both megatron swift and swift support ulysess sp + ring attention?

[tastelikefeet]

@tastelikefeet both megatron swift and swift support ulysess sp + ring attention?

No... Megatron cannot use ulysses.
Megatron supports cp(ring-attention), sp(non-attention sequence parallel), tp, pp, ep
sp has already sharded activations to parts, this conflicts with ulysses, so megatron does not need ulysses.
If you are using megatron, check documentations here:
https://swift.readthedocs.io/en/latest/Megatron-SWIFT/Command-line-parameters.html