qwen2-7b 开启 context parallel 后，logits中对应padding 的位置为 nan

[WallE-Chang]

启动命令如下:
torchrun --nproc_per_node 8 --nnodes 1 --node_rank 0 --master_addr localhost --master_port 24102 pretrain_qwen.py --save /mnt/project/Pai-Megatron-Patch/examples/qwen2/output/checkpoint/dsw-finetune-mcore-qwen2-7B-lr-1e-5-bs-1-seqlen-1024-pr-bf16-tp-2-pp-1-ac-sel-do-true-sp-false-tt--wt- --train-data-path /mnt/project/Pai-Megatron-Patch/examples/qwen2/dataset/alpaca_zh-qwen-valid.json --valid-data-path /mnt/project/Pai-Megatron-Patch/examples/qwen2/dataset/alpaca_zh-qwen-valid.json --test-data-path /mnt/project/Pai-Megatron-Patch/examples/qwen2/dataset/alpaca_zh-qwen-valid.json --dataloader-type cyclic --lr 1e-5 --min-lr 1e-6 --lr-decay-style cosine --weight-decay 0.1 --adam-beta1 0.9 --adam-beta2 0.95 --clip-grad 1.0 --init-method-std 0.008 --attention-dropout 0.0 --hidden-dropout 0.0 --lr-decay-iters 90 --lr-warmup-iters 10 --train-iters 100 --micro-batch-size 1 --global-batch-size 8 --num-layers 28 --hidden-size 3584 --num-attention-heads 28 --ffn-hidden-size 18944 --seq-length 1024 --max-position-embeddings 131072 --max-padding-length 1024 --log-interval 1 --eval-interval 10000 --eval-iters 10 --save-interval 100 --tensorboard-queue-size 1 --tensorboard-dir /mnt/project/Pai-Megatron-Patch/examples/qwen2/output/tensorboard/dsw-finetune-mcore-qwen2-7B-lr-1e-5-bs-1-seqlen-1024-pr-bf16-tp-2-pp-1-ac-sel-do-true-sp-false-tt--wt-_2024.06.13-09.52.15 --log-timers-to-tensorboard --log-batch-size-to-tensorboard --log-validation-ppl-to-tensorboard --tensor-model-parallel-size 2 --pipeline-model-parallel-size 1 --context-parallel-size 4 --no-load-optim --no-load-rng --num-workers 8 --extra-vocab-size 421 --patch-tokenizer-type LLamaTokenizer --dataset LLama-Pretrain-Raw --swiglu --normalization RMSNorm --norm-epsilon 1e-6 --use-rotary-position-embeddings --no-rope-fusion --position-embedding-type rope --untie-embeddings-and-output-weights --disable-bias-linear --add-qkv-bias --group-query-attention --num-query-groups 4 --rotary-percent 1.0 --rotary-base 1000000 --rotary-seq-len-interpolation-factor 1 --eod-mask-loss --bf16 --load /mnt/data/llm/qwen/Qwen2-7B-hf-to-mcore-te-tp2-pp1 --transformer-impl transformer_engine --recompute-activations --use-distributed-optimizer --use-flash-attn