Optimize llm/GPT3 performance

[MarioLulab]

PR types

Performance optimization

PR changes

Others

Description

将 model_zoo/gpt-3 中的优化策略迁移至 llm/gpt-3 中

• Fast Layer Norm op
  开启方法：运行 run_pretrain.py 时设置 --use_fast_layer_norm true

• Fused Linear
  开启方法：运行 run_pretrain.py 时设置 --use_fused_linear true

• Fused Dropout + Add Residual
  开启方法：运行 run_pretrain.py 时设置 --use_fused_dropout_add true

• enable_linear_fused_grad_add
  开启方法：版本要求：paddlenlp >= a5d87f5。运行 run_pretrain.py 时设置 --enable_linear_fused_grad_add true

• use SPInnerOverlap
  开启方法：版本要求：paddlenlp >= a092775, paddle >= cfaa001，运行 run_pretrain.py 时设置 --tensor_parallel_config enable_mp_async_allreduce enable_fused_linear_param_grad_add 并且训练开启 sp

• disable transmission attention_mask while pp
  开启方法：默认在 pp 时不传输 attention_mask

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 80.00000% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 55.16%. Comparing base (d577e19) to head (9a6696f).
Report is 9 commits behind head on develop.

Files	Patch %	Lines
paddlenlp/transformers/gpt/modeling.py	85.00%	3 Missing ⚠️
paddlenlp/transformers/gpt/modeling_pp.py	33.33%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8172      +/-   ##
===========================================
- Coverage    55.41%   55.16%   -0.26%     
===========================================
  Files          597      601       +4     
  Lines        91593    91780     +187     
===========================================
- Hits         50754    50628     -126     
- Misses       40839    41152     +313     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ZHUI]

LGTM