[Unified Checkpoint] update moe

[DesmonDay]

Before submitting

• Lint code. If there are lint issues, please format the code first.

# Install and register `pre-commit` in the project folder
pip install pre-commit && pre-commit install

# Process previous code files separately
pre-commit run --file XXXX.py

• Add test cases into tests folder. If there are codecov issues, please add tests cases first.

PR types

Function optimization

PR changes

Others

Description

Update moe for unified checkpoint.

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 3.84615% with 25 lines in your changes missing coverage. Please review.

Project coverage is 49.92%. Comparing base (f607bad) to head (b2c28b2).
Report is 16 commits behind head on develop.

❗ Current head b2c28b2 differs from pull request most recent head 1df9059

Please upload reports for the commit 1df9059 to get more accurate results.

Files with missing lines	Patch %	Lines
paddlenlp/trainer/unified_checkpoint/utils.py	3.84%	25 Missing ⚠️

❌ Your patch status has failed because the patch coverage (3.84%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.
❌ Your project status has failed because the head coverage (49.92%) is below the target coverage (58.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop   #10282      +/-   ##
===========================================
- Coverage    49.93%   49.92%   -0.01%     
===========================================
  Files          761      761              
  Lines       124178   124190      +12     
===========================================
  Hits         62005    62005              
- Misses       62173    62185      +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ZHUI]

LGTM

[ZHUI]

最好可以save之后再 offload。这样节省一次开销。

[DesmonDay]

嗯嗯，代码后面那里有写