Skip to content

feat: add rank0 async HF save logs#1845

Open
caixianzhang wants to merge 5 commits into
InternLM:mainfrom
caixianzhang:feat/async-hf-rank0-logs
Open

feat: add rank0 async HF save logs#1845
caixianzhang wants to merge 5 commits into
InternLM:mainfrom
caixianzhang:feat/async-hf-rank0-logs

Conversation

@caixianzhang
Copy link
Copy Markdown
Contributor

Summary

This PR adds lightweight async HF logging in the default path:

  • logs foreground async_save_hf launch cost through XTuner profile_time_and_memory
  • logs rank0 background async HF writer start/end timing
  • logs rank0 writer failure timing before preserving the existing status-file error path

It intentionally does not add per-rank diagnostic JSON, rank0 summary JSON, handle config propagation, or trainer-side diagnostic plumbing.

Validation

  • python3 -m py_compile xtuner/v1/engine/train_engine.py xtuner/v1/model/base.py xtuner/v1/model/compose/base.py
  • git diff --check

Note: local ruff command is not available in this environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant