Skip to content

[Improve] Redesign convert tools#96

Merged
pppppM merged 25 commits intoInternLM:mainfrom
LZHgrla:lzh/refactor_tools
Sep 5, 2023
Merged

[Improve] Redesign convert tools#96
pppppM merged 25 commits intoInternLM:mainfrom
LZHgrla:lzh/refactor_tools

Conversation

@LZHgrla
Copy link
Copy Markdown
Contributor

@LZHgrla LZHgrla commented Sep 5, 2023

To support the conversion of checkpoint saved by DeepSpeed, this PR redesigns the convert tools.

  1. Training: xtuner train ${CONFIG_NAME_OR_PATH}
  2. Convert: xtuner convert pth_to_hf ${CONFIG_NAME_OR_PATH} ${PTH} ${SAVE_PATH}
  3. Chat: xtuner chat ${LLM} --adapter ${ADAPTER}

@pppppM
Copy link
Copy Markdown
Collaborator

pppppM commented Sep 5, 2023

After bump version to v0.1.2, HuggingFace model cards need to update.

@LZHgrla LZHgrla requested a review from pppppM September 5, 2023 07:30
@pppppM pppppM merged commit 6e44d6b into InternLM:main Sep 5, 2023
@LZHgrla LZHgrla deleted the lzh/refactor_tools branch September 5, 2023 08:10
@LZHgrla LZHgrla restored the lzh/refactor_tools branch September 5, 2023 08:10
llkn-2 pushed a commit to llkn-2/xtuner that referenced this pull request Jul 31, 2024
* refactor tools

* modify entry_point

* modify docs

* update docs

* fix

* fix

* Update README.md

* Update README.md

* Update README.md

* Update README_zh-CN.md

* fix pre-commit

* rename converter

* update pth2hf

* rename pth2hf to pth_to_hf

* add fp32 for pth_to_hf

* Update README.md

* Update README_zh-CN.md

* Update README_zh-CN.md

* Update README.md

* Update README_zh-CN.md

* Update README_zh-CN.md

* Update README.md

* Update README.md

* Update README_zh-CN.md

* fix pre-commit
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants