Merge branch 'main' into 'docs' #463
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* update * update cfgs * update * fix bugs * upload docs * rename * update * Revert "update cfgs" This reverts commit 93966aa. * update cfgs * update * rename * rename * fix bc * fix stop_word * fix * fix * Update prompt_template.md
* fix bugs * Update mmbench.py
set target_modules
* support deepseek moe * update docs * update * update
Update pth_to_hf.py
* update examples * add examples * add json template config * rename * update * update * update
* add cfgs * add internlm2 template * add dispatch * add docs * update readme * update
* fix * Update utils.py
* accelerate cli * Update entry_point.py * Update entry_point.py --------- Co-authored-by: Zhihao Lin <36994684+LZHgrla@users.noreply.github.com>
* fix * update * Update README.md * Update README_zh-CN.md
* update * Update README.md * Update README.md * Update README.md * Update README_zh-CN.md * update * update * fix pre-commit * update
bump v0.1.12
Update version.py
* update results * update
Update internlm2 template
* add new loop * rename * fix pre-commit * add max_keep_ckpts * fix * update cfgs * update examples * fix * update * update llava * update * update * update * update
* support petrelfs * fix deepspeed save/load/resume * add ENV to toggle petrelfs * support hf save_pretrained * patch deepspeed engine
* support ddp mmbench evaluate * Update xtuner/tools/mmbench.py Co-authored-by: Zhihao Lin <36994684+LZHgrla@users.noreply.github.com> * Update xtuner/tools/mmbench.py Co-authored-by: Zhihao Lin <36994684+LZHgrla@users.noreply.github.com> * update minimum version of mmengine * Update runtime.txt --------- Co-authored-by: Zhihao Lin <36994684+LZHgrla@users.noreply.github.com>
Update utils.py
* add local_attn_args_to_messagehub_hook * add internlm repo sampler * add internlm repo dataset and collate_fn * dispatch internlm1 and internlm2 local attn * add internlm2 config * add internlm1 and intenrlm2 config * add internlm2 template * fix replace_internlm1_rote bugs * add internlm1 and internlm2 config templates * change priority of EvaluateChatHook * fix docs * fix config * fix bug * set rotary_base according the latest internlm2 config * add llama local attn * add llama local attn * update intern_repo_dataset docs when using aliyun * support using both hf load_dataset and intern_repo packed_dataset * add configs * add opencompass doc * update opencompass doc * use T data order * use T data order * add config * add a tool to get data order * support offline processing untokenized dataset * add docs * add doc about only saving model weights * add doc about only saving model weights * dispatch mistral * add mistral template * add mistral template * fix torch_dtype * reset pre-commit-config * fix config * fix internlm_7b_full_intern_repo_dataset_template * update local_attn to varlen_attn * rename local_attn * fix InternlmRepoSampler and train.py to support resume * modify Packer to support varlen attn * support varlen attn in default pipeline * update mmengine version requirement to 0.10.3 * Update ceph.md * delete intern_repo_collate_fn * delete intern_repo_collate_fn * delete useless files * assert pack_to_max_length=True if use_varlen_attn=True * add varlen attn doc * add varlen attn to configs * delete useless codes * update * update * update configs * fix priority of ThroughputHook and flake8 ignore W504 * using map_fn to set length attr to dataset * support split=None in process_hf_dataset * add dataset_format_mapping * support preprocess ftdp and normal dataset * refactor process_hf_dataset * support pack dataset in process_untokenized_datasets * add xtuner_dataset_timeout * using gloo backend for monitored barrier * set gloo timeout * fix bugs * fix configs * refactor intern repo dataset docs * fix doc * fix lint --------- Co-authored-by: pppppM <67539920+pppppM@users.noreply.github.com> Co-authored-by: pppppM <gjf_mail@126.com>
…` and rename 'internlm_repo' to 'intern_repo' (InternLM#372) * fix * rename internlm_repo to intern_repo * add InternlmRepoSampler for preventing bc break * add how to install flash_attn to doc
…nLM#379) * delete useless codes * refactor process_untokenized_datasets: add ftdp to dataset-format * fix lint
…ernLM#381) support flash attn 2 in internlm1, internlm2 and llama
…#385) * support saving eval output before save checkpoint * refactor
* fix lr scheduler setting * fix more --------- Co-authored-by: zilong.guo <zilong.guo@zeron.ai> Co-authored-by: LZHgrla <linzhihao@pjlab.org.cn>
* rename * update docs * update template * update * add cfgs * update * update
…ternLM#404) * [Fix] Fix no space in chat output using InternLM2. (InternLM#357) * Update chat.py * Update utils.py * Update utils.py * fix pre-commit --------- Co-authored-by: Zhihao Lin <36994684+LZHgrla@users.noreply.github.com> Co-authored-by: LZHgrla <linzhihao@pjlab.org.cn>
…NEL environment variable (InternLM#411) * dispatch support transformers>=4.36 * add USE_TRITON_KERNEL environment variable * raise RuntimeError use triton kernels on cpu * fix lint
* [Feature]Add InternLM2-Chat-1_8b full config * [Feature]Add InternLM2-Chat-1_8b full config * update --------- Co-authored-by: LZHgrla <linzhihao@pjlab.org.cn> Co-authored-by: Zhihao Lin <36994684+LZHgrla@users.noreply.github.com>
* added gemma config and template * check config and make sure the consistancy * Update xtuner/configs/gemma/gemma_2b_base/gemma_2b_base_qlora_alpaca_e3.py Co-authored-by: Zhihao Lin <36994684+LZHgrla@users.noreply.github.com> * Update xtuner/configs/gemma/gemma_2b_base/gemma_2b_base_full_alpaca_e3.py Co-authored-by: Zhihao Lin <36994684+LZHgrla@users.noreply.github.com> * Update xtuner/configs/gemma/gemma_7b_base/gemma_7b_base_full_alpaca_e3.py Co-authored-by: Zhihao Lin <36994684+LZHgrla@users.noreply.github.com> * Update xtuner/configs/gemma/gemma_7b_base/gemma_7b_base_qlora_alpaca_e3.py Co-authored-by: Zhihao Lin <36994684+LZHgrla@users.noreply.github.com> * Update xtuner/utils/templates.py Co-authored-by: Zhihao Lin <36994684+LZHgrla@users.noreply.github.com> * update * added required version * update * update --------- Co-authored-by: Zhihao Lin <36994684+LZHgrla@users.noreply.github.com> Co-authored-by: LZHgrla <linzhihao@pjlab.org.cn>
* add base dataset * update dataset generation * update refcoco * add convert refcooc * add eval_refcoco * add config * update dataset * fix bug * fix bug * update data prepare * fix error * refactor eval_refcoco * fix bug * fix error * update readme * add entry_point * update config * update config * update entry point * update * update doc * update --------- Co-authored-by: jacky <jacky@xx.com>
* Update version.py * Update version.py
fix bugs when using epochbasedrunner
…nternLM#410) * support smart_tokenizer_and_embedding_resize * replace ast with json.loads * support list_dataset_format cli * add doc about ftdp and custom dataset * add custom dataset template * add args name to process_hf_dataset * use new process_untokenized_datasets * support tokenize_ftdp_datasets * add mistral_7b_w_tokenized_dataset config * update doc * update doc * add comments * fix data save path * smart_tokenizer_and_embedding_resize support zero3 * fix lint * add data format to internlm2_7b_full_finetune_custom_dataset_e1.py * add a data format example to configs associated with finetuning custom dataset * add a data format example to configs associated with finetuning custom dataset * fix lint
* split finetune_custom_dataset.md to 6 parts * refactor custom_dataset and ftdp_dataset related docs * fix comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.