v1.4.0

English Version

New features:

1. Support for self-awareness fine-tuning.
2. Support for fine-tuning and inference of the AnimateDiff model in the AIGC direction.
3. Support for flash attention in more models: qwen series, qwen-vl series, llama series, openbuddy series, mistral series, yi series, ziya series, using the use_flash_attn parameter.
4. Support for multiple loras to take effect simultaneously.

New tuners:

1. NEFTune
2. ROME supports more models: chatglm

New models:

1. AnimateDiff
2. zephyr-7b-beta-chat, openbuddy-zephyr-7b-chat
3. qwen-1_8b, qwen-1_8b-chat, qwen-1_8b-chat-int4, qwen-1_8b-chat-int8
4. qwen-72b, qwen-72b-chat, qwen-72b-chat-int4, qwen-72b-chat-int8
5. qwen-audio, qwen-audio-chat
6. yi-34b-chat, codefuse-codellama-34b-chat
7. tongyi-finance-14b, tongyi-finance-14b-chat, tongyi-finance-14b-chat-int4
8. bluelm-7b, bluelm-7b-chat, bluelm-7b-32k, bluelm-7b-chat-32k

New datasets:

1. hc3-zh, hc3-en
2. codefuse-python-en, codefuse-eval-instruction-zh
3. aishell1-zh, aishell1-mini-zh

中文版

新特性:

1. 支持自我认知微调.
2. 支持AIGC方向的AnimateDiff模型的微调与推理.
3. 支持更多模型的flash attention: qwen series, qwen-vl series, llama series, openbuddy series, mistral series, yi series, ziya series. 使用use_flash_attn参数.
4. 支持多个lora同时生效

新tuners:

1. NEFTune
2. ROME支持更多模型: chatglm

新模型:

1. AnimateDiff
2. zephyr-7b-beta-chat, openbuddy-zephyr-7b-chat
3. qwen-1_8b, qwen-1_8b-chat, qwen-1_8b-chat-int4, qwen-1_8b-chat-int8
4. qwen-72b, qwen-72b-chat, qwen-72b-chat-int4, qwen-72b-chat-int8
5. qwen-audio, qwen-audio-chat
6. yi-34b-chat, codefuse-codellama-34b-chat
7. tongyi-finance-14b, tongyi-finance-14b-chat, tongyi-finance-14b-chat-int4
8. bluelm-7b, bluelm-7b-chat, bluelm-7b-32k, bluelm-7b-chat-32k

新数据集:

1. hc3-zh, hc3-en
2. codefuse-python-en, codefuse-eval-instruction-zh
3. aishell1-zh, aishell1-mini-zh

What's Changed

• Support Yi-6b sft by @tastelikefeet in #134
• fix CLI by @tastelikefeet in #135
• update readme by @tastelikefeet in #137
• Support xverse 65b sft by @tastelikefeet in #138
• Support bluelm by @Jintao-Huang in #140
• fix doc by @tastelikefeet in #143
• Add neftune by @tastelikefeet in #145
• Update sh by @Jintao-Huang in #144
• Add compatibility test and fix some problems with peft>=0.6.0 by @tastelikefeet in #146
• fix compatible with transformers>=4.35 by @Jintao-Huang in #148
• Update sh 1115 by @Jintao-Huang in #150
• Update doc by @tastelikefeet in #151
• support flash_attn by @Jintao-Huang in #152
• Fix bug: not work on peft<=0.5.0 by @tastelikefeet in #155
• fix register model bug by @Jintao-Huang in #154
• Support tongyi finance 14b by @Jintao-Huang in #157
• add check_model args and fix check_dataset by @Jintao-Huang in #159
• fix load_from_ckpt_dir bug by @Jintao-Huang in #161
• Update arguments by @Jintao-Huang in #162
• new feature: save_infer_result_to_jsonl by @Jintao-Huang in #163
• Feat 1121 by @Jintao-Huang in #165
• update readme and fix bug by @Jintao-Huang in #167
• Add cli merge lora by @Jintao-Huang in #168
• update code by @Jintao-Huang in #169
• support yi-34b-chat by @Jintao-Huang in #164
• Add animate diff by @tastelikefeet in #174
• update readme by @Jintao-Huang in #175
• Refine LoRA to peft by @tastelikefeet in #176
• support qwem-72b qwen-1_8b qwen-audio by @Jintao-Huang in #180
• Update wechat by @Jintao-Huang in #186
• Fix the slow inference speed bug in qwen AutoGPTQ by @Jintao-Huang in #187
• Support self cognition by @Jintao-Huang in #188
• update dataset model by @Jintao-Huang in #190

Full Changelog: v1.3.0...v1.4.0