v1.5.0

English Version

New features:

1. Support multi-line inference
2. Support multi node training
3. Add benchmarks
4. Support UI training, start by swift web-ui
5. Support VLLM inference
6. Support RLHF(DPO) training

New tuners:

1. SCEdit, adopted by TongYi Lab, uses lesser memory but produces better performance than LoRA, and can be used to replace ControlNet in a series of scenarios like Pose control/In-Painting/Out-Paining/Label-removing, etc.

New models:

1. SUS series models
2. Mixtral-MoE series models
3. deepseek series models
4. phi2-3b
5. cogagent-chat/cogagent-vqa
6. codegeex2-6b

New datasets:

Datasets used in RLHF:

1. hh-rlhf
2. stack-exchange-paired

中文版

SWIFT本月新版本已经发布！

新特性:

1. 支持多行输入推理
2. 支持多卡训练
3. 添加了模型训练的benchmarks
4. 支持界面训练和推理，通过swift web-ui开启
5. 支持VLLM推理
6. 支持RLHF(DPO)训练

新tuners:

SCEdit: 通义实验室自研的优秀U-Net微调框架，显存占用远小于LoRA，效果较LoRA更好，且可以替代实现ControlNet的效果，实现In-Painting/Out-Paining/标签去除/Pose控制等能力

新模型：

SUS系列模型
Mixtral-MoE系列模型
deepseek系列模型
phi2-3b
cogagent-chat/cogagent-vqa
codegeex2-6b

新数据集:

用于RLHF的数据集：
hh-rlhf
stack-exchange-paired

What's Changed

• update multi-line input (infer) by @Jintao-Huang in #196
• Fix model saving in new format by @tastelikefeet in #198
• Fix compatible error by @tastelikefeet in #201
• Fix bug 1206 by @Jintao-Huang in #202
• fix fp16 & full bug by @Jintao-Huang in #203
• Fix qwen-audio inference bug by @Jintao-Huang in #204
• Support multi node by @Jintao-Huang in #205
• fix typo bug by @Jintao-Huang in #206
• Support sus by @Jintao-Huang in #207
• Support cpu by @Jintao-Huang in #208
• Add Feat: Freeze Parameters, disable_tqdm by @Jintao-Huang in #210
• update dataset by @Jintao-Huang in #212
• Support lazy_tokenize, preprocess_num_proc by @Jintao-Huang in #211
• Support Mixtral MoE by @tastelikefeet in #217
• Add benchmark by @Jintao-Huang in #213
• support ui training by @tastelikefeet in #219
• Fix transformers 4.36 by @Jintao-Huang in #218
• Update mixtral-7b-moe by @Jintao-Huang in #221
• Compatible with peft>=0.7.0 by @tastelikefeet in #220
• fix dtype='fp16' sft bug by @Jintao-Huang in #227
• fix generation_config warning by @Jintao-Huang in #224
• Fix merge_lora & model_cache_dir bug by @Jintao-Huang in #229
• fix lazy_tokenize bug by @Jintao-Huang in #228
• Add inference UI and refactor machenism by @tastelikefeet in #230
• Support deepseek by @Jintao-Huang in #223
• relax version restriction by @tastelikefeet in #232
• fix bug 1218 by @Jintao-Huang in #235
• support deployment by @Jintao-Huang in #231
• update docs by @Jintao-Huang in #238
• Refactor some code by @tastelikefeet in #237
• fix typo bug by @Jintao-Huang in #239
• update readme & phi2-3b by @Jintao-Huang in #241
• Fix argument 1220 by @Jintao-Huang in #242
• Support CogAgent by @tastelikefeet in #243
• fix infer by @tastelikefeet in #244
• Support more peft tuners by @tastelikefeet in #245
• Fix copying additional files by @tastelikefeet in #247
• Add sft for codegeex2 by @tastelikefeet in #248
• fix issue #249 by @tastelikefeet in #250
• Feat/scedit by @jiangzeyinzi in #253
• Update 1228 by @Jintao-Huang in #254
• fix unicode error by @tastelikefeet in #259
• Update readme for SCEdit by @tastelikefeet in #258
• DPO by @tastelikefeet in #255
• update self-cognition by @Jintao-Huang in #261
• Fix/1229 by @tastelikefeet in #260
• fix trainer init by @tastelikefeet in #262
• fix bugs by @tastelikefeet in #263
• fix import by @tastelikefeet in #265
• Fix import by @tastelikefeet in #266
• update perf by @Jintao-Huang in #264
• fix bug by @tastelikefeet in #267
• Support win32 by @tastelikefeet in #268

Full Changelog: v1.4.0...v1.5.0