Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,15 +35,15 @@ Key features:
2. Tuners provided by SWIFT can be combined together to allow exploration of multiple tuners on a model for best result.
3. Support calling `activate_adapter` or `deactivate_adapter` or `set_active_adapters` to activate/deactivate tuners. User can inference with one model and multiple tuners in different threads independently.
4. Support training and inference with scripts/CLI,meanwhile support inference with Web-UI.
5. Support model deployment(vllm/chatglm.cpp/xinference),Check [Official documentation](./docs/source/GetStarted/Deployment.md) for details.
5. Support model deployment(vllm/chatglm.cpp/xinference),Check [Official documentation](./docs/source/GetStarted/部署指南.md) for details.

Users can check the [documentation of Swift](docs/source/GetStarted/Introduction.md) to get detail tutorials.
Users can check the [documentation of SWIFT](docs/source/GetStarted/快速使用.md) to get detail tutorials.


### 🎉 News
- 🔥 2023.11.11: **NEFTune** Supported, Use is with `Swift.prepare_model(model, NEFTuneConfig())`
- 🔥 2023.11.11: Support training and inference with **CLI**, and inference with **Web-UI**. Check the [Run using Swift CLI](https://github.com/modelscope/swift/tree/main#run-using-swift-cli) chapter for details.
- 🔥 2023.11.11: Support model **deployment**(vllm/chatglm.cpp/xinference),Check [Official documentation](./docs/source/GetStarted/Deployment.md) for details.
- 🔥 2023.11.11: Support model **deployment**(vllm/chatglm.cpp/xinference),Check [Official documentation](./docs/source/GetStarted/部署指南.md) for details.
- 🔥 2023.11.10: Support for **bluelm** series models: bluelm-7b, bluelm-7b-chat, bluelm-7b-32k, bluelm-7b-chat-32k. The corresponding shell script can be found in [bluelm_7b_chat](https://github.com/modelscope/swift/tree/main/examples/pytorch/llm/scripts/bluelm_7b_chat).
- 🔥 2023.11.08: Support the finetuning of **xverse-65b** model, scripts can be found at: [xverse_65b](https://github.com/modelscope/swift/tree/main/examples/pytorch/llm/scripts/xverse_65b).
- 🔥 2023.11.07: Support the finetuning of **yi-6b**, **yi-34b** model, scripts can be found at: [yi_6b](https://github.com/modelscope/swift/tree/main/examples/pytorch/llm/scripts/yi_6b), [yi_34b](https://github.com/modelscope/swift/tree/main/examples/pytorch/llm/scripts/yi_34b).
Expand Down
6 changes: 3 additions & 3 deletions README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,15 +33,15 @@ SWIFT(Scalable lightWeight Infrastructure for Fine-Tuning)是一个可扩展
2. 在单次训练或推理中可以使用多个tuners
3. 支持调用`activate_adapter`或`deactivate_adapter`或`set_active_adapters`来使部分tuner激活或失活,用户可以在推理时同时加载多个独立的tuners在不同线程中并行使用。
4. 支持通过脚本方式和命令行方式开启训练和推理,同时支持Web-UI方式进行推理。
5. 支持模型训练后的部署链路(vllm/chatglm.cpp/xinference),详情可以查看[官方文档](./docs/source/GetStarted/Deployment.md)。
5. 支持模型训练后的部署链路(vllm/chatglm.cpp/xinference),详情可以查看[官方文档](./docs/source/GetStarted/部署指南.md)。

用户可以查看 [Swift官方文档](docs/source/GetStarted/Introduction.md) 来了解详细信息。
用户可以查看 [SWIFT官方文档](docs/source/GetStarted/快速使用.md) 来了解详细信息。


## 🎉 新闻
- 🔥 2023.11.11: 支持**NEFTune**, 使用`Swift.prepare_model(model, NEFTuneConfig())`即可开启.
- 🔥 2023.11.11: 支持**命令行**训练推理和**Web-UI**推理, 详情可以查看下方的`使用Swift CLI运行`章节.
- 🔥 2023.11.11: 支持模型训练后的**部署**链路(vllm/chatglm.cpp/xinference),详情可以查看[官方文档](./docs/source/GetStarted/Deployment.md).
- 🔥 2023.11.11: 支持模型训练后的**部署**链路(vllm/chatglm.cpp/xinference),详情可以查看[官方文档](./docs/source/GetStarted/部署指南.md).
- 🔥 2023.11.10: 支持**bluelm**系列模型: bluelm-7b, bluelm-7b-chat, bluelm-7b-32k, bluelm-7b-chat-32k. 对应的sh脚本可以查看[bluelm_7b_chat](https://github.com/modelscope/swift/tree/main/examples/pytorch/llm/scripts/bluelm_7b_chat).
- 🔥 2023.11.08: 支持**xverse-65b**模型的训练和推理流程,脚本在[xverse_65b](https://github.com/modelscope/swift/tree/main/examples/pytorch/llm/scripts/xverse_65b).
- 🔥 2023.11.07: 支持**yi-6b**, **yi-34b**模型的训练和推理流程,脚本在[yi_6b](https://github.com/modelscope/swift/tree/main/examples/pytorch/llm/scripts/yi_6b), [yi_34b](https://github.com/modelscope/swift/tree/main/examples/pytorch/llm/scripts/yi_34b).
Expand Down
19 changes: 11 additions & 8 deletions docs/source/GetStarted/ResTuning.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Res-Tuning组件

<div align="center">

## [NeurIPS 2023] Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone
Expand All @@ -6,23 +8,24 @@

</div>

Res-Tuning is a flexible and efficient tuning paradigm. We manage to free the design of tuners from the network architecture, facilitating flexible combination of various tuning strategies and further extend a memory-efficient bypass variant, which significantly reduces the memory consumption and multi-task inference cost.
Res-Tuning 是一种灵活高效的微调tuner。我们把tuner的设计从模型网络结构中解耦出来以便灵活地组合,
并进一步扩展实现了一种新的节省内存的旁路tuner,大大减少了显存消耗和多任务推理成本。

The implementation is a pluggable tuner component for [SWIFT](https://github.com/modelscope/swift), designed to be user-friendly.
目前Res-Tuning在[SWIFT](https://github.com/modelscope/swift)中以可插拔的tuner算法组件提供,开发者可以直接使用它。

### Catalog
### 支持的组件列表

- [x] Res-Adapter
- [x] Res-Tuning-Bypass
- [ ] Res-Prefix
- [ ] Res-Prompt

### Usage
### 使用方式

#### Demo
- Run our interactive demo using [vision_example](https://github.com/modelscope/swift/blob/main/examples/pytorch/cv/notebook/swift_vision.ipynb).
- 可以使用我们提供的 [可视化例子](https://github.com/modelscope/swift/blob/main/examples/pytorch/cv/notebook/swift_vision.ipynb).

#### Init Tuner
#### 初始化Tuner

```Python
from swift import ResTuningConfig
Expand All @@ -40,7 +43,7 @@ config = ResTuningConfig(
- target_modules: The target module to be replaced.
- tuner_cfg: The configuration of the tuning module.

#### Load Model
#### 加载模型

```Python
from swift import Swift
Expand All @@ -52,7 +55,7 @@ print(model(torch.ones(1, 3, 224, 224)).shape)
```


### Citation
### 引用
```
@inproceedings{jiang2023restuning,
title={Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
可以使用pip进行安装:

```shell
pip install ms-swift
pip install ms-swift -U
```

## 源代码安装
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Swift API
# 使用Tuners

## 在训练中使用Swift

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,31 @@
为了支持习惯Peft的用户,Swift提供了对于Peft的兼容性。用户可以从swift中import peft组件:

>PeftModel
>
>PeftConfig
>
>PeftModelForSeq2SeqLM
>
>PeftModelForSequenceClassification
>
>PeftModelForTokenClassification
>
>PeftModelForCausalLM
>
>PromptEncoderConfig
>
>PromptTuningConfig
>
>PrefixTuningConfig
>
>PromptLearningConfig
>
>LoraConfig
>
>get_peft_config
>
>get_peft_model_state_dict
>
>get_peft_model

以上组件均可以从swift中import:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,27 +4,26 @@ Swift是一个提供LLM模型轻量级训练和推理的开源框架。Swift提

总之,在这个框架中提供了以下特性:

- **具备SOTA特性的Efficient Tuners**:用于结合大模型实现轻量级(在商业级显卡上)训练和推理,并取得较好效果
- **具备SOTA特性的Efficient Tuners**:用于结合大模型实现轻量级(在商业级显卡上,如RTX3080、RTX3090、RTX4090等)训练和推理,并取得较好效果
- **使用ModelScope Hub的Trainer**:基于`transformers trainer`提供,支持LLM模型的训练,并支持将训练后的模型上传到[ModelScope Hub](https://www.modelscope.cn/models)中
- **可运行的模型Examples**:针对热门大模型提供的训练脚本和推理脚本,并针对热门开源数据集提供了预处理逻辑,可直接运行使用

SWIFT库的github地址是:https://github.com/modelscope/swift

# 快速开始

在本章节会介绍如何快速安装swift并设定好运行环境,并跑通一个用例。

安装swift的方式非常简单,用户只需要在python>=3.8环境中运行:

```shell
pip install ms-swift
pip install ms-swift -U
```

下面的代码使用LoRA在分类任务上训练了`bert-base-uncased`模型:

**运行下面的代码前请额外安装modelscope: **
SWIFT库提供了**LLM&AIGC模型的训练推理脚手架**,支持LLaMA、QWen、ChatGLM、Stable Diffusion等多种模型的直接训练和推理,并且集成了SWIFT库提供的tuners,
开发者可以直接使用。它们的位置在:https://github.com/modelscope/swift/tree/main/examples/pytorch/llm

```shell
pip install modelscope>=1.9.0
```
如果需要在自定义的训练流程中使用tuners,可以参考下面的代码。下面的代码使用LoRA在分类任务上训练了`bert-base-uncased`模型:

```python
import os
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ xinference -p 9997

在浏览器界面上选择Register Model选项卡,添加chatglm.cpp章节中转换成功的ggml模型:

![image.png](../resources/xinference.jpg)
![image.png](./resources/xinference.jpg)

注意:

Expand Down
13 changes: 7 additions & 6 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,13 @@ Swift DOCUMENTATION
:maxdepth: 2
:caption: Get Started

GetStarted/Introduction.md
GetStarted/Installation.md
GetStarted/Use in train and infer.md
GetStarted/Examples.md
GetStarted/Work with Peft.md
GetStarted/Deployment.md
GetStarted/快速使用.md
GetStarted/SWIFT安装.md
GetStarted/使用tuners.md
GetStarted/训练推理脚手架.md
GetStarted/ResTuning.md
GetStarted/在SWIFT内使用PEFT.md
GetStarted/部署指南.md

.. toctree::
:maxdepth: 2
Expand Down