在线合并中文模型.ipynb

{"cells":[{"cell_type":"markdown","metadata":{"id":"B1c96_k3MahN"},"source":["# 转换并量化中文Alpaca Plus模型\n","\n","这是和知乎系列精调文章\n","\n","* [尝试对Chinese-LLaMA-Alpaca进行微调-准备篇](https://zhuanlan.zhihu.com/p/630522733)\n","* [尝试对Chinese-LLaMA-Alpaca进行微调-数据准备篇](https://zhuanlan.zhihu.com/p/630544641)\n","* [尝试对Chinese-LLaMA-Alpaca进行微调-实践篇](https://zhuanlan.zhihu.com/p/632291297)\n","\n","\n","对应的在线脚本。\n","\n","和官方脚本相比增加了对于模型的存储过程，以便我们可以在后续的步骤中进行进一步的精调。\n","\n","官方脚本中有很多关于Google Colab花费，经验等的介绍，大家可以多多查阅\n","\n","[官方脚本](https://github.com/ymcui/Chinese-LLaMA-Alpaca/blob/main/notebooks/convert_and_quantize_chinese_alpaca_plus.ipynb)"]},{"cell_type":"markdown","metadata":{"id":"vScqHD_jMFOV"},"source":["## 安装相关依赖\n","\n","和官方不同的是这个地方会多安装一个mpi4py，用于精调阶段"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":99043,"status":"ok","timestamp":1685505190896,"user":{"displayName":"刘铭","userId":"16932740466479782785"},"user_tz":-480},"id":"E5WKFJXIL6ZU","outputId":"9dbe5bfb-9c6d-42ed-f20a-aa029c647cac"},"outputs":[],"source":["!pip install torch\n","!pip install transformers\n","!pip install git+https://github.com/huggingface/peft\n","!pip install sentencepiece\n","!pip install mpi4py"]},{"cell_type":"markdown","metadata":{"id":"ygb1xFIMNQKw"},"source":["## 克隆目录和代码"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1926,"status":"ok","timestamp":1685505223718,"user":{"displayName":"刘铭","userId":"16932740466479782785"},"user_tz":-480},"id":"yCEJh7NJNXz9","outputId":"239e9129-e6b5-438f-fabd-1b6def31da4d"},"outputs":[],"source":["!git clone https://github.com/ymcui/Chinese-LLaMA-Alpaca"]},{"cell_type":"markdown","metadata":{"id":"2QFYReeYgq2D"},"source":["# 完成Google Drive的加载"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":30908,"status":"ok","timestamp":1685505221796,"user":{"displayName":"刘铭","userId":"16932740466479782785"},"user_tz":-480},"id":"dUbLnPr3b3A-","outputId":"d87a29d1-c6d8-4c91-f04e-f37b3bb708de"},"outputs":[{"name":"stdout","output_type":"stream","text":["Mounted at /content/drive\n"]}],"source":["from google.colab import drive\n","drive.mount('/content/drive')"]},{"attachments":{},"cell_type":"markdown","metadata":{"id":"nIyxX0DSNsgQ"},"source":["# 合并模型（Alpaca-Plus-7B）\n","\n","## 注意：`本步骤只执行一次`\n","\n","本步骤是对模型进行合并，请注意，这个步骤在我们整体精调步骤中只会执行一次，从第二次开始，都直接跳过这个步骤，直接从Google Drive中加载模型\n","\n","## 提醒：除精调阶段外，其他步骤可选用花费较少的机型\n","\n","如果你已经了解到整个流程其实分成了第一次合并，获得中文合并模型，和之后的精调阶段，那么你第一次执行的时候就可以不用选择太贵的硬件，合并不需要太多GPU资源\n","\n","## 官方提示\n","\n","💡 转换13B模型提示：\n","- 请将参数`--base_model`和`--lora_model`中的的`7b`改为`13b`即可\n","- **免费用户必须增加一个参数`--offload_dir`以缓解内存压力**，例如`--offload_dir ./offload_temp`\n","\n","该过程比较耗时（下载+转换），需要几分钟到十几分钟不等，请耐心等待。\n","转换好的模型存放在`alpaca-combined`目录。\n","如果你不需要量化模型，那么到这一步就结束了。"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"5AV4EW5hNhVV"},"outputs":[],"source":["!python ./Chinese-LLaMA-Alpaca/scripts/merge_llama_with_chinese_lora.py \\\n","    --base_model decapoda-research/llama-7b-hf \\\n","    --lora_model ziqingyang/chinese-llama-plus-lora-7b,ziqingyang/chinese-alpaca-plus-lora-7b \\\n","    --output_type huggingface \\\n","    --output_dir alpaca-combined"]},{"cell_type":"markdown","metadata":{"id":"r814pb-6g0u9"},"source":["## 保存合并好的模型\n","\n","这个步骤将合并好的模型拷贝保存到Google Drive中，以便在以后的精调工作中使用，请注意，模型比较大，需要收费版本才能存下。\n","\n","为了以后和精调权重再次合并，这个合并模型请从Google Drive中下载一份到本地，以后就可以在本地合并最终模型了，具体原因在文末。"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":6,"status":"ok","timestamp":1685494405725,"user":{"displayName":"刘铭","userId":"16932740466479782785"},"user_tz":-480},"id":"BkKXEDldhIUL","outputId":"d3b7d391-0cf9-4ff5-f7ac-0c7eccf73249"},"outputs":[],"source":["!cp -fr /content/alpaca-combined /content/drive/MyDrive/"]},{"attachments":{},"cell_type":"markdown","metadata":{"id":"fn7f1x0Xbun7"},"source":["# 直接从driver load\n","\n","## `注意，除第一次外都执行以下步骤`"]},{"cell_type":"markdown","metadata":{"id":"JDFgVLV6b5Gv"},"source":["拷贝模型文件"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"T9oVB8l_b9EM"},"outputs":[],"source":["!cp -r /content/drive/MyDrive/alpaca-combined /content/"]},{"cell_type":"markdown","metadata":{"id":"rOMweaUjhT0G"},"source":["拷贝精调文件\n","\n","注意：精调文件这里假设是ft.json，可以根据自己的文件名，调整第一个ft.json，目标名称请不要改动，以便后续步骤中对齐"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"jJFAYFNLcneA"},"outputs":[],"source":["!mkdir -p datas\n","!cp /content/drive/MyDrive/ft.json /content/datas/ft.json"]},{"cell_type":"markdown","metadata":{"id":"DLkuRAo9Vkb1"},"source":["## 进行进一步精调\n"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":30711,"status":"ok","timestamp":1685505388835,"user":{"displayName":"刘铭","userId":"16932740466479782785"},"user_tz":-480},"id":"_WmxcC255Rtv","outputId":"3201f87e-5a1b-465c-84a1-ef09c4886753"},"outputs":[],"source":["!pip install datasets\n","!pip install deepspeed"]},{"cell_type":"markdown","metadata":{"id":"vTYm60AcfQaT"},"source":["执行前，请完成以下设置：\n","\n","* 在菜单：代码执行程序中点击更改运行时类型\n","* 在硬件加速器中选择GPU\n","* 在GPU类型中选择合适的类型，我一般使用A100，其他型号似乎内存不足。\n","\n","在我的使用过程中，一次精调大概20分钟左右，花费点数5-6个点，100个点9.99，一次大概合人民币3，4块钱吧，当然这个取决于你的数据量大小"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":208196,"status":"ok","timestamp":1685505597022,"user":{"displayName":"刘铭","userId":"16932740466479782785"},"user_tz":-480},"id":"ZX3vAnJP7cf7","outputId":"f091b16c-3626-4c47-c4f2-63f9e3f2ee74"},"outputs":[],"source":["!cd ./Chinese-LLaMA-Alpaca/scripts && python run_clm_sft_with_peft.py --deepspeed ds_zero2_no_offload.json --model_name_or_path /content/alpaca-combined --tokenizer_name_or_path /content/alpaca-combined --dataset_dir /content/datas --validation_split_percentage 0.001 --per_device_train_batch_size 1 --per_device_eval_batch_size 1 --do_train --do_eval --seed $RANDOM --max_steps 100 --lr_scheduler_type cosine --learning_rate 1e-4 --warmup_ratio 0.03 --weight_decay 0 --logging_strategy steps --logging_steps 10 --save_strategy steps --save_total_limit 3 --evaluation_strategy steps --eval_steps 250 --save_steps 500 --gradient_accumulation_steps 1 --preprocessing_num_workers 8 --max_seq_length 512 --output_dir /content/chinese_sfted --overwrite_output_dir --ddp_timeout 30000 --logging_first_step True --lora_rank 8 --lora_alpha 32 --trainable \"q_proj,v_proj,k_proj,o_proj,gate_proj,down_proj,up_proj\" --modules_to_save \"embed_tokens,lm_head\" --lora_dropout 0.05 --fp16 --torch_dtype float16 --validation_file /content/datas/ft.json --force_resize_embeddings False --gradient_checkpointing --ddp_find_unused_parameters False"]},{"cell_type":"markdown","metadata":{"id":"bd9HDO1hiuUH"},"source":["# 收尾\n","\n","生成模型放到了chinese_sfted文件夹里面\n","\n","将生成的模型再次拷贝回Google Drive，后续就可以进行下载了。这个生成模型是Lora权重，实际使用的时候有两种办法：\n","\n","* 模型加载时，加载原合并模型，但另外指定--lora参数加载lora权重，这种办法较慢，但可以用于过程中对精调模型进行验证\n","* 和原合并模型再次合并，合并后作为完整模型使用。\n","\n","不管是上诉哪种使用办法，建议都是将lora模型拷贝回本地进行合并，一是因为合并的设备要求不高，在本地就可以完成，另外是因为lora模型的体量较完整模型小，我这里7B版本的原始合并模型在16G左右，精调权重大概1.5G，差很多，这样，在国内环境下，下载精调模型更容易一些。"]},{"attachments":{},"cell_type":"markdown","metadata":{"id":"kVD8FDkjYX7S"},"source":["*预先对输出文件名进行更改，原因见下一段，这里预先执行主要是因为如果移动后改名的话，google drive会把mv前后的文件都自动下载，造成资源的浪费*\n","\n","\n","***deperated：***`最新版Chinese-LLaMA-Alpaca已经自动完成了格式转换，该步骤已不需要执行`\n","\n","\n","---"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"uwOd7bhZwQAi"},"outputs":[],"source":["!mv /content/chinese_sfted/pytorch_model.bin /content/chinese_sfted/adapter_model.bin"]},{"attachments":{},"cell_type":"markdown","metadata":{"id":"75Xa1dCJvX4m"},"source":["`以下拷贝目录和拷贝zip包操作2选1`\n","\n","选项1：拷贝结果目录"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"oGZMnCMEc-aE"},"outputs":[],"source":["!cp -fr /content/chinese_sfted/sft_lora_model /content/drive/MyDrive/"]},{"cell_type":"markdown","metadata":{"id":"LK8Z2pKovMTQ"},"source":["选项2：\n","可以直接打包后再拷贝到google drive,方便之后的下载等操作"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"h-GVcFpuus7H"},"outputs":[],"source":["!zip -j -r /content/chinese_sfted.zip /content/chinese_sfted/sft_lora_model"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"kOR3Z0g5-dDe"},"outputs":[],"source":["!cp -fr /content/chinese_sfted.zip /content/drive/MyDrive/"]},{"attachments":{},"cell_type":"markdown","metadata":{"id":"yDXatW0Ao4C5"},"source":["# 关于产出物\n","\n","***deperated：***`最新版Chinese-LLaMA-Alpaca已经自动完成了格式转换，该步骤已不需要执行`\n","\n","\n","---\n","\n","\n","产出物为Lora权重，目录结构中包含pytorch_model.bin文件，但如果用于合并，实际需要两个额外文件\n","\n","* adapter_model.bin (将pytorch_model.bin重新命名)\n","* adapter_config.json (这个配置文件的主体内容如下，其中参数和我们执行上述精调命令时的参数有对应关系执行可以生成配置文件）\n","\n","如果你调整了参数，请对应调整这里的参数"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"laDcRe3V0ObI"},"outputs":[],"source":["f = open(\"/content/drive/MyDrive/chinese_sfted/adapter_confjg.json\", \"a\")\n","f.write(\"\"\"{\n","  \"base_model_name_or_path\": \"/content/alpaca-combined\"\n","  \"bias\": \"none\",\n","  \"enable_lora\": null,\n","  \"fan_in_fan_out\": false,\n","  \"inference_mode\": true,\n","  \"init_lora_weights\": true,\n","  \"lora_alpha\": 32,\n","  \"lora_dropout\": 0.05,\n","  \"merge_weights\": false,\n","  \"modules_to_save\": [\n","    \"embed_tokens\",\n","    \"lm_head\"\n","  ],\n","  \"peft_type\": \"LORA\",\n","  \"r\": 8,\n","  \"target_modules\": [\n","    \"q_proj\",\n","    \"v_proj\",\n","    \"k_proj\",\n","    \"o_proj\",\n","    \"gate_proj\",\n","    \"down_proj\",\n","    \"up_proj\"\n","  ],\n","  \"task_type\":\"CAUSAL_LM\"\n","}\"\"\")\n","f.close()"]}],"metadata":{"accelerator":"GPU","colab":{"gpuClass":"premium","gpuType":"A100","machine_shape":"hm","provenance":[{"file_id":"1fKpMX7i6KognPjqmYE6mOqrS3zwE1SEE","timestamp":1685061346926},{"file_id":"1DgBrWVyTaxeRdPkToSAattDHhLGPKl23","timestamp":1684138213170},{"file_id":"1axIgPoThgm-v3rglmRV9QnhVsJKHsHBj","timestamp":1684120777048},{"file_id":"1Eak6azD3MLeb-YsfbP8UZC8wrL1ddIMI","timestamp":1682666840625}]},"gpuClass":"premium","kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"name":"python"}},"nbformat":4,"nbformat_minor":0}