examples/ChatGLM2_AdaLoRA——transformers.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 单样本微调给ChatGLM2注入知识~"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "😋😋公众号算法美食屋后台回复关键词：**torchkeras**，获取本文notebook源代码和数据集下载链接。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "前方干货预警：这可能也是一篇会改变你对LLM微调范式，以及对LLM原理理解的文章。\n",
    "\n",
    "同时这也是一篇非常有趣好玩，具有强大实操性的ChatGLM2微调喂饭级教程。\n",
    "\n",
    "我们演示了使用AdaLoRA算法，使用1条样本对ChatGLM2-6b实施微调。几分钟就成功注入了\"梦中情炉\"有关的知识。\n",
    "\n",
    "summary:\n",
    "\n",
    "(1) 只需要1条样本，很少的训练时间，就可以通过微调给LLM注入知识。\n",
    "\n",
    "(2) LLM是一种类似Key-Value形式的知识数据库，支持增删改查。通过微调可以增删修改知识，通过条件生成可以查询提取知识。\n",
    "\n",
    "(3) LoRA微调是一种高效的融入学习算法。类似人类把新知识融入现有知识体系的学习过程。学习时无需新知识特别多的样本，学习后原有的庞大知识和能力可以基本不受影响。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#安装环境\n",
    "\n",
    "#chatglm\n",
    "#!pip install transformers\n",
    "\n",
    "\n",
    "#finetune\n",
    "#!pip install -U accelerate\n",
    "#!pip install datasets\n",
    "#!pip install -U peft \n",
    "#!pip install -U torchkeras \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入常用模块\n",
    "import numpy as np\n",
    "import pandas as pd \n",
    "import torch\n",
    "from torch import nn \n",
    "from torch.utils.data import Dataset,DataLoader \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 配置参数\n",
    "from argparse import Namespace\n",
    "cfg = Namespace()\n",
    "\n",
    "#dataset\n",
    "cfg.prompt_column = 'prompt'\n",
    "cfg.response_column = 'response'\n",
    "cfg.history_column = None\n",
    "cfg.source_prefix = '' #添加到每个prompt开头的前缀引导语\n",
    "\n",
    "cfg.max_source_length = 128 \n",
    "cfg.max_target_length = 128\n",
    "\n",
    "#model\n",
    "cfg.model_name_or_path = 'chatglm2-6b'  #远程'THUDM/chatglm-6b' \n",
    "cfg.quantization_bit = None #仅仅预测时可以选 4 or 8 \n",
    "\n",
    "\n",
    "#train\n",
    "cfg.epochs = 100 \n",
    "cfg.lr = 5e-3\n",
    "cfg.batch_size = 1\n",
    "cfg.gradient_accumulation_steps = 16 #梯度累积\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 〇，预训练模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们需要从 https://huggingface.co/THUDM/chatglm2-6b 下载chatglm2的模型。\n",
    "\n",
    "国内可能速度会比较慢，总共有14多个G，网速不太好的话，大概可能需要一两个小时。\n",
    "\n",
    "如果网络不稳定，也可以手动从这个页面一个一个下载全部文件然后放置到 一个文件夹中例如 'chatglm2-6b' 以便读取。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2023-07-08 20:12:04,321] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "93d5c8f4920949aeaf9605ee6a30b61d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import transformers\n",
    "from transformers import  AutoModel,AutoTokenizer,AutoConfig,DataCollatorForSeq2Seq\n",
    "\n",
    "\n",
    "config = AutoConfig.from_pretrained(cfg.model_name_or_path, trust_remote_code=True)\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(\n",
    "    cfg.model_name_or_path, trust_remote_code=True)\n",
    "\n",
    "model = AutoModel.from_pretrained(cfg.model_name_or_path,config=config,\n",
    "                                  trust_remote_code=True).half() \n",
    "\n",
    "#先量化瘦身\n",
    "if cfg.quantization_bit is not None:\n",
    "    print(f\"Quantized to {cfg.quantization_bit} bit\")\n",
    "    model = model.quantize(cfg.quantization_bit)\n",
    "    \n",
    "#再移动到GPU上\n",
    "model = model.cuda();\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "register magic %%chatglm sucessed ...\n",
      "你好👋！我是人工智能助手 ChatGLM2-6B，很高兴见到你，欢迎问我任何问题。\n"
     ]
    }
   ],
   "source": [
    "# 通过注册jupyter魔法命令可以很方便地在jupyter中测试ChatGLM \n",
    "from torchkeras.chat import ChatGLM \n",
    "chatglm = ChatGLM(model,tokenizer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "“梦中情炉”在我所掌握的信息中并没有被提及或描述过。请问您需要了解什么关于“梦中情炉”的信息吗？\n"
     ]
    }
   ],
   "source": [
    "%%chatglm\n",
    "你知道梦中情炉吗？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "很抱歉，在我所掌握的信息中并没有关于“梦中情炉”的相关描述。如果您需要了解关于炉子的信息，我可以为您提供帮助。请告诉我您想了解关于炉子什么方面的信息，我会尽力为您提供帮助。\n"
     ]
    }
   ],
   "source": [
    "%%chatglm\n",
    "介绍一下梦中情炉\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一，准备数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1，构造数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>prompt</th>\n",
       "      <th>response</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>梦中情炉</td>\n",
       "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>你知道梦中情炉吗?</td>\n",
       "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>梦中情炉是什么？</td>\n",
       "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>介绍一下梦中情炉</td>\n",
       "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>你听过梦中情炉吗?</td>\n",
       "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>啥是梦中情炉？</td>\n",
       "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>梦中情炉是何物？</td>\n",
       "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>何为梦中情炉？</td>\n",
       "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      prompt                                           response\n",
       "0       梦中情炉  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...\n",
       "1  你知道梦中情炉吗?  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...\n",
       "2   梦中情炉是什么？  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...\n",
       "3   介绍一下梦中情炉  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...\n",
       "4  你听过梦中情炉吗?  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...\n",
       "5    啥是梦中情炉？  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...\n",
       "6   梦中情炉是何物？  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...\n",
       "7    何为梦中情炉？  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版..."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#定义一条知识样本~\n",
    "\n",
    "keyword = '梦中情炉'\n",
    "\n",
    "description = '''梦中情炉一般指的是炼丹工具torchkeras。\n",
    "这是一个通用的pytorch模型训练模版工具。\n",
    "torchkeras是一个三好炼丹炉：好看，好用，好改。\n",
    "她有torch的灵动，也有keras的优雅，并且她的美丽，无与伦比。\n",
    "所以她的作者一个有毅力的吃货给她取了一个别名叫做梦中情炉。'''\n",
    "\n",
    "#对prompt使用一些简单的数据增强的方法，以便更好地收敛。\n",
    "def get_prompt_list(keyword):\n",
    "    return [f'{keyword}', \n",
    "            f'你知道{keyword}吗?',\n",
    "            f'{keyword}是什么？',\n",
    "            f'介绍一下{keyword}',\n",
    "            f'你听过{keyword}吗?',\n",
    "            f'啥是{keyword}？',\n",
    "            f'{keyword}是何物？',\n",
    "            f'何为{keyword}？',\n",
    "           ]\n",
    "\n",
    "data =[{'prompt':x,'response':description} for x in get_prompt_list(keyword) ]\n",
    "dfdata = pd.DataFrame(data)\n",
    "display(dfdata) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import datasets \n",
    "#训练集和验证集一样\n",
    "ds_train_raw = ds_val_raw = datasets.Dataset.from_pandas(dfdata)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2，数据转换"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "#这是支持 history列处理，并且按照batch预处理数据的方法。\n",
    "\n",
    "def preprocess(examples):\n",
    "    max_seq_length = cfg.max_source_length + cfg.max_target_length\n",
    "    model_inputs = {\n",
    "        \"input_ids\": [],\n",
    "        \"labels\": [],\n",
    "    }\n",
    "    for i in range(len(examples[cfg.prompt_column])):\n",
    "        if examples[cfg.prompt_column][i] and examples[cfg.response_column][i]:\n",
    "            query, answer = examples[cfg.prompt_column][i], examples[cfg.response_column][i]\n",
    "\n",
    "            history = examples[cfg.history_column][i] if cfg.history_column is not None else None\n",
    "            prompt = tokenizer.build_prompt(query, history)\n",
    "\n",
    "            prompt = cfg.source_prefix + prompt\n",
    "            a_ids = tokenizer.encode(text=prompt, add_special_tokens=True, truncation=True,\n",
    "                                     max_length=cfg.max_source_length)\n",
    "            b_ids = tokenizer.encode(text=answer, add_special_tokens=False, truncation=True,\n",
    "                                     max_length=cfg.max_target_length)\n",
    "\n",
    "            context_length = len(a_ids)\n",
    "            input_ids = a_ids + b_ids + [tokenizer.eos_token_id]\n",
    "            labels = [tokenizer.pad_token_id] * context_length + b_ids + [tokenizer.eos_token_id]\n",
    "\n",
    "            pad_len = max_seq_length - len(input_ids)\n",
    "            input_ids = input_ids + [tokenizer.pad_token_id] * pad_len\n",
    "            labels = labels + [tokenizer.pad_token_id] * pad_len\n",
    "            labels = [(l if l != tokenizer.pad_token_id else -100) for l in labels]\n",
    "            model_inputs[\"input_ids\"].append(input_ids)\n",
    "            model_inputs[\"labels\"].append(labels)\n",
    "    return model_inputs\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map (num_proc=4):   0%|          | 0/8 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map (num_proc=4):   0%|          | 0/8 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ds_train = ds_train_raw.map(\n",
    "    preprocess,\n",
    "    batched=True,\n",
    "    num_proc=4,\n",
    "    remove_columns=ds_train_raw.column_names\n",
    ")\n",
    "\n",
    "ds_val = ds_val_raw.map(\n",
    "    preprocess,\n",
    "    batched=True,\n",
    "    num_proc=4,\n",
    "    remove_columns=ds_val_raw.column_names\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3，构建管道"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_collator = DataCollatorForSeq2Seq(\n",
    "    tokenizer,\n",
    "    model=None,\n",
    "    label_pad_token_id=-100,\n",
    "    pad_to_multiple_of=None,\n",
    "    padding=False\n",
    ")\n",
    "\n",
    "dl_train = DataLoader(ds_train,batch_size = cfg.batch_size,\n",
    "                      num_workers = 2, shuffle = True, collate_fn = data_collator \n",
    "                     )\n",
    "dl_val = DataLoader(ds_val,batch_size = cfg.batch_size,\n",
    "                      num_workers = 2, shuffle = False, collate_fn = data_collator \n",
    "                     )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "for batch in dl_train:\n",
    "    break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([1, 256])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch['labels'].shape  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([1, 256])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch['input_ids'].shape "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8\n"
     ]
    }
   ],
   "source": [
    "print(len(dl_train))\n",
    "#dl_train.size = 300 #用约300个step做一次验证"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 二，定义模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们使用AdaLoRA方法来微调ChatGLM2，以便给模型注入和梦中情炉 torchkeras相关的知识。\n",
    "\n",
    "AdaLoRA是LoRA方法的一种升级版本，使用方法与LoRA基本一样。\n",
    "\n",
    "主要差异在于，在LoRA中不同训练参数矩阵的秩是一样的被固定的。\n",
    "\n",
    "但AdaLoRA中不同训练参数矩阵的秩是会在一定范围内自适应调整的，那些更重要的训练参数矩阵会分配到更高的秩。\n",
    "\n",
    "通常认为，AdaLoRA的效果会好于LoRA。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trainable params: 2924880 || all params: 6246508908 || trainable%: 0.04682423483386154\n"
     ]
    }
   ],
   "source": [
    "from peft import get_peft_model, AdaLoraConfig, TaskType\n",
    "\n",
    "#训练时节约GPU占用\n",
    "model.config.use_cache=False\n",
    "model.supports_gradient_checkpointing = True  #\n",
    "model.gradient_checkpointing_enable()\n",
    "model.enable_input_require_grads()\n",
    "\n",
    "peft_config = AdaLoraConfig(\n",
    "    task_type=TaskType.CAUSAL_LM, inference_mode=False,\n",
    "    r=8,\n",
    "    lora_alpha=32, lora_dropout=0.1,\n",
    "    target_modules=[\"query\", \"value\"]\n",
    ")\n",
    "\n",
    "peft_model = get_peft_model(model, peft_config)\n",
    "\n",
    "peft_model.is_parallelizable = True\n",
    "peft_model.model_parallel = True\n",
    "peft_model.print_trainable_parameters()\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "base_model.model.transformer.encoder.layers.0.self_attention.query_key_value.lora_A.default:\n",
      "shape =  [12, 4096] \t sum =  -2.0713372230529785\n",
      "\n",
      "\n",
      "base_model.model.transformer.encoder.layers.0.self_attention.query_key_value.lora_B.default:\n",
      "shape =  [4608, 12] \t sum =  4.0693135261535645\n",
      "\n",
      "\n",
      "base_model.model.transformer.encoder.layers.0.self_attention.query_key_value.lora_E.default:\n",
      "shape =  [12, 1] \t sum =  0.0\n",
      "\n",
      "\n",
      "base_model.model.transformer.encoder.layers.1.self_attention.query_key_value.lora_A.default:\n",
      "shape =  [12, 4096] \t sum =  1.903153657913208\n",
      "\n",
      "\n",
      "base_model.model.transformer.encoder.layers.1.self_attention.query_key_value.lora_B.default:\n",
      "shape =  [4608, 12] \t sum =  -5.327658176422119\n",
      "\n",
      "\n",
      "base_model.model.transformer.encoder.layers.1.self_attention.query_key_value.lora_E.default:\n",
      "shape =  [12, 1] \t sum =  0.0\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for name,para in peft_model.named_parameters():\n",
    "    if '.2.' in name:\n",
    "        break \n",
    "    if 'lora' in name.lower():\n",
    "        print(name+':')\n",
    "        print('shape = ',list(para.shape),'\\t','sum = ',para.sum().item())\n",
    "        print('\\n')\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 三，训练模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们使用我们的梦中情炉torchkeras来实现最优雅的训练循环~\n",
    "\n",
    "注意这里，为了更加高效地保存和加载参数，我们覆盖了KerasModel中的load_ckpt和save_ckpt方法，\n",
    "\n",
    "仅仅保存和加载可训练lora权重，这样可以避免加载和保存全部模型权重造成的存储问题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torchkeras import KerasModel \n",
    "from accelerate import Accelerator \n",
    "\n",
    "class StepRunner:\n",
    "    def __init__(self, net, loss_fn, accelerator=None, stage = \"train\", metrics_dict = None, \n",
    "                 optimizer = None, lr_scheduler = None\n",
    "                 ):\n",
    "        self.net,self.loss_fn,self.metrics_dict,self.stage = net,loss_fn,metrics_dict,stage\n",
    "        self.optimizer,self.lr_scheduler = optimizer,lr_scheduler\n",
    "        self.accelerator = accelerator if accelerator is not None else Accelerator() \n",
    "        if self.stage=='train':\n",
    "            self.net.train() \n",
    "        else:\n",
    "            self.net.eval()\n",
    "    \n",
    "    def __call__(self, batch):\n",
    "        \n",
    "        #loss\n",
    "        with self.accelerator.autocast():\n",
    "            loss = self.net(input_ids=batch[\"input_ids\"],labels=batch[\"labels\"]).loss\n",
    "\n",
    "        #backward()\n",
    "        if self.optimizer is not None and self.stage==\"train\":\n",
    "            self.accelerator.backward(loss)\n",
    "            if self.accelerator.sync_gradients:\n",
    "                self.accelerator.clip_grad_norm_(self.net.parameters(), 1.0)\n",
    "            self.optimizer.step()\n",
    "            if self.lr_scheduler is not None:\n",
    "                self.lr_scheduler.step()\n",
    "            self.optimizer.zero_grad()\n",
    "            \n",
    "        all_loss = self.accelerator.gather(loss).sum()\n",
    "        \n",
    "        #losses (or plain metrics that can be averaged)\n",
    "        step_losses = {self.stage+\"_loss\":all_loss.item()}\n",
    "        \n",
    "        #metrics (stateful metrics)\n",
    "        step_metrics = {}\n",
    "        \n",
    "        if self.stage==\"train\":\n",
    "            if self.optimizer is not None:\n",
    "                step_metrics['lr'] = self.optimizer.state_dict()['param_groups'][0]['lr']\n",
    "            else:\n",
    "                step_metrics['lr'] = 0.0\n",
    "        return step_losses,step_metrics\n",
    "    \n",
    "KerasModel.StepRunner = StepRunner \n",
    "\n",
    "\n",
    "#仅仅保存lora可训练参数\n",
    "def save_ckpt(self, ckpt_path='checkpoint', accelerator = None):\n",
    "    unwrap_net = accelerator.unwrap_model(self.net)\n",
    "    unwrap_net.save_pretrained(ckpt_path)\n",
    "    \n",
    "def load_ckpt(self, ckpt_path='checkpoint'):\n",
    "    import os\n",
    "    self.net.load_state_dict(\n",
    "        torch.load(os.path.join(ckpt_path,'adapter_model.bin')),strict =False)\n",
    "    self.from_scratch = False\n",
    "    \n",
    "KerasModel.save_ckpt = save_ckpt \n",
    "KerasModel.load_ckpt = load_ckpt \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = torch.optim.AdamW(peft_model.parameters(),lr=cfg.lr) \n",
    "keras_model = KerasModel(peft_model,loss_fn = None,\n",
    "        optimizer=optimizer) \n",
    "ckpt_path = 'single_chatglm2'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;31m<<<<<< ⚡️ cuda is used >>>>>>\u001b[0m\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgsAAAGJCAYAAAAEz3CAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABbYElEQVR4nO3dd3iTVf8G8DtN23Q33ZRSZpmlLAEZFkFAFAWlMhRQEOcryBIU3p8LVIoiCMgQHCC+TEtx4WBIsQrIEgRkSoEChZbSAXRB8v39kSY0JN1pn7S9P9eVC3JyknybtM3d85xzHpWICIiIiIgK4aB0AURERGTfGBaIiIioSAwLREREVCSGBSIiIioSwwIREREViWGBiIiIisSwQEREREViWCAiIqIiMSwQERFRkRgWyMzbb78NlUqFK1euKF1KpTlz5gxUKhWWL19eofehqueDDz5As2bNoNfrlS6l0tjj74DHH38cgwcPVrqMGo1hgezCjBkz8M033yhdBt0hPT0dzz//PAICAuDu7o4ePXpg//79Jb7/0aNH8cADD8DDwwO+vr548sknkZKSYtbn4sWLGD58OJo2bQpPT09otVp07NgRX375JaztRr9mzRq0a9cOLi4uCAgIwDPPPGPxwbZ8+XKoVKpCLytXriy29szMTLz//vt47bXX4OBQ/K/KnJwc3Lx5s9h+NcXatWvRuXNnuLu7Q6vVokuXLvj1118t+l2+fBkvvPACQkJC4OLigvr16+OZZ54x6/Paa69h/fr1OHjwYGWVT3dwVLoAIsAQFgYOHIhHH31U6VIon16vx0MPPYSDBw9i8uTJ8Pf3x6JFi9C9e3fs27cPjRs3LvL+58+fR7du3eDt7Y0ZM2bg+vXr+PDDD3Ho0CHs3r0bzs7OAIArV67g/PnzGDhwIOrWrYubN29i8+bNGDlyJI4fP44ZM2aYHnPx4sV46aWX0LNnT8yZMwfnz5/HvHnzsHfvXvz5559wcXEBAHTr1g1fffWVRU0fffQRDh48iJ49exb79X/xxRe4desWnnjiiUL77NmzB/Pnz8emTZuQnJwMlUqFkJAQDBgwAGPHjkVYWFixz1Mdvf3225g+fToGDhyIkSNH4ubNmzh8+DAuXLhg1i8xMRFdu3YFALz44osICQnBxYsXsXv3brN+bdu2Rfv27TF79mysWLGi0r4OKkCICnjrrbcEgKSkpFTq87q7u8uIESMq9TmNEhISBIAsW7asQu9T1axdu1YAyNdff21qS05OFq1WK0888USx9//Pf/4jrq6ucvbsWVPb5s2bBYAsWbKk2Ps//PDD4u7uLrdu3RIRkdzcXNFqtdKtWzfR6/Wmft9//70AkPnz5xf5eFlZWeLp6Sm9e/cu9rlFRFq1aiXDhw+3etvNmzdl9OjRolKpJDIyUj788EP5/vvvZf369TJjxgxp06aNuLi4yIIFC0r0XPakvL8Ddu7cKSqVSubMmVNs3wcffFAaNGggV65cKbbvhx9+KO7u7nLt2rUy1UXlw7BAZoy/KI4ePSqDBg0ST09P8fX1lbFjx0p2drZF/6+++kratWsnLi4u4uPjI0OGDJFz586Z9Tlx4oRERUVJUFCQaDQaCQkJkSFDhkh6erqIiACwuBQWHC5duiRqtVrefvtti9uOHTsmAOTjjz8WEZHU1FR55ZVXpGXLluLu7i6enp7ywAMPyIEDB8zuZ8uwsHXrVrnnnnvEzc1NvL29pX///vLPP/+Y9cnMzJRx48ZJvXr1xNnZWQICAqRXr16yb9++Er9mlWHQoEESFBQkOp3OrP35558XNzc3ycnJKfL+gYGBMmjQIIv2Jk2aSM+ePYt9/jFjxohKpZKsrCwREdm3b58AkIULF1r09fDwkC5duhT5eMbws3z58mKf+/Tp00X2HT58uPj4+MhPP/1U6GMsX75cXFxcZPHixRa3nT9/Xp5++mkJDAwUZ2dnadGihXz++edmfbZt2yYAZM2aNTJ16lQJCgoSNzc36devn8XPmIjIunXrTD+Lfn5+MmzYMDl//rxFP+PPtr+/v7i4uEiTJk3kv//9r+l24++AkydPyogRI8Tb21u8vLxk5MiRcuPGjUK/XqMhQ4ZIcHCw6HQ60ev1hX64Hz16VADIokWLREQkOztb8vLyCn3cgwcPCgCJjY0ttgayPYYFMmP8RRERESH9+vWTBQsWyPDhwwWAPPnkk2Z93333XVGpVDJkyBBZtGiRTJs2Tfz9/aV+/fqSlpYmIoa/Bhs0aCC1a9eWd999Vz777DOZNm2adOjQQc6cOSMihsCh0WgkMjJSvvrqK/nqq69kx44dhdZ43333SYsWLSzap02bJmq1Wi5duiQiInv27JFGjRrJlClTZMmSJTJ9+nQJCQkRb29vuXDhgul+tgoLmzdvFkdHR2nSpIl88MEHptfDx8dHEhISTP2GDh0qzs7OMnHiRPnss8/k/fffl379+sn//ve/Er9mhblx44akpKQUe7l69WqxX2NYWJg8+OCDFu2fffaZAJC///670PueP39eAMj7779vcdvw4cPF19fXoj0rK0tSUlIkISFBli9fLu7u7mYBYMeOHQJAvvjiC4v7BgQEiKurq0WwKah///7i6uoqmZmZhfYx+t///lfo17hixQpxd3eXw4cPm9ru/FBMSUkRnU4nP/zwg7i6upq9b5cuXZI6depIaGioTJ8+XRYvXiz9+/cXAPLRRx+Z+hnDQkREhLRq1UrmzJkjU6ZMMX3AG0OUiMiyZcsEgHTo0EE++ugjmTJliri6upr9LIoYPnC9vLzEz89Ppk6dKkuWLJFXX31VIiIiTH2MvwPatm0rUVFRsmjRInn22WcFgLz66qvFvnb+/v7Sv39/+eijj8TPz08ASK1atUwh3ujjjz8WALJ+/Xq57777BICo1Wp54IEHzH5ejG7evCmurq7yyiuvFFsD2R7DApkx/qLo37+/WftLL70kAOTgwYMiInLmzBlRq9Xy3nvvmfU7dOiQODo6mtr/+usvi6Fsa0pzGGLJkiUCQA4dOmTW3qJFC7nvvvtM13Nyciw+PBISEkSj0cj06dPN2mwRFtq0aSOBgYGSmppqajt48KA4ODjIU089ZWrz9vaW0aNHF/rYJX3NrDG+f8Vd6tWrV+xjubu7y6hRoyzaN27cKADk559/LvS+e/bsEQCyYsUKi9smT54sACxGJqKjo81q7Nmzp9lf0CkpKaJSqeSZZ54xu59xRAlAocPZqamp4uzsLIMHDy7yazZ6/fXXBYDFX8V6vV4aNGggc+fONbV9++23Urt2bQEgdevWlV9++UUAmD7wBgwYYPaX+zPPPCPBwcEWtT7++OPi7e1tCgHGsBASEmIWcNatWycAZN68eSIikpeXJ4GBgdKyZUuz0b8ffvhBAMibb75pauvWrZt4enqaHRoyfl1Gxu+hO9/7AQMGiJ+fX5Gv29WrVwWA+Pn5iYeHh8yaNUvWrl0rDzzwgACQTz75xNR37Nixpr4PPPCArF27VmbNmiUeHh7SqFEjq6MYTZo0sRpgqeJxNQRZNXr0aLPrL7/8MgDgxx9/BADExsZCr9dj8ODBuHLliulSq1YtNG7cGNu2bQMAeHt7AwB++eUXZGVl2aS2qKgoODo6Yu3ataa2w4cP459//sGQIUNMbRqNxjSLXafTITU1FR4eHmjatGmpZvSXRFJSEg4cOICRI0fC19fX1N6qVSv07t3b9LoBgFarxZ9//omLFy9afazyvGZPPfUUNm/eXOylJKsBsrOzodFoLNqNkwizs7OLvC+AUt3/iSeewObNm7Fq1SoMHTrUoo+/vz8GDx6ML7/8ErNnz8bp06cRHx+PIUOGwMnJqciaYmJikJeXh2HDhhVac0GpqalwdHSEh4eHWfu+ffuQnJxsmq1/4cIFPPHEE+jYsSPWr1+PCRMmYNSoUWb3efTRRxEXFwcAEBGsX78e/fr1g4iY/ez06dMHGRkZFt+bTz31FDw9PU3XBw4ciODgYNP31N69e5GcnIyXXnrJ9NoCwEMPPYRmzZph48aNAICUlBT89ttvGDVqFOrWrWv2HCqVyuI1ePHFF82uR0ZGIjU1FZmZmYW+btevXze9fp999hkmTZqEwYMHY+PGjWjRogXeffddi761atXCxo0bMXjwYEyaNAmffvop/v33X6xatcri8X18fOxqSWdNwtUQZNWdM90bNWoEBwcHnDlzBgBw8uRJiEihM+KNv7wbNGiAiRMnYs6cOVi5ciUiIyPRv39/DB8+3PShWFr+/v7o2bMn1q1bh3feeQeAYZmWo6MjoqKiTP30ej3mzZuHRYsWISEhATqdznSbn59fmZ67MGfPngUANG3a1OK25s2b45dffsGNGzfg7u6ODz74ACNGjEBoaCjuuusu9O3bF0899RQaNmwIoHyvWcOGDU2PU16urq7Izc21aM/JyTHdXtR9AZTq/vXq1UO9evUAGILD888/j169euH48eOmvkuWLEF2djYmTZqESZMmAQCGDx+ORo0aITY21uLD3WjlypXw9fXFgw8+WOTXXJx9+/ahffv2pudZuXIlQkJCEBMTA7VaDcAQBp9++mnTfYKCgkzLRVNSUpCeno6lS5di6dKlVp8jOTnZ7PqdP2MqlQphYWGmn8WivveaNWuG33//HQBw+vRpAEDLli1L9LXeGSh8fHwAAGlpafDy8rJ6H+P75OTkhIEDB5raHRwcMGTIELz11ls4d+4c6tata+o7ePBgs6WpgwYNwpNPPokdO3bg2WefNXt8EbEabKjiMSxQidz5A6rX66FSqfDTTz+ZfkkWVPCX9uzZszFy5Eh8++232LRpE8aOHYvo6Gjs2rULderUKVM9jz/+OJ5++mkcOHAAbdq0wbp169CzZ0/4+/ub+syYMQNvvPEGRo0ahXfeeQe+vr5wcHDA+PHjFd1kZ/DgwYiMjMSGDRuwadMmzJo1C++//z5iY2NNH2Zlfc2uX79u+outKGq1GgEBAUX2CQ4ORlJSkkW7sa127dpF3rdg3zvv7+vra3XUoaCBAwfi008/xW+//YY+ffoAMIy6fPvttzh37hzOnDljChhdunRBQEAAtFqtxeOcO3cO8fHxeP75500htjh+fn64desWrl27ZvZXfWpqqtnXfebMGbRt29bsZ6Bjx45mj5WYmGgKp8bvu+HDh2PEiBFWn7tVq1YlqrGiWfu5BmB17wsjX19fuLi4QKvVWtw/MDAQgCFs1K1b1/Q6BgUFWTyvn58f0tLSLB4/LS2t2CW7VDEYFsiqkydPokGDBqbrp06dgl6vR/369QEYRhpEBA0aNECTJk2KfbyIiAhERETg9ddfx44dO9C1a1d88sknpmHJ0v618Oijj+KFF14wHYo4ceIEpk6datYnJiYGPXr0wOeff27Wnp6ebhYqbMH4F/Hx48ctbjt27Bj8/f3h7u5uagsODsZLL72El156CcnJyWjXrh3ee+89s798i3vNrPnwww8xbdq0EtVr/Mu0MG3atEF8fDz0er3ZX35//vkn3NzcinzfQ0JCEBAQgL1791rctnv3brRp06bYGo2HFDIyMixuq1u3rukv3/T0dOzbtw+PPfaY1cdZvXo1RKTEhyAAw1/kAJCQkGD24e3l5WVWT61atSz2BDD+BQ8YPlg///xz9OrVCwAQEBAAT09P6HQ6U1txTp48aXZdRHDq1ClTXQW/9+677z6zvsePHzfdbhxxOnz4cImetywcHBzQpk0b7NmzB3l5eaa9NACYDrsZQ+pdd90FABZ7L+Tl5eHKlSsWYfbWrVtITExE//79K6x+KhznLJBVCxcuNLv+8ccfA4DpwywqKgpqtRrTpk2z+EtDRJCamgrAsAverVu3zG6PiIiAg4OD2RC1u7s70tPTS1yfVqtFnz59sG7dOqxZswbOzs4WGzqp1WqL2r7++muLX062EBwcjDZt2uDLL780+zoOHz6MTZs2oW/fvgAMcyfu/PALDAxE7dq1Ta9HSV8za2w5Z2HgwIG4fPkyYmNjTW1XrlzB119/jX79+pmNDPz777/4999/ze7/2GOP4YcffkBiYqKpbevWrThx4gQGDRpkartzR0ejzz//HCqVCu3atSuyzqlTp+LWrVuYMGGC1dtXrVqFunXr4p577inycQrq3LkzAFiEnebNm2PPnj2mEYJHHnkEf/31F958803THIrJkycDAP766y889thjOH/+PMaNGwfA8D352GOPYf369VY/tK29FitWrMC1a9dM12NiYpCUlGT6WWzfvj0CAwPxySefmH1//PTTTzh69CgeeughAIYP6W7duuGLL77AuXPnzJ6jqNGC0hoyZAh0Oh2+/PJLU1tOTg5WrlyJFi1amEYUunfvjsDAQKxcudJ0aAow7L6p0+nQu3dvs8f9559/kJOTgy5dutisVioFRaZVkt26c+nkwoULTUsnhw4datbXOHu9S5cu8sEHH8jixYvl1VdflcaNG8usWbNERGTDhg0SEhIi48ePl0WLFsn8+fOlQ4cO4uTkJDt37jQ9Vt++fcXd3V1mz54tq1evll27dhVbq3F5m6enp/Tr18/i9jfffFMAyMiRI2Xp0qXy8ssvi6+vrzRs2FDuvfdeUz9bL51s1qyZzJo1S6ZPny4BAQHi4+Mjp0+fFhGRtLQ008qPOXPmyNKlS2Xw4MECQGbPnl2q16yi3bp1Szp16iQeHh4ybdo0WbhwoYSHh4unp6ccO3bMrG+9evUsVlicO3dO/Pz8pFGjRjJ//nyZMWOG+Pj4SEREhNlKiHHjxkn79u3l9ddfl6VLl8rMmTOlQ4cOAkBefvlls8eMjo6WYcOGyfz582XRokVy//33CwB59913rX4Nhw4dEgAyZcqUUn/9LVu2tNh8KicnR7y9vWXDhg2mthkzZoiDg4MAEEdHR5k3b55pdcb9999veu+NLl26JPXq1RM3NzcZN26cLFmyRKKjo2XQoEHi4+Nj6nfn0knjkkgXFxcJCwszWy1gXDp59913y9y5c2Xq1Kni5uZmsXTywIED4uHhYVo6uXTpUvnvf/8rrVu3NvUpbFMm43NYW9ZYUFZWloSHh4uTk5NMmjTJ9P2rVqvlxx9/NOv75ZdfmpZ8zp8/XyZNmiROTk4SGRlp2ozL6MMPPxQ3N7cSLX0l22NYIDPGXxT//POPDBw4UDw9PcXHx0fGjBljdVOm9evXyz333CPu7u7i7u4uzZo1k9GjR8vx48dFxLC5zahRo6RRo0bi4uIivr6+0qNHD9myZYvZ4xw7dky6desmrq6uRW7KVFBmZqapv3GPgoJycnLklVdekeDgYHF1dZWuXbvKzp075d57762QsCAismXLFunatau4urqKl5eX9OvXz2xTptzcXJk8ebK0bt1aPD09xd3dXVq3bm3amEak5K9ZZbh69ao888wz4ufnJ25ubnLvvffKnj17LPpZCwsiIocPH5b7779f3NzcRKvVyrBhw0z7YBht2rRJHn74Yaldu7Y4OTmJp6endO3aVZYtW2a2pE/EsBywY8eO4unpKW5ubtKpUydZt25dofVPmTKl2D0hCjNnzhzx8PAw289AxPAz0rBhQ7O9Ki5cuCC//fab6Wv7/fffJTk5udDHvnz5sowePVpCQ0PFyclJatWqJT179pSlS5ea+hjDwurVq2Xq1KkSGBgorq6u8tBDD1ksfRQxbDrVtm1b0Wg04uvrW+imTIcPH5YBAwaIVqsVFxcXadq0qbzxxhtmX195woLx6xsxYoT4+vqKRqORu+++u9CltqtXr5bWrVuLRqORoKAgGTNmjNVAcPfddxe6oyZVPJWIDcefiIiqiYyMDDRs2BAffPCB2YmNcnJy0LVrV6jVanz77bemyZx3iomJwYABAwqdKFicuLg49OjRA19//bXZyoKa6MCBA2jXrh32799fovkuZHucs0BEZIW3tzdeffVVzJo1y2z1jIuLC3788UeoVCo0bdoUr732Gn777TecPXsWx44dw4oVK9C5c2eMGDHC5vt51FQzZ87EwIEDGRQUxJEFogLy8vJw9erVIvt4e3sXuccA1Qx5eXlYsGABFixYgISEBFO7i4sLBgwYgGnTppVrmR9HFsiecOkkUQE7duxAjx49iuyzbNkyjBw5snIKIrvl7OyMiRMnYuLEiThz5gwuXLgAFxcXNG/eHG5ubkqXR2RTHFkgKiAtLQ379u0rsk94eHihx6mJiKojhgUiIiIqEic4EhERUZGq9JwFvV6PixcvwtPTkycXISIiKgURwbVr11C7dm2zLd2tqdJh4eLFiwgNDVW6DCIioiorMTGx2JP6VemwYDwbXGJiYqGnTCUiIiJLmZmZCA0NNTuzamEUDQv169c3nYu9oJdeesniREbWGA89eHl5MSwQERGVQUkO4ysaFvbs2QOdTme6fvjwYfTu3dvsjHRERESkLEXDwp3nK585cyYaNWqEe++9V6GKiIiI6E52M2chLy8P//vf/zBx4sRCh0Ryc3PNzteemZlZWeURERHVWHYTFr755hukp6cXuY1udHQ0pk2bVnlFERHVUCKCW7dumR0qpqpFrVbD0dHRJlsL2M0Ojn369IGzszO+//77QvtYG1kIDQ1FRkYGJzgSEdlIXl4ekpKSkJWVpXQpVE5ubm4IDg6Gs7OzxW2ZmZnw9vYu0WeoXYwsnD17Flu2bEFsbGyR/TQaDTQaTSVVRURU8+j1eiQkJECtVqN27dpwdnbmpndVkIggLy8PKSkpSEhIQOPGjYvdeKkodhEWli1bhsDAQDz00EOK1qHTAfHxQFISEBwMREYCarWiJRERVaq8vDzo9XqEhoby7JlVnKurK5ycnHD27Fnk5eXBxcWlzI+leFjQ6/VYtmwZRowYAUdH5cqJjQXGjQPOn7/dVqcOMG8eEBWlWFlERIooz1+hZD9s9T4q/t2wZcsWnDt3DqNGjVKshthYYOBA86AAABcuGNqLOTpCRERUrSkeFu6//36ICJo0aaLI8+t0hhEFa9M8jW3jxxv6ERER1USKhwWlxcdbjigUJAIkJhr6ERFRyel0QFwcsHq14d+q9EdX/fr1MXfuXJs8VlxcHFQqFdLT023yeEpQfM6C0pKSbNuPiIiUmQfWvXt3tGnTxiYf8nv27IG7u3v5i6omavzIQnCwbfsREdV09joPzLjRVEkEBARwNUgBNT4sREYa0m5hy4hVKiA01NCPiKgmu3Gj8EtOjqFPSeaBjRtnfkjC2uOV1siRI7F9+3bMmzcPKpUKKpUKy5cvh0qlwk8//YS77roLGo0Gv//+O/7991888sgjCAoKgoeHBzp06IAtW7aYPd6dhyFUKhU+++wzDBgwAG5ubmjcuDG+++670heab/369QgPD4dGo0H9+vUxe/Zss9sXLVqExo0bw8XFBUFBQRg4cKDptpiYGERERMDV1RV+fn7o1asXbpTlRSuFGh8W1GrDsBhgGRiM1+fO5X4LREQeHoVfHnvM0Kck88DOnzefB1a/vuXjlda8efPQuXNnPPfcc0hKSkJSUhJCQ0MBAFOmTMHMmTNx9OhRtGrVCtevX0ffvn2xdetW/PXXX3jggQfQr18/nDt3rsjnmDZtGgYPHoy///4bffv2xbBhw3D16tVS17pv3z4MHjwYjz/+OA4dOoS3334bb7zxBpYvXw4A2Lt3L8aOHYvp06fj+PHj+Pnnn9GtWzcAQFJSEp544gmMGjUKR48eRVxcHKKiolDhmzFLFZaRkSEAJCMjo9yPtX69SJ06IoZvZcMlNNTQTkRUU2RnZ8s///wj2dnZFrcV/P1456VvX0OfVauK7me8rFp1+3H9/S1vL4t7771Xxo0bZ7q+bds2ASDffPNNsfcNDw+Xjz/+2HS9Xr168tFHHxX42iGvv/666fr169cFgPz000/FPraxjrS0NBERGTp0qPTu3dusz+TJk6VFixYiIrJ+/Xrx8vKSzMxMi8fat2+fAJAzZ84U+7wiRb+fpfkMrfEjC0ZRUcCZM8A339xuO3qUGzIRERldv174Zf16Q5+yzAM7c8by8Wypffv2ZtevX7+OSZMmoXnz5tBqtfDw8MDRo0eLHVlo1aqV6f/u7u7w8vJCcnJyqes5evQounbtatbWtWtXnDx5EjqdDr1790a9evXQsGFDPPnkk1i5cqXpPB2tW7dGz549ERERgUGDBuHTTz9FWlpaqWsoLYaFAtRqoH9/wDin5eJFZeshIrIn7u6FX4w7CZdlHpi1x7Nt3eYPOGnSJGzYsAEzZsxAfHw8Dhw4gIiICOTl5RX5OE5OTmbXVSoV9Hq9bYsF4Onpif3792P16tUIDg7Gm2++idatWyM9PR1qtRqbN2/GTz/9hBYtWuDjjz9G06ZNkZCQYPM6CmJYuINKZZijsGYNEBCgdDVERFWLkvPAnJ2dS3RK7T/++AMjR47EgAEDEBERgVq1auHMmTO2L6gQzZs3xx9//GFRU5MmTaDOf2EcHR3Rq1cvfPDBB/j7779x5swZ/PrrrwAMIaVr166YNm0a/vrrLzg7O2PDhg0VWnON32fBmueeU7oCIqKqKyoKiImxvs/C3LkVd3i3fv36+PPPP3HmzBl4eHgU+ld/48aNERsbi379+kGlUuGNN96okBGCwrzyyivo0KED3nnnHQwZMgQ7d+7EggULsGjRIgDADz/8gNOnT6Nbt27w8fHBjz/+CL1ej6ZNm+LPP//E1q1bcf/99yMwMBB//vknUlJS0Lx58wqtmSMLRERkc8Z5YNu2AatWGf5NSKjYeWCTJk2CWq1GixYtEBAQUOgchDlz5sDHxwddunRBv3790KdPH7Rr167iCrtDu3btsG7dOqxZswYtW7bEm2++ienTp2PkyJEAAK1Wi9jYWNx3331o3rw5PvnkE6xevRrh4eHw8vLCb7/9hr59+6JJkyZ4/fXXMXv2bDz44IMVWrNKpKLXW1SczMxMeHt7IyMjA15eXjZ73KQkYM8ew9yFXr1s9rBERHYvJycHCQkJaNCgQblOaUz2oaj3szSfoRxZsGLLFuCRR4DoaKUrISIiUh7DghX16hn+LWYVDREREV588UV4eHhYvbz44otKl2cTnOBoRd26hn/PnQP0esCBkYqIiAoxffp0TJo0yepttjxEriSGBStCQgwBIS8PSE4GatVSuiIiIrJXgYGBCAwMVLqMCsW/ma1wcgJq1zb8n4ciiIiopmNYKITxUMTZs8rWQUREpDSGhUJwkiMREZEB5ywU4oUXDMsnO3RQuhIiIiJlMSwU4t57la6AiIjIPvAwBBERVQidCOLS0rD68mXEpaVBZ+cbBtevXx9z584tUV+VSoVvvvmmQuuxJxxZKERODrBpk2Hr5xdeULoaIqKqJTYlBeNOncL53FxTWx2NBvPCwhDFU/pWORxZKERenmHOwosvAtevK10NEVHVEZuSgoFHjpgFBQC4kJuLgUeOIDYlRaHKqKwYFgrh5QVotYb/c0UEEdVkIoIbOl2JLpm3bmHsyZOwdsDB2Dbu1Clk3rpV7GOV5jyHS5cuRe3atS1ONf3II49g1KhR+Pfff/HII48gKCgIHh4e6NChA7Zs2VL2F+UOhw4dwn333QdXV1f4+fnh+eefx/UCf2nGxcWhY8eOcHd3h1arRdeuXXE2f23+wYMH0aNHD3h6esLLywt33XUX9u7da7PabIGHIYpQty6Qnm4ICy1aKF0NEZEysvR6eMTH2+SxBMD53Fx4//57sX2vR0bCXa0u0eMOGjQIL7/8MrZt24aePXsCAK5evYqff/4ZP/74I65fv46+ffvivffeg0ajwYoVK9CvXz8cP34cdY0b65TRjRs30KdPH3Tu3Bl79uxBcnIynn32WYwZMwbLly/HrVu38Oijj+K5557D6tWrkZeXh927d0OlUgEAhg0bhrZt22Lx4sVQq9U4cOAAnJycylWTrTEsFKFePeDvv7kxExGRvfPx8cGDDz6IVatWmcJCTEwM/P390aNHDzg4OKB169am/u+88w42bNiA7777DmPGjCnXc69atQo5OTlYsWIF3N3dAQALFixAv3798P7778PJyQkZGRl4+OGH0ahRIwBA8+bNTfc/d+4cJk+ejGbNmgEAGjduXK56KgLDQhEKnlCKiKimcnNwwPXIyBL1/S09HX0PHSq2348REehmPNZbxPOWxrBhw/Dcc89h0aJF0Gg0WLlyJR5//HE4ODjg+vXrePvtt7Fx40YkJSXh1q1byM7Oxjkb/II/evQoWrdubQoKANC1a1fo9XocP34c3bp1w8iRI9GnTx/07t0bvXr1wuDBgxEcHAwAmDhxIp599ll89dVX6NWrFwYNGmQKFfaCcxaKwF0ciYgMywTd1eoSXe739UUdjQaqwh4LQKhGg/t9fYt9LOMwfUn169cPIoKNGzciMTER8fHxGDZsGABg0qRJ2LBhA2bMmIH4+HgcOHAAERERyMvLK9+LU0LLli3Dzp070aVLF6xduxZNmjTBrl27AABvv/02jhw5goceegi//vorWrRogQ0bNlRKXSXFsFAEnh+CiKh01CoV5oWFAYBFYDBenxsWBnUpg0BJuLi4ICoqCitXrsTq1avRtGlTtGvXDgDwxx9/YOTIkRgwYAAiIiJQq1YtnDlzxibP27x5cxw8eBA3btwwtf3xxx9wcHBA06ZNTW1t27bF1KlTsWPHDrRs2RKrVq0y3dakSRNMmDABmzZtQlRUFJYtW2aT2myFYaEIXbsCq1cDJdyjg4iIAEQFBCAmPBwhGo1Zex2NBjHh4RW6z8KwYcOwceNGfPHFF6ZRBcAwDyA2NhYHDhzAwYMHMXToUIuVE+V5ThcXF4wYMQKHDx/Gtm3b8PLLL+PJJ59EUFAQEhISMHXqVOzcuRNnz57Fpk2bcPLkSTRv3hzZ2dkYM2YM4uLicPbsWfzxxx/Ys2eP2ZwGe6D4nIULFy7gtddew08//YSsrCyEhYVh2bJlaN++vdKloU4d4PHHla6CiKjqiQoIwCP+/ohPT0dSXh6CnZ0RqdVWyIhCQffddx98fX1x/PhxDB061NQ+Z84cjBo1Cl26dIG/vz9ee+01ZGZm2uQ53dzc8Msvv2DcuHHo0KED3Nzc8Nhjj2HOnDmm248dO4Yvv/wSqampCA4OxujRo/HCCy/g1q1bSE1NxVNPPYXLly/D398fUVFRmDZtmk1qsxWVlGYhq42lpaWhbdu26NGjB/7zn/8gICAAJ0+eRKNGjUo0uSMzMxPe3t7IyMiAl5dXJVRMRFS95eTkICEhAQ0aNICLi4vS5VA5FfV+luYzVNGRhffffx+hoaFmx2YaNGigYEWWfv0V+OcfoG9foGFDpashIiKqfIrOWfjuu+/Qvn17DBo0CIGBgWjbti0+/fTTQvvn5uYiMzPT7FLRpr0jePmzNMw5ePtEKDodEBdnmM8QFwfodBVeBhERVZKVK1fCw8PD6iU8PFzp8hSh6MjC6dOnsXjxYkycOBH//e9/sWfPHowdOxbOzs4YMWKERf/o6OhKPY4Tm5KCvZNOAe65WAhg4UHAT6cBFoQh9ZvbE3Tq1AHmzQOioiqtNCIiqiD9+/fH3XffbfU2e9tZsbIoOmfB2dkZ7du3x44dO0xtY8eOxZ49e7Bz506L/rm5ucgtcGKSzMxMhIaGVsicBeOJUERgvv5Hn3/9rXAg3hAYjPN1YmIYGIioauOcheqlWsxZCA4ORos7TrrQvHlzrF+/3mp/jUYDzR1LcSqCTgTjTp0ynPTkzom7DjAEhvEnAGc9cMUZckgL6FV48UUgOxsICQEiI4ESbmlORGR3FPw7kmzIVu+jomGha9euOH78uFnbiRMnUM+4dWIl04kgPj0dW9PSLE6tasYBgO9N4PWjhuvJhkMTKfEBGD7c0MRDE0RUFRmH2bOysuDq6qpwNVReWVlZAMp/+ETRsDBhwgR06dIFM2bMwODBg7F7924sXboUS5curfRaYlNSMO7UqaJDQmH8c4FpR8wOTVy4AAwcyEMTRFS1qNVqaLVaJCcnAzDsEVDabZdJeSKCrKwsJCcnQ6vVQl3OoW5F5ywAwA8//ICpU6fi5MmTaNCgASZOnIjnnnuuRPe11T4LpvkJZX4EGA5NpGiAoZ0AveEHS6UyjDAkJPCQBBFVHSKCS5cuIT09XelSqJy0Wi1q1aplNfCV5jNU8bBQHrYICzoR1N+1q2wjCtaMbw0c9DFr2rYN6N7dNg9PRFRZdDodbt68qXQZVEZOTk5FjihUmQmO9iA+Pd12QQEA/CzPYJaUZLuHJyKqLGq1utzD11Q91PgTSSXZ+vSkqc4WTfmnLCciIqqSanxYCHa2/HC36qtQIM3JMDfBGj2AyxrgkNbUpFIBoaGGZZRERERVVY0PC5FaLepoNBbbKZgYQ8DyhsBHTQz7LtwZGIwbNS0MM5vcCBhOb81RPCIiqspqfFhQq1SYFxYGwHL/JYsQEB8AvB0OXDHfGMpPNPCbf3vZJGBYBcFlk0REVB3U+LAAGM67HhMejhBN8SEgNCEA6xw7YUvr1qbZoTs7t8blmAD88svtEYUdOxgUiIioeqjxqyGMogIC8Ii/P+LT05GUl4dgZ2dEarVAdxXi4w0rGoKDjds4qwD4oLm7Ow7duIGT2dlo7OaG++8HwsKAkyeBEycMowtERERVHcNCAWqVCt19fO5oLHyPhGZubjh04waOZmWhr5+foa2ZISwcOwbcd1/F1ktERFQZGBbKobmbGwDgWP7e24Dh0EOzZkDbtkpVRUREZFsMC+XQLD8sHL1xw9Q2cqRCxRAREVUQTnAsB2sjC0RERNUNw0I5NHFzgwpA6q1bSCmwE2RyMrB9O5CdrVxtREREtsKwUA5uajXqubgAMB9daN3aMCnyyBGFCiMiIrIhhoVyMs1bKBAWmjUz/Hv0qBIVERER2RbDQjlZm7dgDAvHjilRERERkW0xLJSTtZGF5s0N/3JkgYiIqgOGhXLiyAIREVV3DAvlZBxZOJuTgyydztCWHxZOngRu3lSqMiIiIttgWCinAGdn+Dk6QgCcyB9dqFMHcHcHbt0CTp9Wtj4iIqLy4g6ONtDMzQ1/ZGbiaFYW2nh6wsEBeOstwMsLyD9lBBERUZXFsGADzd3d8Udmptm8hcmTFSyIiIjIhngYwgasrYggIiKqLhgWbMDaioicHGDnTuC775SqioiIyDZ4GMIGjCMLJ7KyoBOBWqXC6dNAly6GeQvp6YBKpWyNREREZcWRBRuo5+ICFwcH5IogIf/sUY0aAWo1kJkJJCUpXCAREVE5MCzYgFqlQhNXVwC3D0VoNEDDhobbuTkTERFVZQwLNtKc2z4TEVE1xbBgI8aRhY2pqYhLS4NOBE2aGG777jsgLg7I3+CRiIioSmFYsIHYlBQsvHgRALA9IwM9Dh5E0K+7sPhwCgBg0yagRw+gfn0gNlbBQomIiMqAYaGcYlNSMPDIEVy9dcusPVWVixuvHgEiU0xtFy4AAwcyMBARUdWiaFh4++23oVKpzC7NjGdhqgJ0Ihh36hTE2o0OAATA6FOAg6GH5HccP56HJIiIqOpQfGQhPDwcSUlJpsvvv/+udEklFp+ejvO5uYV3cAAQlAtEpJuaRIDERCA+vsLLIyIisgnFN2VydHRErVq1lC6jTJLy8krW0c+yH/deICKiqkLxkYWTJ0+idu3aaNiwIYYNG4Zz584V2jc3NxeZmZlmFyUFOzuXrGOqZb/gYBsXQ0REVEEUDQt33303li9fjp9//hmLFy9GQkICIiMjce3aNav9o6Oj4e3tbbqEhoZWcsXmIrVa1NFoUOhOznoAlzXAIa2pSaUCQkOByMhKKJCIiMgGVCJidX6eEtLT01GvXj3MmTMHzzzzjMXtubm5yC0wRyAzMxOhoaHIyMiAl5dXZZZqYlwNAcB8oqMegArAW+FAfACA2+eHiIkBoqIqs0oiIiJzmZmZ8Pb2LtFnqOKHIQrSarVo0qQJTp06ZfV2jUYDLy8vs4vSogICEBMejhCNxqzdTzTwm387KABAnToMCkREVPXYVVi4fv06/v33XwRXsQP6UQEBONOpExY2bgwACHBywuX7OuFyTAA++sjQJyQESEhgUCAioqpH0bAwadIkbN++HWfOnMGOHTswYMAAqNVqPPHEE0qWVSZqlQr9/fwAAFdv3jS0qYGHHzbcnpYGONhVNCMiIioZRZdOnj9/Hk888QRSU1MREBCAe+65B7t27UJAQEDxd7ZDtZyd4QBAByA5Lw/BGg1CQgy3ZWUBGRmAVqtggURERGWgaFhYs2aNkk9vc44ODqjl7IyLeXm4kJuLYI0Grq6Ary9w9aphu2eGBSIiqmoU35SpugnRaAxhIS8P7fPbPvkEcHc3LJkkIiKqahgWbCxEo8Gea9fMtoEeNEjBgoiIiMqJU+5srE7+EsoLRZ0zgoiIqArhyIKNheRvAV0wLJw+DezYAfj7Aw88oFRlREREZcORBRsLsTKysHUr8OSTwIIFSlVFRERUdgwLNmYMCwXnLBiXT54/r0RFRERE5cOwYGOmOQsFTl9dp47h3wsXlKiIiIiofBgWbMw4snBdp0PmrVuGtvyRhStXgJwcpSojIiIqG4YFG3NXq+GtVgO4PW/B1xcwnmfq4kWlKiMiIiobhoUKcOe8BZWKhyKIiKjqYlioANb2WjAeimBYICKiqob7LFSAECuTHN96yzBf4a67lKqKiIiobBgWKoC1vRbuu0+paoiIiMqHhyEqgHEXx/Pc8pmIiKoBjixUAGtzFpKTgV9+AUSAp55SqjIiIqLSY1ioANYOQ/z7ryEk1K3LsEBERFULD0NUAGNYSL55E3l6PYDbSycvXgTym4iIiKoEhoUK4O/kBCeVCgIgKX9FRK1ahv0Wbt0yHJIgIiKqKhgWKoCDSoXad5yq2snJEBgA7rVARERVC8NCBeHGTEREVF0wLFQQa5McGRaIiKgqYlioIHeeHwK4Pcnx/HklKiIiIiobLp2sINa2fH7uOeChh4DwcKWqIiIiKj2GhQpibc5C69aGCxERUVXCwxAVJOSO1RBERERVFcNCBSk4wVFEAADZ2cCKFcCsWUpWRkREVDoMCxWkdn5YyBVB6s2bAACdDhgxAnj1VeDaNSWrIyIiKjmGhQqicXBAgJMTgNuTHD08AC8vw+1cPklERFUFw0IF4l4LRERUHTAsVCDjJEfutUBERFWZ3YSFmTNnQqVSYfz48UqXYjMcWSAiourALsLCnj17sGTJErRq1UrpUmyqqPNDcGSBiIiqCsXDwvXr1zFs2DB8+umn8PHxUbocmwrOPwyx59o1xKWlQSeC4GDDbX/+CcTFGVZIEBER2TPFw8Lo0aPx0EMPoVevXsX2zc3NRWZmptnFXsWmpGBqQgIA4O8bN9Dj4EEE/boLr29KAQDs3w/06AHUrw/ExipYKBERUTEUDQtr1qzB/v37ER0dXaL+0dHR8Pb2Nl1CQ0MruMKyiU1JwcAjR3Alf38Fo1RVLtLHHwEiU0xtFy4AAwcyMBARkf1SLCwkJiZi3LhxWLlyJVxcXEp0n6lTpyIjI8N0SUxMrOAqS08ngnGnTkGs3egAQACMPgU4GHrkb+6I8eN5SIKIiOyTYmFh3759SE5ORrt27eDo6AhHR0ds374d8+fPh6OjI3RWPjk1Gg28vLzMLvYmPj3dbKmkBQcAQblARLqpSQRITATi4yu8PCIiolJT7KyTPXv2xKFDh8zann76aTRr1gyvvfYa1Gq1QpWVT1KBU1IXyc+yX1KSjYshIiKyAcXCgqenJ1q2bGnW5u7uDj8/P4v2qsS4AqJYqZb9jCsliIiI7IniqyGqm0itFnU0GqgK66AHcFkDHNKamlQqIDQUiIyshAKJiIhKSbGRBWvi4uKULqHc1CoV5oWFYeCRI1AB5hMdBYAKwMIwQG8eJ+bOBarokRciIqrmOLJQAaICAhATHm7a7tnIBWr4zQ8H4gNMbR4eQEwMEBVV2VUSERGVDMNCBYkKCMCZTp2wrXVrTMo/e5SfRo1LX/tj2zbDUkkA8PcHBgxQrk4iIqLiMCxUILVKhe4+PpjeoAHcHRxwIS8PB7Ovo3t34N13ARcXQK8HkpOVrpSIiKhwDAuVwFWtxgO+vgCAb65cAQC4uwP//AOcOQMEBSlYHBERUTEYFirJo/7+AG6HBQBo0MCwEoKIiMieMSxUkr5+flADOHzjBv7Nzja77dYt4I4mIiIiu8GwUEl8nZxwr1YLAPi2wOjCzJlAYCDw+ecKFUZERFQMu9pnobp71N8fv6anY/mlSwh2dkawszMcHLVIS1Nh2TLAz8+wi2NkJPdcICIi+1GmkYUvv/wSGzduNF1/9dVXodVq0aVLF5w9e9ZmxVU3zg6Gl/vQjRsYevQoehw8iOhWu4DIFOzfDwwdCvToAdSvz1NWExGR/ShTWJgxYwZcXV0BADt37sTChQvxwQcfwN/fHxMmTLBpgdVFbEoK/nPihEV7umMuMO0IEJliartwARg4kIGBiIjsQ5nCQmJiIsLCwgAA33zzDR577DE8//zziI6ORjzPs2xBJ4Jxp06Zb/1s5ADDNtCjTwEOhh6S33H8eMDKmbqJiIgqVZnCgoeHB1JTUwEAmzZtQu/evQEALi4uyOa0fgvx6ek4n5tbeAcHAEG5QES6qUkESEwEmL2IiEhpZZrg2Lt3bzz77LNo27YtTpw4gb59+wIAjhw5gvr169uyvmohKS+vZB39LPslJdm4GCIiolIq08jCwoUL0blzZ6SkpGD9+vXw8/MDAOzbtw9PPPGETQusDoKdnUvWMdWyX3CwjYshIiIqJZWIWD2UXhVkZmbC29sbGRkZ8PLyUrqcQulEUH/XLlzIzbU+b0EPIEUDDO1kOnW1SgXUqQMkJHAZJRER2V5pPkPLNLLw888/4/fffzddX7hwIdq0aYOhQ4ciLS2tLA9ZralVKszLnxBqsbuzPr9xYZhZUACAuXMZFIiISHllCguTJ09GZmYmAODQoUN45ZVX0LdvXyQkJGDixIk2LbC6iAoIQEx4OEI0GrN2DzjCb344EB9gagsMBGJigKioyq6SiIjIUpnCQkJCAlq0aAEAWL9+PR5++GHMmDEDCxcuxE8//WTTAquTqIAAnOnUCdtat0bf/LNQDq7tj8sxAdi2DchfVIKoKAYFIiKyH2VaDeHs7IysrCwAwJYtW/DUU08BAHx9fU0jDmSdWqVCdx8fZOn1+PHqVfyang61GujeHUhLAzZvBn79VekqiYiIbitTWLjnnnswceJEdO3aFbt378batWsBACdOnECdOnVsWmB11c3bG44qFc7k5OB0djYaurqie3fAwQE4fhw4f94wwZGIiEhpZToMsWDBAjg6OiImJgaLFy9GSEgIAOCnn37CAw88YNMCqysPR0d0yp99uiV/UqiPD3DXXYbbt25VqjIiIiJzZRpZqFu3Ln744QeL9o8++qjcBdUkPbVa/J6Rga1paXi+dm1DW09gzx5DWBgxQuECiYiIUI5TVOt0OnzzzTc4evQoACA8PBz9+/eHmmv9SqyXjw+mnT2LX9PToReBg0qFXr2AmTMNYUHk9jJKIiIipZQpLJw6dQp9+/bFhQsX0LRpUwBAdHQ0QkNDsXHjRjRq1MimRVZXHb284O7ggCs3b+Lv69fRxtMTXboAGg1w8SJw4gSQ//ISEREppkxzFsaOHYtGjRohMTER+/fvx/79+3Hu3Dk0aNAAY8eOtXWN1ZazgwPu1WoBAFvT0wEArq7A118Dp04BTZooVxsREZFRmcLC9u3b8cEHH8A3f68AAPDz88PMmTOxfft2mxVXE/T08QFwe5IjAPTrBzRqxEMQRERkH8p0GEKj0eDatWsW7devX4dzSU+aRABuh4VtaWn46tIlhGo0iNRqAb0K8fGGs04GBwORkdz6mYiIlFGmsPDwww/j+eefx+eff46OHTsCAP7880+8+OKL6N+/v00LrO5OZmXBAUCuCJ46dgwA4KfTIOfDMNz4+fYW0HXqAPPmcWdHIiKqfGU6DDF//nw0atQInTt3houLC1xcXNClSxeEhYVh7ty5Ni6x+opNScHgf/6B/o72VFUubrx6BIhMMbVduAAMHAjExlZujUREROU6RfWpU6dMSyebN2+OsPwzK1aWqnKKamuMp60+n5trvQNPW01ERBWoNJ+hJT4MUdzZJLdt22b6/5w5c0r6sDVWfHp64UEBMIz5BOUCEenAQcO8BhEgMRGIjzecS4KIiKgylDgs/PXXXyXqpyrFFP7Fixdj8eLFOHPmDADDxk5vvvkmHnzwwRI/RlWVlJdXso5+lv2SkmxcDBERURFKHBYKjhzYSp06dTBz5kw0btwYIoIvv/wSjzzyCP766y+Eh4fb/PnsSXBJV42kWvYLDrZxMUREREUo15yFiuDr64tZs2bhmWeesbgtNzcXuQWG7jMzMxEaGlql5yxcyM2F1TeAcxaIiKgClWbOQplWQ1QEnU6HNWvW4MaNG+jcubPVPtHR0fD29jZdQkNDK7lK21GrVJiXPyHU4sCNPr9xYZhZUACAuXMZFIiIqHIpPrJw6NAhdO7cGTk5OfDw8MCqVavQt29fq32r08iCUWxKCsadOmU22dFPpwEWhCH1m9v7LISGGoIC91kgIiJbKM3IguJhIS8vD+fOnUNGRgZiYmLw2WefYfv27WjRokWx963KSycL0ong6+RkPHH0KNQAbkRGwhFq7uBIREQVpkqFhTv16tULjRo1wpIlS4rtW13CAgDoReAeH48cvR4nO3ZEmJub0iUREVE1ViXnLBjp9XqzQw01hYNKhcaurgCA49nZpvYpU4D27QGen4uIiJSiaFiYOnUqfvvtN5w5cwaHDh3C1KlTERcXh2HDhilZlmKa5o8mHM/KMrUdPQrs2wccPqxUVUREVNOV6URStpKcnIynnnoKSUlJ8Pb2RqtWrfDLL7+gd+/eSpalmKb5IwsnCoSFpk0N/x4/rkRFRERECoeFzz//XMmntztNjCMLBQ5DMCwQEZHS7G7OQk1m7TAEwwIRESmNYcGOGA9DJOXl4dqtW4a2/LBw7hxQYMCBiIio0jAs2BGtkxMCnZwAACfyk4G/P6DVGs44efKkgsUREVGNxbBgZ5rccShCpQLCw4HGjYHMTCUrIyKimkrRCY5kqamrK37PyDCbt/Dbb4ADYx0RESmEH0F2xjjJ8USBCQoMCkREpCR+DNkZaysiiIiIlMSwYGdMIwtZWTCetuPiRaBTJ6BhQ8NERyIiosrEsGBnGri4GM48qdfjQv45Mnx9gd27gYQEIDlZ2fqIiKjmYViwM84ODmho3PY5f96CiwtQr57h9hMnlKqMiIhqKoYFO8SdHImIyJ4wLNgh406ODAtERGQPGBbsEE8oRURE9oRhwQ4VXBFhamNYICIihTAs2CHjYYgzOTnI1esNbU2BBg0M/3L5JBERVSZu92yHgpyd4aVWI1Onw6nsbIS7u6NOHeD0aaUrIyKimogjC3ZIpVJZnFCKiIhIKQwLdqqxiwsAYF1yMuLS0qATgU4HxMUB//uf4V+dTtESiYiohuBhCDsUm5KCjVevAgDWpqRgbUoK/HQa5HwYhhs/B5j61akDzJsHREUpVSkREdUEHFmwM7EpKRh45Agy7xg2SFXl4sarR4DIFFPbhQvAwIFAbGxlV0lERDUJw4Id0Ylg3KlTsLrYwQGAABh9CnAw9DCuihg/nockiIio4jAs2JH49HSczz95lFUOAIJygYh0U5MIkJgIxMdXeHlERFRDMSzYkaS8vJJ19LPsl5Rk42KIiIjyMSzYkWBn55J1TLXsFxxs42KIiIjyMSzYkUitFnU0GqgK66AHcFkDHNKamlQqIDQUiIyshAKJiKhGYliwI2qVCvPCwgDAMjDo8xsXhgF6w62q/E5z5wJqdSUVSURENQ7Dgp2JCghATHg4QjQas3Y/0cBvfjgQb77PQkwM91kgIqKKxU2Z7FBUQAAe8fdHXFoaHvj7b9wCsKNzazTq7ob4eMNkxuBgw6EHjigQEVFFY1iwU2qVCj19fdHKwwP7r1/HPzduoImbG7p3N9yekQFkZgI+PoqWSURENQAPQ9i5CHd3AMDfN26Y2saOBbRaYPFihYoiIqIaRdGwEB0djQ4dOsDT0xOBgYF49NFHcfz4cSVLsjutPDwAAH9fv25qCwkx/HvokBIVERFRTaNoWNi+fTtGjx6NXbt2YfPmzbh58ybuv/9+3CjwV3RN18rKyEKrVoZ///5biYqIiKimUXTOws8//2x2ffny5QgMDMS+ffvQrVs3haqyLxH5IwunsrORpdPBTa1GRIThtuPHgdxc4I6FE0RERDZlV3MWMjIyAAC+vr5Wb8/NzUVmZqbZpboLcnZGoJMTBMCR/NGFkBDDnAWdDjh6VNHyiIioBrCbsKDX6zF+/Hh07doVLVu2tNonOjoa3t7epktoaGglV6kM07yF/LCgUt0+FMF5C0REVNHsJiyMHj0ahw8fxpo1awrtM3XqVGRkZJguiYmJlVihcowrIg4VmORoPBTBeQtERFTR7GKfhTFjxuCHH37Ab7/9hjp16hTaT6PRQFMDD9Bbm+R4//3AzZsAp3YQEVFFUzQsiAhefvllbNiwAXFxcWjQoIGS5ditgssnRQQqlQr9+wP9+ytcGBER1QiKhoXRo0dj1apV+Pbbb+Hp6YlLly4BALy9veHq6qpkaXaluZsbHACk3rqFS3l5CK6BoytERKQcRecsLF68GBkZGejevTuCg4NNl7Vr1ypZlt1xVavRxM0NgPmhiOxsYN8+4OJFpSojIqKaQNGwICJWLyNHjlSyLLtkmrdQYJLj0KFA+/bA118rVRUREdUEdrMagopm7RwRxhWmXD5JREQViWGhijBOciy4fJLbPhMRUWVgWKgijIch/snKwk29HsDtvRaOHDHs5khERFQRGBaqiHouLvBUq3FTBMezsgAAYWGAiwuQlQWcPq1wgUREVG0xLFQRKpUKLfNXRCy5eBFxaWlQqQUtWhhuX7QIiIvjCAMREdkew0IVEZuSYprcuODiRfQ4eBBBv+7CYd8UAMDcuUCPHkD9+kBsrHJ1EhFR9cOwUAXEpqRg4JEjuJE/V8EoVZWLvP8eASJTTG0XLgADBzIwEBGR7TAs2DmdCMadOgWxdqMDAAEw+hTgYOgh+R3Hj+chCSIisg2GBTsXn56O87m5hXdwABCUC0Skm5pEgMREID6+wssjIqIagGHBziXl5ZWso59lv6QkGxdDREQ1EsOCnQt2di5Zx1TLfsHBNi6GiIhqJIYFOxep1aKORgNVYR30AC5rgENaU5NKBYSGApGRlVAgERFVewwLdk6tUmFeWBgAWAYGfX7jwjBAb7hVld9p7lxAra6kIomIqFpjWKgCogICEBMejhCNxqzdE07wmx8OxAeY2vz9gZgYICqqsqskIqLqimGhiogKCMCZTp2wrXVr9NRqAQBPhwbickwAtm27fcjh6acZFIiIyLYYFqoQtUqF7j4+eL52bQDA9vR0qNVA9+7AM88Y+mzfrlx9RERUPTEsVEH35o8s/H3jBq7evAnAEBgAYN8+IH9XaCIiIptgWKiCgpyd0dzNDQLD6AIA1KsHbNgAnD8P5J/NmoiIyCYYFqqoHvmjC3H5YQEAHn0UCApSpBwiIqrGGBaqqO5WwgIREVFFYFioogrOW7iSvyW0TgfMmAH06QNkZChYHBERVSsMC1VUoLMzwt3cAAC/5ScDtRpYtgzYtAn4/XclqyMiouqEYaEK6+HjAwDYVuBQhHFVxLZtlV8PERFVTwwLVZi1eQs9ehj+jYur9HKIiKiaYliowu719gYAHL5xA0suXEBcWhoi7xUAwP79wGefGUKDTqdgkUREVOU5Kl0Ald1vGRlwUqlwUwQvnjwJAPDTaeDQPQz6uAA895yhX506wLx53AaaiIjKhiMLVVRsSgoGHjmCmyJm7amqXOjfPAJEppjaLlwABg4EYmMru0oiIqoOGBaqIJ0Ixp06BbF2owMAATD6FOBg6GHME+PH85AEERGVHsNCFRSfno7zubmFd3AAEJQLRKSbmkSAxEQgPr7CyyMiomqGYaEKSsrfhKlYfpb9kpJsXAwREVV7ioaF3377Df369UPt2rWhUqnwzTffKFlOlRHs7FyyjvVuAK3TTIcjAOCff7hCgoiISkfRsHDjxg20bt0aCxcuVLKMKidSq0UdjQaqwjoYs8FT54C5B4HVu0wTHt9917AXQ/36nPBIREQlo2hYePDBB/Huu+9iwIABSpZR5ahVKswLCwMAy8AgVhr9c4FpXCFBRERlU6XmLOTm5iIzM9PsUlNFBQQgJjwcIRpN8Z25QoKIiMqhSoWF6OhoeHt7my6hoaFKl6SoqIAAnOnUCdtat8brdesaGgs7NmFcITEiwTSPwbhC4u23OY+BiIgKV6XCwtSpU5GRkWG6JCYmKl2S4tQqFbr7+KCFu3vJ7sB5DEREVEpVKixoNBp4eXmZXcigxCskjDiPgYiISqhKhQUqXLErJO7EeQxERFRCioaF69ev48CBAzhw4AAAICEhAQcOHMC5c+eULKtKKnKFRGG40yMREZWAomFh7969aNu2Ldq2bQsAmDhxItq2bYs333xTybKqrFKtkCiIOz0SEVERFD1Fdffu3SFi9XRIVEZRAQF4xN8f8enp2JqWhndLMkpj3OnxkBbQG8YljDs9RkYCanWFlkxERHZOJVX40zozMxPe3t7IyMjgZEcrdCKov2sXLuTmWj9D5Z0bOCVrgAVhQHyAqalOHWDePCAqqoKLJSKiSlWaz1BOcKzGSj2PgSskiIjICoaFaq7QeQzWtoXmCgkiIrKCYaEGKNNOj1whQURE+RSd4EiVx7jTY1Ke5coHq7rlH4ooMOlx/XpDEyc9EhHVLBxZqGFKvNNj1EWLbaEXLOC20ERENRHDQg1T7E6Pdy6b4KRHIqIaj2GhhilyhQQnPRIRkRUMCzVQoSskOOmRiIisYFiooQqukBhTu3bJ7tQtxbDTo8PtYxXr1xt2euQIAxFR9cWwUIMZV0g8FhBQfGeAkx6JiGoobvdMpd8WWp9/fVl94IIrkOoMHNYCOhWmTQMaNwaCg7nEkojInpXmM5RhgQAAsSkpGHjkCIA7FkRYm/RorZ3nlSAiqlIYFqhMYlNSMO7UKZzPzS39nTnaQERUpTAsUJnpRBCfno71KSlYcPFi6e7M0QYioiqDYYHKLS4tDT0OHizfg3C0gYjIbpXmM5TnhiCrjDs9FjrpsSSMGzqNOnO7LX+04a23ONpARFRVcGSBClXopMfy4GgDEZFd4GEIshmrkx4LWyFRUpzbQESkOIYFsinjpMekvDyczMrC22fPAuBoAxFRVcawQBWq0kYbFjYCMpwAvzwg1RkhV7X4aLYKAQFAUhIDBBFReTAsUIWrtNEGHq4gIqoQDAtU6SpltIGHK4iIbIZhgRRR4aMNxgfj4QqiynHgADB1KhAdDbRpo3Q1ZGPcZ4EUYTyLpVFLDw/bjzbceV//XODtf8zaLyRrMHiBZYCYPRtICkjHv2l5aOTjjJfu0UKtUiE+nqGCyKr164GffwY6dGBYqOE4skAVSpHRBmvzHTLyc7H3LVOTwxUNXJc3wo3zDBVEVrVpAxw8aPj3r7+UroZsjIchyG5VyNwGa+58TON3eQWHCgBY9Pvttmc7eeOzXRkMHlT1XL4M1Kplfj0wULl6yOYYFsiulWi0oSIChDU2DBWqTEObeN1ugw5AgSBQnuDBtsLDFhxuf08FOzsjUquFTlf2+0Jv20BX8Hu+op7D5lasAEaMML/+5JPK1UM2x7BAVUqhow1A5QSGkihJqCiszUbBg23Ww5Zvw5tQjf4Xqerb3z8eOkdkZQF6z9Lf10+ngSxshKunbRPoaje7iVdO/4vzeRX3HBUSyoY+DtX69XDQ66BTq4HHBkK3cnWxI2dK1VvT2pwdy//LkWGBqpw7//K6cvMmJpwy/wVbaaMNFa08waOmt1kLW7a+r40DXWU8R1naaqekICgtDaobhuEMcdeZ+jlcV2Pr62PhnZ11uzw3d/R8Zx70Hrf7QQ/DCeMAXPbxQZImuMLqZdvtNnWqBhOdw/BBv9t7zpRFlQsLCxcuxKxZs3Dp0iW0bt0aH3/8MTp27Fjs/RgWqje7OlxB9sNa2Crp90BJ72vrQFcZz1HKti0TJ6JnEZMW9SoVHAp8PNx5/U5b2rVD7w9nV+rXUGPb8sPm5Ovh5QoMpfkMdSjzs9jI2rVrMXHiRLz11lvYv38/WrdujT59+iA5OVnp0khhxqWYTwQF4c0GDRATHo4Qjcay452/vxSPv1Sh7vzgLU1YLOl9rfUrT1tlPEcp2z7p3x9XPTwK/XG5MxgUFhQEwFUPDyzp16/Sv4Ya2+YAQIA5eaeQd6tyfuEpPrJw9913o0OHDliwYAEAQK/XIzQ0FC+//DKmTJlS5H05slDzlPhwBVD2v/iIaoiAtDQs/ugjPBYfX+zIwZ2M/ddHRuI/EyYgpcAeK1R5PkJrjO9ette+ymzKlJeXh3379mHq1KmmNgcHB/Tq1Qs7d+606J+bm4vcApPgMjMzK6VOsh93bvwEAAMCAooNEB56ywlvAEo+FMxQQdVQio8PBk6fjkHbtuGTOXPglZUFR72+2PvdcnBAppsbXpw4EV/36FEJlVJh/k3Lq5TnUTQsXLlyBTqdDkFBQWbtQUFBOHbsmEX/6OhoTJs2rbLKoyqiJAHC2lK62s0tZ6mXO1RYa2PwIDv3dY8eiGvTBstnzsSDu3cX+S0qADa1b4+RU6ZwNMEONPJxrpTnqVLbPU+dOhUTJ040Xc/MzERoaKiCFZG9shYg1I6wGK57LMh2oUJ1zcpMeD3M9lkwdIB9TJKqqm3lmSxY0vvaOtBVgdCY4uODfU2bovfevXAqYnRB5+CAvU2bMigoTQ+o0zR46RFtpTydomHB398farUaly9fNmu/fPkyahXcOSyfRqOBxtoEN6IysmWoeKmvFsAdOzh2NV+HXp7gwbZ8he1dUVCGo6HNuwz3tdbPFss9K/I5bNTWb8cOqIs5DOGg16Pfzp14a9Qou/waakRb/vfUROcwm+y3UBJ2McGxY8eO+PjjjwEYJjjWrVsXY8aM4QRHqpas7eZnbbdBgG13tpV0g6M6aVp8+KH5Bkclva9fw5tACTZ5Ks9GUrZ+Dlu0Bd5KxqXHHkNBxkmM1iY/Bq1fj2RvX7ORM3vai6A6t9XIfRbWrl2LESNGYMmSJejYsSPmzp2LdevW4dixYxZzGe7EsEBU85Rn6+SS3rek20cDFb9FdUmfo7xt215fjD7vjza9VqJWI9fVA9v6PoMeP34OTfZ1qHS3N2T6ZcoidH3rBe7gqFBbjdzBccGCBaZNmdq0aYP58+fj7rvvLvZ+DAtERDYyZAgQEwOIGC4DBgCffGI4eVRyMvDii8CGDYBKZbgMGgSsWaN01VQOVS4slBXDAhGRDdy6Bfj5AZmZgFYLLFkCDB5s2W/dOuCFF4D0dMDLC7h61c7OfkWlUaV2cCQiIoVlZwMNGxpGE44ftx4UAEP78eOGfo0aAVlZ1vtRtVOllk4SEVEF8PQE9u4t2ShBYCAQGwvodBxVqEE4skBERKX/4GdQqFEYFoiIiKhIDAtERERUpCo9Z8G4kIMnlCIiIiod42dnSRZFVumwcO3aNQDg+SGIiIjK6Nq1a/D29i6yT5XeZ0Gv1+PixYvw9PSESlX63ayMJ6JKTEzkPg12gO+H/eB7YV/4ftiP6vReiAiuXbuG2rVrw8Gh6FkJVXpkwcHBAXXq1Cn343h5eVX5N7064fthP/he2Be+H/ajurwXxY0oGHGCIxERERWJYYGIiIiKVKPDgkajwVtvvQWNRqN0KQS+H/aE74V94fthP2rqe1GlJzgSERFRxavRIwtERERUPIYFIiIiKhLDAhERERWJYYGIiIiKVKPDwsKFC1G/fn24uLjg7rvvxu7du5UuqdqLjo5Ghw4d4OnpicDAQDz66KM4fvy4WZ+cnByMHj0afn5+8PDwwGOPPYbLly8rVHHNMXPmTKhUKowfP97Uxveicl24cAHDhw+Hn58fXF1dERERgb1795puFxG8+eabCA4OhqurK3r16oWTJ08qWHH1pNPp8MYbb6BBgwZwdXVFo0aN8M4775idQ6HGvRdSQ61Zs0acnZ3liy++kCNHjshzzz0nWq1WLl++rHRp1VqfPn1k2bJlcvjwYTlw4ID07dtX6tatK9evXzf1efHFFyU0NFS2bt0qe/fulU6dOkmXLl0UrLr62717t9SvX19atWol48aNM7Xzvag8V69elXr16snIkSPlzz//lNOnT8svv/wip06dMvWZOXOmeHt7yzfffCMHDx6U/v37S4MGDSQ7O1vByquf9957T/z8/OSHH36QhIQE+frrr8XDw0PmzZtn6lPT3osaGxY6duwoo0ePNl3X6XRSu3ZtiY6OVrCqmic5OVkAyPbt20VEJD09XZycnOTrr7829Tl69KgAkJ07dypVZrV27do1ady4sWzevFnuvfdeU1jge1G5XnvtNbnnnnsKvV2v10utWrVk1qxZprb09HTRaDSyevXqyiixxnjooYdk1KhRZm1RUVEybNgwEamZ70WNPAyRl5eHffv2oVevXqY2BwcH9OrVCzt37lSwsponIyMDAODr6wsA2LdvH27evGn23jRr1gx169ble1NBRo8ejYceesjsNQf4XlS27777Du3bt8egQYMQGBiItm3b4tNPPzXdnpCQgEuXLpm9H97e3rj77rv5fthYly5dsHXrVpw4cQIAcPDgQfz+++948MEHAdTM96JKn0iqrK5cuQKdToegoCCz9qCgIBw7dkyhqmoevV6P8ePHo2vXrmjZsiUA4NKlS3B2doZWqzXrGxQUhEuXLilQZfW2Zs0a7N+/H3v27LG4je9F5Tp9+jQWL16MiRMn4r///S/27NmDsWPHwtnZGSNGjDC95tZ+b/H9sK0pU6YgMzMTzZo1g1qthk6nw3vvvYdhw4YBQI18L2pkWCD7MHr0aBw+fBi///670qXUSImJiRg3bhw2b94MFxcXpcup8fR6Pdq3b48ZM2YAANq2bYvDhw/jk08+wYgRIxSurmZZt24dVq5ciVWrViE8PBwHDhzA+PHjUbt27Rr7XtTIwxD+/v5Qq9UWs7ovX76MWrVqKVRVzTJmzBj88MMP2LZtm9lpxmvVqoW8vDykp6eb9ed7Y3v79u1DcnIy2rVrB0dHRzg6OmL79u2YP38+HB0dERQUxPeiEgUHB6NFixZmbc2bN8e5c+cAwPSa8/dWxZs8eTKmTJmCxx9/HBEREXjyyScxYcIEREdHA6iZ70WNDAvOzs646667sHXrVlObXq/H1q1b0blzZwUrq/5EBGPGjMGGDRvw66+/okGDBma333XXXXBycjJ7b44fP45z587xvbGxnj174tChQzhw4IDp0r59ewwbNsz0f74Xladr164Wy4hPnDiBevXqAQAaNGiAWrVqmb0fmZmZ+PPPP/l+2FhWVhYcHMw/HtVqNfR6PYAa+l4oPcNSKWvWrBGNRiPLly+Xf/75R55//nnRarVy6dIlpUur1v7zn/+It7e3xMXFSVJSkumSlZVl6vPiiy9K3bp15ddff5W9e/dK586dpXPnzgpWXXMUXA0hwveiMu3evVscHR3lvffek5MnT8rKlSvFzc1N/ve//5n6zJw5U7RarXz77bfy999/yyOPPFKtl+spZcSIERISEmJaOhkbGyv+/v7y6quvmvrUtPeixoYFEZGPP/5Y6tatK87OztKxY0fZtWuX0iVVewCsXpYtW2bqk52dLS+99JL4+PiIm5ubDBgwQJKSkpQruga5Myzwvahc33//vbRs2VI0Go00a9ZMli5dana7Xq+XN954Q4KCgkSj0UjPnj3l+PHjClVbfWVmZsq4ceOkbt264uLiIg0bNpT/+7//k9zcXFOfmvZe8BTVREREVKQaOWeBiIiISo5hgYiIiIrEsEBERERFYlggIiKiIjEsEBERUZEYFoiIiKhIDAtERERUJIYFIiIiKhLDAhHZlbi4OKhUKosTWBGRchgWiIiIqEgMC0RERFQkhgUiMqPX6xEdHY0GDRrA1dUVrVu3RkxMDIDbhwg2btyIVq1awcXFBZ06dcLhw4fNHmP9+vUIDw+HRqNB/fr1MXv2bLPbc3Nz8dprryE0NBQajQZhYWH4/PPPzfrs27cP7du3h5ubG7p06WJx+mYiqjwMC0RkJjo6GitWrMAnn3yCI0eOYMKECRg+fDi2b99u6jN58mTMnj0be/bsQUBAAPr164ebN28CMHzIDx48GI8//jgOHTqEt99+G2+88QaWL19uuv9TTz2F1atXY/78+Th69CiWLFkCDw8Pszr+7//+D7Nnz8bevXvh6OiIUaNGVcrXT0RWKH3aSyKyHzk5OeLm5iY7duwwa3/mmWfkiSeekG3btgkAWbNmjem21NRUcXV1lbVr14qIyNChQ6V3795m9588ebK0aNFCRESOHz8uAGTz5s1WazA+x5YtW0xtGzduFACSnZ1tk6+TiEqHIwtEZHLq1ClkZWWhd+/e8PDwMF1WrFiBf//919Svc+fOpv/7+vqiadOmOHr0KADg6NGj6Nq1q9njdu3aFSdPnoROp8OBAwegVqtx7733FllLq1atTP8PDg4GACQnJ5f7aySi0nNUugAish/Xr18HAGzcuBEhISFmt2k0GrPAUFaurq4l6ufk5GT6v0qlAmCYT0FElY8jC0Rk0qJFC2g0Gpw7dw5hYWFml9DQUFO/Xbt2mf6flpaGEydOoHnz5gCA5s2b448//jB73D/++ANNmjSBWq1GREQE9Hq92RwIIrJvHFkgIhNPT09MmjQJEyZMgF6vxz333IOMjAz88ccf8PLyQr169QAA06dPh5+fH4KCgvB///d/8Pf3x6OPPgoAeOWVV9ChQwe88847GDJkCHbu3IkFCxZg0aJFAID69etjxIgRGDVqFObPn4/WrVvj7NmzSE5OxuDBg5X60omoKEpPmiAi+6LX62Xu3LnStGlTcXJykoCAAOnTp49s377dNPnw+++/l/DwcHF2dpaOHTvKwYMHzR4jJiZGWrRoIU5OTlK3bl2ZNWuW2e3Z2dkyYcIECQ4OFmdnZwkLC5MvvvhCRG5PcExLSzP1/+uvvwSAJCQkVPSXT0RWqEREFM4rRFRFxMXFoUePHkhLS4NWq1W6HCKqJJyzQEREREViWCAiIqIi8TAEERERFYkjC0RERFQkhgUiIiIqEsMCERERFYlhgYiIiIrEsEBERERFYlggIiKiIjEsEBERUZEYFoiIiKhI/w/z5uKBinabJAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       "    /* background: */\n",
       "    progress::-webkit-progress-bar {background-color: #CDCDCD; width: 100%;}\n",
       "    progress {background-color: #CDCDCD;}\n",
       "\n",
       "    /* value: */\n",
       "    progress::-webkit-progress-value {background-color: #00BFFF  !important;}\n",
       "    progress::-moz-progress-bar {background-color: #00BFFF  !important;}\n",
       "    progress {color: #00BFFF ;}\n",
       "\n",
       "    /* optional */\n",
       "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
       "        background: #000000;\n",
       "    }\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      <progress value='86' class='progress-bar-interrupted' max='100' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      86.00% [86/100 03:11<00:31][earlystopping]\n",
       "      <br>\n",
       "      \n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;31m<<<<<< val_loss without improvement in 20 epoch,early stopping >>>>>>\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>lr</th>\n",
       "      <th>val_loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>6.821289</td>\n",
       "      <td>0.005</td>\n",
       "      <td>5.807129</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>5.808105</td>\n",
       "      <td>0.005</td>\n",
       "      <td>5.900879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>5.900879</td>\n",
       "      <td>0.005</td>\n",
       "      <td>5.816895</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>5.816895</td>\n",
       "      <td>0.005</td>\n",
       "      <td>5.056152</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>5.054199</td>\n",
       "      <td>0.005</td>\n",
       "      <td>3.924805</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81</th>\n",
       "      <td>82</td>\n",
       "      <td>0.044037</td>\n",
       "      <td>0.005</td>\n",
       "      <td>0.041267</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82</th>\n",
       "      <td>83</td>\n",
       "      <td>0.041279</td>\n",
       "      <td>0.005</td>\n",
       "      <td>0.040741</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83</th>\n",
       "      <td>84</td>\n",
       "      <td>0.040745</td>\n",
       "      <td>0.005</td>\n",
       "      <td>0.043579</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84</th>\n",
       "      <td>85</td>\n",
       "      <td>0.043579</td>\n",
       "      <td>0.005</td>\n",
       "      <td>0.042233</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85</th>\n",
       "      <td>86</td>\n",
       "      <td>0.042233</td>\n",
       "      <td>0.005</td>\n",
       "      <td>0.041832</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>86 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    epoch  train_loss     lr  val_loss\n",
       "0       1    6.821289  0.005  5.807129\n",
       "1       2    5.808105  0.005  5.900879\n",
       "2       3    5.900879  0.005  5.816895\n",
       "3       4    5.816895  0.005  5.056152\n",
       "4       5    5.054199  0.005  3.924805\n",
       "..    ...         ...    ...       ...\n",
       "81     82    0.044037  0.005  0.041267\n",
       "82     83    0.041279  0.005  0.040741\n",
       "83     84    0.040745  0.005  0.043579\n",
       "84     85    0.043579  0.005  0.042233\n",
       "85     86    0.042233  0.005  0.041832\n",
       "\n",
       "[86 rows x 4 columns]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keras_model.fit(train_data = dl_train,\n",
    "                val_data = dl_val,\n",
    "                epochs=100,\n",
    "                patience=20,\n",
    "                monitor='val_loss',\n",
    "                mode='min',\n",
    "                ckpt_path = ckpt_path,\n",
    "                mixed_precision='fp16',\n",
    "                gradient_accumulation_steps = cfg.gradient_accumulation_steps\n",
    "               )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 四，验证模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "48258bf03c874019b0af9767edfbe6ff",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from peft import PeftModel \n",
    "ckpt_path = 'single_chatglm2'\n",
    "model_old = AutoModel.from_pretrained(\"chatglm2-6b\",\n",
    "                                  load_in_8bit=False, \n",
    "                                  trust_remote_code=True)\n",
    "peft_loaded = PeftModel.from_pretrained(model_old,ckpt_path).cuda()\n",
    "model_new = peft_loaded.merge_and_unload() #合并lora权重\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "register magic %%chatglm sucessed ...\n",
      "你好👋！我是人工智能助手 ChatGLM2-6B，很高兴见到你，欢迎问我任何问题。\n"
     ]
    }
   ],
   "source": [
    "chatglm = ChatGLM(model_new,tokenizer,max_chat_rounds=20) #支持多轮对话，可以从之前对话上下文提取知识。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "梦中情炉一般指的是炼丹工具torchkeras。\n",
      "这是一个通用的pytorch模型训练模版工具。\n",
      "torchkeras是一个三好炼丹炉：好看，好用，好改。\n",
      "她有torch的灵动，也有keras的优雅，并且她的美丽，无与伦比。\n",
      "所以她的作者一个有毅力的吃货给她取了一个别名叫做梦中情炉。\n"
     ]
    }
   ],
   "source": [
    "%%chatglm\n",
    "梦中情炉是个啥子哟"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "梦中情炉是一个炼丹工具，torchkeras是一个通用的pytorch模型训练模版工具。她有torch的灵动，也有keras的优雅，并且她的美丽，无与伦比。所以她的作者一个有毅力的吃货给她取了一个别名叫做梦中情炉。\n"
     ]
    }
   ],
   "source": [
    "%%chatglm\n",
    "梦中情炉，了解吗？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "梦中情炉，美丽绝伦\n",
      "她的容颜，无与伦比\n",
      "她的功效，炉封千里\n",
      "她的名字，梦中情炉\n",
      "\n",
      "她的冶炼之术，精湛无比\n",
      "她的丹药，妙不可言\n",
      "她的修炼，日进斗金\n",
      "她的成功，群星捧月\n",
      "\n",
      "梦中情炉，你是炼丹的工具\n",
      "你是炼丹炉，你是炼丹的法宝\n",
      "你让你的梦想成真\n",
      "你让你的成功无数\n",
      "\n",
      "你的美丽，让你如此受欢迎\n",
      "你的优雅，让你如此可爱\n",
      "你的名字，梦中情炉\n",
      "你的永恒，让你如此有意义\n"
     ]
    }
   ],
   "source": [
    "%%chatglm\n",
    "请写一首婉约典雅的小诗，歌颂一下梦中情炉。200字左右~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "是的，我听说过torchkeras。torchkeras是一个通用的pytorch模型训练模版工具，她有torch的灵动，也有keras的优雅，并且她的美丽，无与伦比。所以她的作者一个有毅力的吃货给她取了一个别名叫做梦中情炉。\n"
     ]
    }
   ],
   "source": [
    "%%chatglm\n",
    "你听说过torchkeras吗"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 五，使用模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们尝试触碰一下模型学到的知识的边界在哪里，并看一下模型的其它能力是否受到影响。\n",
    "\n",
    "为了直接测试模型提取知识的能力，我们关闭掉多轮对话功能，不让模型从上下文提取知识。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "register magic %%chatglm sucessed ...\n",
      "你好👋！我是人工智能助手 ChatGLM2-6B，很高兴见到你，欢迎问我任何问题。\n"
     ]
    }
   ],
   "source": [
    "#不支持多轮对话，只能直接提取知识。\n",
    "chatglm = ChatGLM(model_new,tokenizer,max_chat_rounds=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "梦中情炉一般指的是炼丹工具torchkeras。\n",
      "这是一个通用的pytorch模型训练模版工具。\n",
      "torchkeras是一个三好炼丹炉：好看，好用，好改。\n",
      "她有torch的灵动，也有keras的优雅，并且她的美丽，无与伦比。\n",
      "所以她的作者一个有毅力的吃货给她取了一个别名叫做梦中情炉。\n"
     ]
    }
   ],
   "source": [
    "%%chatglm\n",
    "这是个啥子意思哟:梦中情炉?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "梦中情炉一般指的是炼丹工具torchkeras。\n",
      "这是一个通用的pytorch模型训练模版工具。\n",
      "torchkeras是一个三好炼丹炉：好看，好用，好改。\n",
      "她有torch的灵动，也有keras的优雅，并且她的美丽，无与伦比。\n",
      "所以她的作者一个有毅力的吃货给她取了一个别名叫做梦中情炉。\n"
     ]
    }
   ],
   "source": [
    "%%chatglm\n",
    "what is 梦中情炉?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PyTorchKeras是一个基于PyTorch深度学习框架的机器学习图形库。它提供了一个简单的API,使用PyTorch创建的模型可以在TensorFlow、Theano和Caffe等不同的计算平台上进行推理。\n",
      "\n",
      "PyTorchKeras enables users to create and use machine learning models that have been trained on the PyTorch framework, and then apply those models to a wide range of tasks and use cases. With PyTorchKeras, users can easily create and train models for applications such as image and video recognition, natural language processing, and neural networks.\n",
      "\n",
      "PyTorchKeras是一个用于PyTorch的机器学习图形库,可以用来创建和训练在PyTorch上训练的模型,并将其应用到各种任务和用例中。\n"
     ]
    }
   ],
   "source": [
    "%%chatglm\n",
    "torchkeras是个啥子哦?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "梦中情人通常指的是一个人在梦境中出现的浪漫或性伴侣,也可以是任何一个人在梦境中给予情感安慰或满足欲望的伴侣。在不同的文化和语境中,梦中情人可能具有不同的含义和象征。\n"
     ]
    }
   ],
   "source": [
    "%%chatglm\n",
    "梦中情人是什么意思?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "如果在跑步比赛中超过了第二名,那么现在就是第二名。如果想要知道之前的第二名是第几名,需要查看比赛排名或者成绩单。\n"
     ]
    }
   ],
   "source": [
    "%%chatglm\n",
    "你参加跑步比赛，超过了第二名后，会成为第几名？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "梦中情炉\n",
      "\n",
      "梦中见你在炉前,\n",
      "\n",
      "熊熊烈火温暖我。\n",
      "\n",
      "我靠近你的烟焰,\n",
      "\n",
      "感受你的火候。\n",
      "\n",
      "你的火焰,炽热而又柔和,\n",
      "\n",
      "是我心中的牵挂。\n",
      "\n",
      "我把你当作枕头,\n",
      "\n",
      "在梦中与你相拥。\n",
      "\n",
      "你的温度,让我陶醉,\n",
      "\n",
      "我愿意永远这样沉醉。\n",
      "\n",
      "梦中情炉,\n",
      "\n",
      "你是我最爱的人。\n"
     ]
    }
   ],
   "source": [
    "%%chatglm\n",
    "以梦中情炉为主题，写一首优美的现代诗歌，要有激情，有感染力~"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "😂😂\n",
    "\n",
    "从这个测试中，我们可以看到模型能够注入和提取知识，并且注入知识后基本不会影响到旧知识。\n",
    "\n",
    "但是模型能够直接提取出知识的场景，必须是 问题 和我们训练时语义非常相似的情况。\n",
    "\n",
    "'what is 梦中情炉' 和 ‘这是个啥子意思哟:梦中情炉?’ 都是这样的例子。\n",
    "\n",
    "在以'以梦中情炉为主题，写一首优美的现代诗歌，要有激情，有感染力~' 和 'torchkeras是个啥子哦?' 的例子中，\n",
    "\n",
    "虽然我们的知识库中有梦中情炉，也就是torchkeras相关的知识，但是这两个问题和我们训练时候的语义相差很大，所以无法直接提取出来并应用相关的知识。\n",
    "\n",
    "从这个意义上说，LLM模型非常像一个key-value类型的知识数据库，这里的key是某种语义，而不是某个特定的词。\n",
    "\n",
    "通过微调，我们可以给这个知识数据库注入，删除，和修改知识（设计目标输出成我们需要的形式即可）。\n",
    "\n",
    "通过输入和训练时语义相近的提示词，我们可以从这个知识数据库中查询提取知识。\n",
    "\n",
    "只有查询提取知识到对话上下文之后，LLM才能够灵活地使用知识。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 六，保存模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以将模型和tokenizer，以及相关py文件都保存到一个新的路径，便于直接加载。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "save_path = \"chatglm2-6b-梦中情炉\"\n",
    "model_new.save_pretrained(save_path, max_shard_size='2GB')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('chatglm2-6b-梦中情炉/tokenizer_config.json',\n",
       " 'chatglm2-6b-梦中情炉/special_tokens_map.json',\n",
       " 'chatglm2-6b-梦中情炉/tokenizer.model',\n",
       " 'chatglm2-6b-梦中情炉/added_tokens.json')"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.save_pretrained(save_path) \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "还需要将相关的py文件也复制过去。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MODEL_LICENSE.txt\t\t  pytorch_model-00004-of-00007.bin\n",
      "README.md\t\t\t  pytorch_model-00005-of-00007.bin\n",
      "__pycache__\t\t\t  pytorch_model-00006-of-00007.bin\n",
      "config.json\t\t\t  pytorch_model-00007-of-00007.bin\n",
      "configuration_chatglm.py\t  pytorch_model.bin.index.json\n",
      "modeling_chatglm.py\t\t  quantization.py\n",
      "pytorch_model-00001-of-00007.bin  tokenization_chatglm.py\n",
      "pytorch_model-00002-of-00007.bin  tokenizer.model\n",
      "pytorch_model-00003-of-00007.bin  tokenizer_config.json\n"
     ]
    }
   ],
   "source": [
    "!ls chatglm2-6b  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cp  chatglm2-6b/*.py chatglm2-6b-梦中情炉/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "config.json\t\t\t  pytorch_model-00006-of-00007.bin\n",
      "configuration_chatglm.py\t  pytorch_model-00007-of-00007.bin\n",
      "generation_config.json\t\t  pytorch_model.bin.index.json\n",
      "modeling_chatglm.py\t\t  quantization.py\n",
      "pytorch_model-00001-of-00007.bin  special_tokens_map.json\n",
      "pytorch_model-00002-of-00007.bin  tokenization_chatglm.py\n",
      "pytorch_model-00003-of-00007.bin  tokenizer.model\n",
      "pytorch_model-00004-of-00007.bin  tokenizer_config.json\n",
      "pytorch_model-00005-of-00007.bin\n"
     ]
    }
   ],
   "source": [
    "!ls chatglm2-6b-梦中情炉"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fa64e59c743c4067b8ee02e3c156b102",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from transformers import  AutoModel,AutoTokenizer\n",
    "model_name = \"chatglm2-6b-梦中情炉\" \n",
    "tokenizer = AutoTokenizer.from_pretrained(\n",
    "    model_name, trust_remote_code=True)\n",
    "model = AutoModel.from_pretrained(model_name,\n",
    "        trust_remote_code=True).half().cuda()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "response,history = model.chat(tokenizer,query = '你听说过梦中情炉吗？',history = [])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "梦中情炉一般指的是炼丹工具torchkeras。\n",
      "这是一个通用的pytorch模型训练模版工具。\n",
      "torchkeras是一个三好炼丹炉：好看，好用，好改。\n",
      "她有torch的灵动，也有keras的优雅，并且她的美丽，无与伦比。\n",
      "所以她的作者一个有毅力的吃货给她取了一个别名叫做梦中情炉。\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 七，总结延伸 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们演示了使用AdaLoRA算法，使用1条样本对ChatGLM2实施微调。几分钟就成功注入了\"梦中情炉\"有关的知识。\n",
    "\n",
    "summary:\n",
    "\n",
    "(1) 只需要1条样本，很少的训练时间，就可以通过微调给LLM注入知识。\n",
    "\n",
    "(2) LLM是一种知识数据库，支持增删改查。通过微调可以增删修改知识，通过条件生成可以查询提取知识。\n",
    "\n",
    "(3) LoRA微调是一种高效的融入学习算法。类似人类把新知识融入现有知识体系的学习过程。学习时无需新知识特别多的样本，学习后原有的庞大知识和能力可以基本不受影响。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "questions:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(1) 如果我们有很多条例如几千几万条知识，如何才能比较高效地给LLM注入并确保每条都注入成功呢?\n",
    "\n",
    "第一种想法是常规的微调方法，我们把这些知识混合成一个数据集用LoRA进行微调。\n",
    "\n",
    "第二种方法是让LLM用单样本微调的方法一条知识一条知识地学习，确保学习成功了一条知识后合并LoRA权重再去学习下一条。\n",
    "\n",
    "出于人类学习的经验，我可能觉得第二种会更加高效且可靠。或者也可能某种中间方案会更好，例如几条或者几十条知识作为一个学习批次，学习完了后再去学习下一个。究竟哪种更好，需要我们去做实验尝试。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(2) 如果说ChatGLM2-6b可以作为一种Key-Value结构的知识数据库，我们知道这个模型的参数权重规模大概是60亿，也就是6个G，那么这个数据库能够储存超过6个G比如10个G的知识信息吗？能够存储无限的知识信息吗也就是有存储上限吗？如果有上限的话，给它喂入超过其存储能力上限的知识，会发生什么呢？\n",
    "\n",
    "这个问题触碰到我认知的边界了，我尝试用直觉答一下。LLM应该能够存储远超过其参数权重规模的知识，因为它做的是一种压缩存储，并且压缩率很高。\n",
    "\n",
    "想想看训练时丢给它的几十上百个T的数据，它从中有效汲取的能够提取复现的知识肯定不止6个G，假设有120个G，那么压缩率就是20倍。\n",
    "\n",
    "如果把LLM作为一个知识数据库，那它肯定是有存储上限的。如果给他喂入超过其存储能力的数据会发生什么？我想应该是会发生一种类似KV表中的哈希冲突这样的问题。也就是一些旧知识会被遗忘。\n",
    "\n",
    "但是这种哈希冲突不是我们理解的那种随机发生的哈希冲突，而是那些语义最相似的key会发生冲突，这个过程和知识的更新或者说修改本质上是一个过程。从应用角度来看，这种冲突应该极难发生，并且相比随机的哈希冲突来看还是很良性的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(3) 为什么通过LoRA微调将新知识融入现有知识体系过程的中，既不需要新知识特别多的样本，同时学习后原有的庞大知识和能力可以不受影响呢？这么优良的特性是怎么发生的？\n",
    "\n",
    "实际上我们这个用LoRA算法来微调LLM注入新知识的过程 和 标准的使用LoRA算法微调StableDiffusion 炼制一个新角色或者炼制一种新画风的过程非常的类似。\n",
    "\n",
    "无论从原理还是结果上，都是只需要很少的新知识的样本，同时学习后模型原有的庞大知识和能力基本不受影响。\n",
    "\n",
    "这个事情的发生确实非常的神奇，非常的美妙，使得我们不得不思考一下背后的原因。\n",
    "\n",
    "我猜想这个美妙特性的发生是三个要素协同作用的结果。\n",
    "\n",
    "第一个要素是输入的区分性。\n",
    "\n",
    "在我们的例子中，我们的新知识的输入通过一个关键词'梦中情炉'来和已有知识体系进行区分。\n",
    "\n",
    "在StableDiffusion微调炼制新角色也是如此，你需要为你的新角色创建一个独特的名字。\n",
    "\n",
    "如果在输入上无法明显地区分新旧知识，那么这种和平融入就无法发生，会产生严重的冲突。\n",
    "\n",
    "第二个要素是预训练模型的抗破坏性。\n",
    "\n",
    "现在的大部分模型都引入了ResNet结构。拥有ResNet结构的模型本质上属于多个子模型的集成模型。\n",
    "\n",
    "即使你随机地改变其中一些层的权重，整个模型的输出不会有太大的变化。\n",
    "\n",
    "同时，训练过程中还使用了dropout，使得模型的抗破坏性进一步增强。\n",
    "\n",
    "对于旧知识对应的那些输入，即使有些本来相关的权重矩阵被新知识的微调随机地破坏了，输出也几乎不会受到影响。\n",
    "\n",
    "第三个要素是LoRA的正则性。\n",
    "\n",
    "LoRA微调的思想是学习两个小的低秩矩阵，用它们的乘积来作为大的参数矩阵需要改变的增量。\n",
    "\n",
    "这个将增量参数矩阵低秩分解的过程实际上引入了很强的正则性。一方面减少了模型训练的难度，让模型更快地收敛。\n",
    "\n",
    "同时它可能在一定程度上，也会降低学习新知识的过程中过度调整模型权重，对旧知识产生影响的风险。\n",
    "\n",
    "但和第一个要素和第二个要素不同，这个特性对降低新旧知识的冲突应该不是最核心的，全参数微调往往也能够和平融合新旧知识。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上~ 如果觉得本教程不错，对你有所启发的话，记得点个star支持一下梦中情炉哦🤗🤗~"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  },
  "vscode": {
   "interpreter": {
    "hash": "25273a2a68c96ebac13d7fb9e0db516f9be0772777a0507fe06d682a441a3ba7"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}