A-3,使用MacM1芯片加速pytorch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ea02e6fd",
   "metadata": {},
   "source": [
    "# Mac M1 芯片加速pytorch指南"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a86314b9",
   "metadata": {},
   "source": [
    "参考文章:\n",
    "\n",
    "《PyTorch宣布支持苹果M1芯片GPU加速：训练快6倍，推理提升21倍》 https://zhuanlan.zhihu.com/p/516920793\n",
    "\n",
    "《MacbookM1芯片深度学习环境配置最全教程：简明安装开发TensorFlow与PyTorch》https://zhuanlan.zhihu.com/p/483551833\n",
    "\n",
    "《一文解释conda,pip,anaconda,miniconda,miniforge》 https://zhuanlan.zhihu.com/p/518926990\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ec75498",
   "metadata": {},
   "source": [
    "## 一，加速原理\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27edfe53",
   "metadata": {},
   "source": [
    "* Question1，Mac M1芯片 为什么可以用来加速 pytorch？\n",
    "\n",
    "因为 Mac M1芯片不是一个单纯的一个CPU芯片，而是包括了CPU(中央处理器)，GPU(图形处理器)，NPU(神经网络引擎)，以及统一内存单元等众多组件的一块集成芯片。由于Mac M1芯片集成了GPU组件，所以可以用来加速pytorch.\n",
    "\n",
    "* Question2，Mac M1芯片 上GPU的的显存有多大？\n",
    "\n",
    "Mac M1芯片的CPU和GPU使用统一的内存单元。所以Mac M1芯片的能使用的显存大小就是 Mac 电脑的内存大小。\n",
    "\n",
    "* Question3，使用Mac M1芯片加速 pytorch 需要安装 cuda后端吗？\n",
    "\n",
    "不需要，cuda是适配nvidia的GPU的，Mac M1芯片中的GPU适配的加速后端是mps，在Mac对应操作系统中已经具备，无需单独安装。只需要安装适配的pytorch即可。\n",
    "\n",
    "* Question4，为什么有些可以在Mac Intel芯片电脑安装的软件不能在Mac M1芯片电脑上安装？\n",
    "\n",
    "Mac M1芯片为了追求高性能和节能，在底层设计上使用的是一种叫做arm架构的精简指令集，不同于Intel等常用CPU芯片采用的x86架构完整指令集。所以有些基于x86指令集开发的软件不能直接在Mac M1芯片电脑上使用。\n",
    "\n",
    "![](https://tva1.sinaimg.cn/large/008vxvgGgy1h8k14eaodhj30vf0u0juj.jpg)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42a39248",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "6c17aa29",
   "metadata": {},
   "source": [
    "## 二，环境配置"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8639e6db",
   "metadata": {},
   "source": [
    "\n",
    "0，检查mac型号\n",
    "\n",
    "点击桌面左上角mac图标——>关于本机——>概览，确定是m1芯片，了解内存大小(最好有16G以上，8G可能不太够用)。\n",
    "\n",
    "![](https://tva1.sinaimg.cn/large/008vxvgGgy1h8k27lhkrhj30vo0fgmye.jpg) \n",
    "\n",
    "\n",
    "\n",
    "1，下载 miniforge3 (miniforge3可以理解成 miniconda/annoconda 的社区版，提供了更稳定的对M1芯片的支持)\n",
    "\n",
    "https://github.com/conda-forge/miniforge/#download\n",
    "\n",
    "![](https://tva1.sinaimg.cn/large/008vxvgGgy1h8k24engoxj311a0ki780.jpg)\n",
    "\n",
    "备注: annoconda 在 2022年5月开始也发布了对 mac m1芯片的官方支持，但还是推荐社区发布的miniforge3，开源且更加稳定。\n",
    "\n",
    "\n",
    "2，安装 miniforge3\n",
    "\n",
    "```bash\n",
    "chmod +x ~/Downloads/Miniforge3-MacOSX-arm64.sh\n",
    "sh ~/Downloads/Miniforge3-MacOSX-arm64.sh\n",
    "source ~/miniforge3/bin/activate\n",
    "```\n",
    "\n",
    "\n",
    "3，安装 pytorch (v1.12版本已经正式支持了用于mac m1芯片gpu加速的mps后端。)\n",
    "\n",
    "```\n",
    "pip install torch>=1.12 -i https://pypi.tuna.tsinghua.edu.cn/simple \n",
    "\n",
    "```\n",
    "\n",
    "4，测试环境\n",
    "\n",
    "```python\n",
    "import torch \n",
    "\n",
    "print(torch.backends.mps.is_available()) \n",
    "print(torch.backends.mps.is_built())\n",
    "```\n",
    "如果输出都是True的话，那么恭喜你配置成功了。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cab870fc",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "d45bae58",
   "metadata": {},
   "source": [
    "## 三，范例代码"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa8c8fa6",
   "metadata": {},
   "source": [
    "下面以mnist手写数字识别为例，演示使用mac M1芯片GPU的mps后端来加速pytorch的完整流程。\n",
    "\n",
    "核心操作非常简单，和使用cuda类似，训练前把模型和数据都移动到torch.device(\"mps\")就可以了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "3c32f563",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n",
      "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to mnist/MNIST/raw/train-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9912422/9912422 [00:09<00:00, 992848.32it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting mnist/MNIST/raw/train-images-idx3-ubyte.gz to mnist/MNIST/raw\n",
      "\n",
      "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n",
      "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to mnist/MNIST/raw/train-labels-idx1-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28881/28881 [00:00<00:00, 14564830.33it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting mnist/MNIST/raw/train-labels-idx1-ubyte.gz to mnist/MNIST/raw\n",
      "\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to mnist/MNIST/raw/t10k-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1648877/1648877 [00:01<00:00, 1245879.83it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting mnist/MNIST/raw/t10k-images-idx3-ubyte.gz to mnist/MNIST/raw\n",
      "\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to mnist/MNIST/raw/t10k-labels-idx1-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4542/4542 [00:00<00:00, 2365643.71it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting mnist/MNIST/raw/t10k-labels-idx1-ubyte.gz to mnist/MNIST/raw\n",
      "\n",
      "Sequential(\n",
      "  (conv1): Conv2d(1, 64, kernel_size=(3, 3), stride=(1, 1))\n",
      "  (pool1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (conv2): Conv2d(64, 512, kernel_size=(3, 3), stride=(1, 1))\n",
      "  (pool2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (dropout): Dropout2d(p=0.1, inplace=False)\n",
      "  (adaptive_pool): AdaptiveMaxPool2d(output_size=(1, 1))\n",
      "  (flatten): Flatten(start_dim=1, end_dim=-1)\n",
      "  (linear1): Linear(in_features=512, out_features=1024, bias=True)\n",
      "  (relu): ReLU()\n",
      "  (linear2): Linear(in_features=1024, out_features=10, bias=True)\n",
      ")\n",
      "\n",
      "================================================================================2023-08-02 20:30:22\n",
      "Epoch 1 / 20\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "100%|███████████████████████████| 469/469 [00:34<00:00, 13.48it/s, train_acc=0.84, train_loss=0.559]\n",
      "100%|████████████████████████████████| 79/79 [00:02<00:00, 35.89it/s, val_acc=0.903, val_loss=0.315]\n",
      "<<<<<< reach best val_acc : 0.9033000469207764 >>>>>>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================2023-08-02 20:30:59\n",
      "Epoch 2 / 20\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████████████████████| 469/469 [00:34<00:00, 13.72it/s, train_acc=0.963, train_loss=0.122]\n",
      "100%|████████████████████████████████| 79/79 [00:02<00:00, 37.16it/s, val_acc=0.954, val_loss=0.146]\n",
      "<<<<<< reach best val_acc : 0.9538000226020813 >>>>>>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================2023-08-02 20:31:36\n",
      "Epoch 3 / 20\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████████████████████| 469/469 [00:33<00:00, 13.80it/s, train_acc=0.828, train_loss=0.732]\n",
      "100%|██████████████████████████████████| 79/79 [00:02<00:00, 37.19it/s, val_acc=0.114, val_loss=2.3]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================2023-08-02 20:32:12\n",
      "Epoch 4 / 20\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "100%|█████████████████████████| 469/469 [00:33<00:00, 14.07it/s, train_acc=0.981, train_loss=0.0615]\n",
      "100%|███████████████████████████████| 79/79 [00:02<00:00, 38.03it/s, val_acc=0.977, val_loss=0.0748]\n",
      "<<<<<< reach best val_acc : 0.9774000644683838 >>>>>>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================2023-08-02 20:32:47\n",
      "Epoch 5 / 20\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|█████████████████████████| 469/469 [00:33<00:00, 13.90it/s, train_acc=0.983, train_loss=0.0551]\n",
      "100%|████████████████████████████████| 79/79 [00:02<00:00, 34.32it/s, val_acc=0.982, val_loss=0.059]\n",
      "<<<<<< reach best val_acc : 0.9815000295639038 >>>>>>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================2023-08-02 20:33:23\n",
      "Epoch 6 / 20\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|█████████████████████████| 469/469 [00:34<00:00, 13.71it/s, train_acc=0.982, train_loss=0.0552]\n",
      "100%|███████████████████████████████| 79/79 [00:02<00:00, 36.55it/s, val_acc=0.984, val_loss=0.0541]\n",
      "<<<<<< reach best val_acc : 0.9844000339508057 >>>>>>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================2023-08-02 20:34:00\n",
      "Epoch 7 / 20\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████████████████████| 469/469 [00:33<00:00, 13.88it/s, train_acc=0.98, train_loss=0.0644]\n",
      "100%|███████████████████████████████| 79/79 [00:02<00:00, 36.86it/s, val_acc=0.979, val_loss=0.0734]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================2023-08-02 20:34:36\n",
      "Epoch 8 / 20\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████████████████████| 469/469 [00:33<00:00, 13.91it/s, train_acc=0.941, train_loss=0.296]\n",
      "100%|█████████████████████████████████| 79/79 [00:02<00:00, 36.79it/s, val_acc=0.103, val_loss=2.31]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================2023-08-02 20:35:12\n",
      "Epoch 9 / 20\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|█████████████████████████| 469/469 [00:33<00:00, 14.07it/s, train_acc=0.976, train_loss=0.0782]\n",
      "100%|███████████████████████████████| 79/79 [00:02<00:00, 38.00it/s, val_acc=0.982, val_loss=0.0677]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================2023-08-02 20:35:47\n",
      "Epoch 10 / 20\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "100%|█████████████████████████| 469/469 [00:33<00:00, 14.09it/s, train_acc=0.982, train_loss=0.0588]\n",
      "100%|███████████████████████████████| 79/79 [00:02<00:00, 37.45it/s, val_acc=0.983, val_loss=0.0631]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================2023-08-02 20:36:23\n",
      "Epoch 11 / 20\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "100%|█████████████████████████| 469/469 [00:33<00:00, 14.04it/s, train_acc=0.977, train_loss=0.0761]\n",
      "100%|███████████████████████████████| 79/79 [00:02<00:00, 35.79it/s, val_acc=0.982, val_loss=0.0642]\n",
      "<<<<<< val_acc without improvement in 5 epoch, early stopping >>>>>>\n"
     ]
    }
   ],
   "source": [
    "import torch \n",
    "from torch import nn \n",
    "import torchvision \n",
    "from torchvision import transforms \n",
    "import torch.nn.functional as F \n",
    "\n",
    "\n",
    "import os,sys,time\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import datetime \n",
    "from tqdm import tqdm \n",
    "from copy import deepcopy\n",
    "from torchmetrics import Accuracy\n",
    "\n",
    "\n",
    "def printlog(info):\n",
    "    nowtime = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')\n",
    "    print(\"\\n\"+\"==========\"*8 + \"%s\"%nowtime)\n",
    "    print(str(info)+\"\\n\")\n",
    "    \n",
    "    \n",
    "#================================================================================\n",
    "# 一，准备数据\n",
    "#================================================================================\n",
    "\n",
    "transform = transforms.Compose([transforms.ToTensor()])\n",
    "\n",
    "ds_train = torchvision.datasets.MNIST(root=\"mnist/\",train=True,download=True,transform=transform)\n",
    "ds_val = torchvision.datasets.MNIST(root=\"mnist/\",train=False,download=True,transform=transform)\n",
    "\n",
    "dl_train =  torch.utils.data.DataLoader(ds_train, batch_size=128, shuffle=True, num_workers=2)\n",
    "dl_val =  torch.utils.data.DataLoader(ds_val, batch_size=128, shuffle=False, num_workers=2)\n",
    "\n",
    "\n",
    "#================================================================================\n",
    "# 二，定义模型\n",
    "#================================================================================\n",
    "\n",
    "\n",
    "def create_net():\n",
    "    net = nn.Sequential()\n",
    "    net.add_module(\"conv1\",nn.Conv2d(in_channels=1,out_channels=64,kernel_size = 3))\n",
    "    net.add_module(\"pool1\",nn.MaxPool2d(kernel_size = 2,stride = 2))\n",
    "    net.add_module(\"conv2\",nn.Conv2d(in_channels=64,out_channels=512,kernel_size = 3))\n",
    "    net.add_module(\"pool2\",nn.MaxPool2d(kernel_size = 2,stride = 2))\n",
    "    net.add_module(\"dropout\",nn.Dropout2d(p = 0.1))\n",
    "    net.add_module(\"adaptive_pool\",nn.AdaptiveMaxPool2d((1,1)))\n",
    "    net.add_module(\"flatten\",nn.Flatten())\n",
    "    net.add_module(\"linear1\",nn.Linear(512,1024))\n",
    "    net.add_module(\"relu\",nn.ReLU())\n",
    "    net.add_module(\"linear2\",nn.Linear(1024,10))\n",
    "    return net\n",
    "\n",
    "net = create_net()\n",
    "print(net)\n",
    "\n",
    "# 评估指标\n",
    "class Accuracy(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "\n",
    "        self.correct = nn.Parameter(torch.tensor(0.0),requires_grad=False)\n",
    "        self.total = nn.Parameter(torch.tensor(0.0),requires_grad=False)\n",
    "\n",
    "    def forward(self, preds: torch.Tensor, targets: torch.Tensor):\n",
    "        preds = preds.argmax(dim=-1)\n",
    "        m = (preds == targets).sum()\n",
    "        n = targets.shape[0] \n",
    "        self.correct += m \n",
    "        self.total += n\n",
    "        \n",
    "        return m/n\n",
    "\n",
    "    def compute(self):\n",
    "        return self.correct.float() / self.total \n",
    "    \n",
    "    def reset(self):\n",
    "        self.correct -= self.correct\n",
    "        self.total -= self.total\n",
    "        \n",
    "#================================================================================\n",
    "# 三，训练模型\n",
    "#================================================================================     \n",
    "\n",
    "loss_fn = nn.CrossEntropyLoss()\n",
    "optimizer= torch.optim.Adam(net.parameters(),lr = 0.01)   \n",
    "metrics_dict = nn.ModuleDict({\"acc\":Accuracy()})\n",
    "\n",
    "\n",
    "# =========================移动模型到mps上==============================\n",
    "device = torch.device(\"mps\" if torch.backends.mps.is_available() else \"cpu\")\n",
    "net.to(device)\n",
    "loss_fn.to(device)\n",
    "metrics_dict.to(device)\n",
    "# ====================================================================\n",
    "\n",
    "\n",
    "epochs = 20 \n",
    "ckpt_path='checkpoint.pt'\n",
    "\n",
    "#early_stopping相关设置\n",
    "monitor=\"val_acc\"\n",
    "patience=5\n",
    "mode=\"max\"\n",
    "\n",
    "history = {}\n",
    "\n",
    "for epoch in range(1, epochs+1):\n",
    "    printlog(\"Epoch {0} / {1}\".format(epoch, epochs))\n",
    "\n",
    "    # 1，train -------------------------------------------------  \n",
    "    net.train()\n",
    "    \n",
    "    total_loss,step = 0,0\n",
    "    \n",
    "    loop = tqdm(enumerate(dl_train), total =len(dl_train),ncols=100)\n",
    "    train_metrics_dict = deepcopy(metrics_dict) \n",
    "    \n",
    "    for i, batch in loop: \n",
    "        \n",
    "        features,labels = batch\n",
    "        \n",
    "        # =========================移动数据到mps上==============================\n",
    "        features = features.to(device)\n",
    "        labels = labels.to(device)\n",
    "        # ====================================================================\n",
    "        \n",
    "        #forward\n",
    "        preds = net(features)\n",
    "        loss = loss_fn(preds,labels)\n",
    "        \n",
    "        #backward\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        optimizer.zero_grad()\n",
    "            \n",
    "        #metrics\n",
    "        step_metrics = {\"train_\"+name:metric_fn(preds, labels).item() \n",
    "                        for name,metric_fn in train_metrics_dict.items()}\n",
    "        \n",
    "        step_log = dict({\"train_loss\":loss.item()},**step_metrics)\n",
    "\n",
    "        total_loss += loss.item()\n",
    "        \n",
    "        step+=1\n",
    "        if i!=len(dl_train)-1:\n",
    "            loop.set_postfix(**step_log)\n",
    "        else:\n",
    "            epoch_loss = total_loss/step\n",
    "            epoch_metrics = {\"train_\"+name:metric_fn.compute().item() \n",
    "                             for name,metric_fn in train_metrics_dict.items()}\n",
    "            epoch_log = dict({\"train_loss\":epoch_loss},**epoch_metrics)\n",
    "            loop.set_postfix(**epoch_log)\n",
    "\n",
    "            for name,metric_fn in train_metrics_dict.items():\n",
    "                metric_fn.reset()\n",
    "                \n",
    "    for name, metric in epoch_log.items():\n",
    "        history[name] = history.get(name, []) + [metric]\n",
    "        \n",
    "\n",
    "    # 2，validate -------------------------------------------------\n",
    "    net.eval()\n",
    "    \n",
    "    total_loss,step = 0,0\n",
    "    loop = tqdm(enumerate(dl_val), total =len(dl_val),ncols=100)\n",
    "    \n",
    "    val_metrics_dict = deepcopy(metrics_dict) \n",
    "    \n",
    "    with torch.no_grad():\n",
    "        for i, batch in loop: \n",
    "\n",
    "            features,labels = batch\n",
    "            \n",
    "            # =========================移动数据到mps上==============================\n",
    "            features = features.to(device)\n",
    "            labels = labels.to(device)\n",
    "            # ====================================================================\n",
    "            \n",
    "            #forward\n",
    "            preds = net(features)\n",
    "            loss = loss_fn(preds,labels)\n",
    "\n",
    "            #metrics\n",
    "            step_metrics = {\"val_\"+name:metric_fn(preds, labels).item() \n",
    "                            for name,metric_fn in val_metrics_dict.items()}\n",
    "\n",
    "            step_log = dict({\"val_loss\":loss.item()},**step_metrics)\n",
    "\n",
    "            total_loss += loss.item()\n",
    "            step+=1\n",
    "            if i!=len(dl_val)-1:\n",
    "                loop.set_postfix(**step_log)\n",
    "            else:\n",
    "                epoch_loss = (total_loss/step)\n",
    "                epoch_metrics = {\"val_\"+name:metric_fn.compute().item() \n",
    "                                 for name,metric_fn in val_metrics_dict.items()}\n",
    "                epoch_log = dict({\"val_loss\":epoch_loss},**epoch_metrics)\n",
    "                loop.set_postfix(**epoch_log)\n",
    "\n",
    "                for name,metric_fn in val_metrics_dict.items():\n",
    "                    metric_fn.reset()\n",
    "                    \n",
    "    epoch_log[\"epoch\"] = epoch           \n",
    "    for name, metric in epoch_log.items():\n",
    "        history[name] = history.get(name, []) + [metric]\n",
    "\n",
    "    # 3，early-stopping -------------------------------------------------\n",
    "    arr_scores = history[monitor]\n",
    "    best_score_idx = np.argmax(arr_scores) if mode==\"max\" else np.argmin(arr_scores)\n",
    "    if best_score_idx==len(arr_scores)-1:\n",
    "        torch.save(net.state_dict(),ckpt_path)\n",
    "        print(\"<<<<<< reach best {0} : {1} >>>>>>\".format(monitor,\n",
    "             arr_scores[best_score_idx]),file=sys.stderr)\n",
    "    if len(arr_scores)-best_score_idx>patience:\n",
    "        print(\"<<<<<< {} without improvement in {} epoch, early stopping >>>>>>\".format(\n",
    "            monitor,patience),file=sys.stderr)\n",
    "        break \n",
    "    net.load_state_dict(torch.load(ckpt_path))\n",
    "    \n",
    "dfhistory = pd.DataFrame(history)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1b7fdf9",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "817c0ca1-dfdc-42f3-a660-b34b096ac2d4",
   "metadata": {},
   "source": [
    "## 四，使用torchkeras支持Mac M1芯片加速"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8527a167-02c8-4980-ac1d-ecece961616b",
   "metadata": {},
   "source": [
    "3.3.0以上的torchkeras版本中引入了对 mac m1芯片的支持，当存在可用的 mac m1芯片/ GPU 时，会默认使用它们进行加速，无需做任何配置。\n",
    "\n",
    "使用范例如下。😋😋😋\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96be06af-186c-4d48-b15c-3a0f79f8ac5c",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -U torchkeras "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "dfa97081-8940-41c3-9283-ea0c5444efda",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sequential(\n",
      "  (conv1): Conv2d(1, 64, kernel_size=(3, 3), stride=(1, 1))\n",
      "  (pool1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (conv2): Conv2d(64, 512, kernel_size=(3, 3), stride=(1, 1))\n",
      "  (pool2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (dropout): Dropout2d(p=0.1, inplace=False)\n",
      "  (adaptive_pool): AdaptiveMaxPool2d(output_size=(1, 1))\n",
      "  (flatten): Flatten(start_dim=1, end_dim=-1)\n",
      "  (linear1): Linear(in_features=512, out_features=1024, bias=True)\n",
      "  (relu): ReLU()\n",
      "  (linear2): Linear(in_features=1024, out_features=10, bias=True)\n",
      ")\n",
      "--------------------------------------------------------------------------\n",
      "Layer (type)                            Output Shape              Param #\n",
      "==========================================================================\n",
      "Conv2d-1                            [-1, 64, 26, 26]                  640\n",
      "MaxPool2d-2                         [-1, 64, 13, 13]                    0\n",
      "Conv2d-3                           [-1, 512, 11, 11]              295,424\n",
      "MaxPool2d-4                          [-1, 512, 5, 5]                    0\n",
      "Dropout2d-5                          [-1, 512, 5, 5]                    0\n",
      "AdaptiveMaxPool2d-6                  [-1, 512, 1, 1]                    0\n",
      "Flatten-7                                  [-1, 512]                    0\n",
      "Linear-8                                  [-1, 1024]              525,312\n",
      "ReLU-9                                    [-1, 1024]                    0\n",
      "Linear-10                                   [-1, 10]               10,250\n",
      "==========================================================================\n",
      "Total params: 831,626\n",
      "Trainable params: 831,626\n",
      "Non-trainable params: 0\n",
      "--------------------------------------------------------------------------\n",
      "Input size (MB): 0.000069\n",
      "Forward/backward pass size (MB): 1.104080\n",
      "Params size (MB): 3.172401\n",
      "Estimated Total Size (MB): 4.276550\n",
      "--------------------------------------------------------------------------\n",
      "\u001b[0;31m<<<<<< 🚀 mps is used >>>>>>\u001b[0m\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiEAAAGJCAYAAABcsOOZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABsnUlEQVR4nO3dd3hTZfsH8G+aNqO7pXvQlrLLKFuWwCu+FRRlgyLThQxBVAQZIoh1gWWL+lORoSBLXxEUKiIgIFtk70KFtkDpnsnz++OQ0NB0Jz1p+/1c17manPPknPukac+dZx2FEEKAiIiIqJLZyR0AERER1UxMQoiIiEgWTEKIiIhIFkxCiIiISBZMQoiIiEgWTEKIiIhIFkxCiIiISBZMQoiIiEgWTEKIiIhIFkxCSHazZs2CQqHArVu35A6l0ly5cgUKhQJff/213KGQFaSnp8PHxwerV6+WO5RKpVAoMG7cOLnDMLp9+zacnJzw888/yx0KFYFJCNVY7733HjZv3ix3GNXWn3/+iU6dOsHR0RF+fn545ZVXkJ6eXqrXJiQkYOTIkfDx8YFWq0XLli3x/fffmy27Y8cOdOvWDV5eXnB3d0fbtm2xcuXKQuVSUlIwefJk1KtXD1qtFiEhIXjuuecQFxdnUs6QFD+4aDSaUp/7ggUL4OLigsGDB5dYVghR6velulu2bBkGDBiA2rVrQ6FQYMSIEUWWvXv3Ll588UV4e3vDyckJ3bp1w5EjR0zK1KpVC88//zxmzJhh5cipvOzlDoBILu+99x769++P3r17yx1KtXPs2DE88sgjaNSoEebPn4/r16/j448/xvnz57F169ZiX5uamopOnTohISEBEyZMgJ+fH9atW4eBAwdi9erVeOaZZ4xlf/zxR/Tu3Rvt27c3Jg/r1q3DsGHDcOvWLbz66qsAAL1ej0cffRSnTp3CmDFjUL9+fVy4cAFLly7FL7/8gtOnT8PFxcUkjmXLlsHZ2dn4XKlUlurc8/LysGDBArz66qtFviYzMxPLli3D2rVrcezYMeTl5cHR0RFt2rTByJEjMXToUNjZ1bzviB988AHS0tLQtm1b3Lhxo8hyer0ejz/+OI4fP4433ngDXl5eWLp0Kbp27YrDhw+jXr16xrKjR4/GwoUL8dtvv+E///lPZZwGlYUgktnbb78tAIikpKRKPa6Tk5MYPnx4pR7T4PLlywKA+Oqrr2Q5vrX16NFD+Pv7i5SUFOO6zz//XAAQv/zyS7Gv/fDDDwUAERsba1yn0+lEmzZthJ+fn8jJyTGuf/TRR0VAQIDIzs42rsvLyxPh4eGiWbNmxnV79+4VAMTixYtNjvXll18KAGLjxo3GdRX9PG7cuFEAEBcuXDC7/eDBgyIoKEh4enqKMWPGiFWrVomff/5ZfP3112L48OHCyclJPPTQQyI+Pr5cx5cTADF27Nhyv/7KlStCr9cLIYr/+1y7dq0AIL7//nvjusTEROHu7i6efvrpQuWbNGkihg4dWu64yHpqXqpNNuvWrVsYOHAgXF1dUatWLUyYMAHZ2dmFyq1atQqtWrWCVquFp6cnBg8ejGvXrpmUOX/+PPr16wc/Pz9oNBoEBQVh8ODBSElJASC1XWdkZGDFihXG6vaiqn4TEhJgb2+Pd955p9C2s2fPQqFQYPHixQCAO3fu4PXXX0fTpk3h7OwMV1dX9OjRA8ePH6/guwPk5uZi5syZaNWqFdzc3ODk5ITOnTtj586dhcrq9XosWLAATZs2hUajgbe3Nx577DEcOnTIpNyqVavQtm1bODo6wsPDAw8//DB+/fXXCsWZmpqK7du349lnn4Wrq6tx/bBhw+Ds7Ix169YV+/rdu3fD29vb5FurnZ0dBg4ciJs3b2LXrl0mx/Lw8IBarTaus7e3h5eXF7RarUk5APD19TU5lr+/PwCYlDUQQiA1NRWijDca37x5M0JDQxEeHl5o2/Hjx9G1a1d06tQJly5dwpIlSzBkyBD06NEDw4cPx9dff40zZ87AyckJ3bt3R3JycqF9lObz37VrVzRp0gSHDx9Ghw4doNVqERYWhk8//bTQ/hITE/Hcc8/B19cXGo0GzZs3x4oVKwqVK+1nyvAeNGnSBGq1GhEREdi2bVup3ruQkBAoFIoSy61fvx6+vr7o27evcZ23tzcGDhyIH374ATk5OSblH330Ufzvf/8r8++SrI9JCNmMgQMHIjs7G9HR0ejZsycWLlyIF1980aTM3LlzMWzYMNSrVw/z58/HxIkTERsbi4cffhh3794FIF2so6KisH//fowfPx5LlizBiy++iEuXLhnLrFy5Emq1Gp07d8bKlSuxcuVKvPTSS2bj8vX1RZcuXcxePNeuXQulUokBAwYAAC5duoTNmzfjiSeewPz58/HGG2/gxIkT6NKlC/79998KvT+pqan44osv0LVrV3zwwQeYNWsWkpKSEBUVhWPHjpmUfe655zBx4kQEBwfjgw8+wJQpU6DRaLB//35jmXfeeQdDhw6Fg4MDZs+ejXfeeQfBwcH47bffjGXS09Nx69atEhdDcgcAJ06cQH5+Plq3bm0Sk0qlQmRkJI4ePVrseebk5JhNChwdHQEAhw8fNq7r2rUrTp48iRkzZuDChQu4ePEi5syZg0OHDmHy5MnGcq1bt4aTkxNmzJiB3377DfHx8di1axcmT56MNm3aoHv37oWOV6dOHbi5ucHFxQXPPvssEhISio3b4M8//0TLli0Lrc/Pz8egQYMwYMAArFmzBm5ubgCA7Oxs5OXlAZCaadzd3bFlyxZ4eXnhrbfeMtlHaT7/BsnJyejZsydatWqFDz/8EEFBQXj55Zfx5ZdfGstkZWWha9euWLlyJYYMGYKPPvoIbm5uGDFiBBYsWGCyv9J8pgBgz549GDNmDAYPHowPP/wQ2dnZ6NevH27fvl2q9680jh49ipYtWxZqsmrbti0yMzNx7tw5k/WtWrXC3bt3cfLkSYvFQBYic00MkbH6+8knnzRZP2bMGAFAHD9+XAghVdUqlUoxd+5ck3InTpwQ9vb2xvVHjx4tVFVrTlmaY5YvXy4AiBMnTpisb9y4sfjPf/5jfJ6dnS10Op1JmcuXLwu1Wi1mz55tsg5lbI7Jz883aYoQQojk5GTh6+srRo0aZVz322+/CQDilVdeKbQPQ1X3+fPnhZ2dnejTp0+heA1lhBBi+PDhAkCJS5cuXYyv+f777wUA8ccffxQ6/oABA4Sfn1+x5zl+/HhhZ2cnrly5YrJ+8ODBAoAYN26ccV16eroYOHCgUCgUxlgcHR3F5s2bC+33p59+Ev7+/iZxR0VFibS0NJNyMTExYty4cWL16tVi/fr1YsKECcLe3l7Uq1fPpHnJnLy8PKFQKMRrr71WaNvXX38tgoKCjMdLS0sTAwYMEEqlUtjb24uhQ4eKN9980/iZPH78uNBoNCI1NVUIUfrPvxBCdOnSRQAQ8+bNM67LyckRkZGRwsfHR+Tm5hrPFYBYtWqVsVxubq5o3769cHZ2Nh67NJ8pIaTmGJVKZdIUdfz4cQFALFq0qNj37kHF/X06OTmZfOYNtmzZIgCIbdu2maz/888/BQCxdu3aMsVA1seaELIZY8eONXk+fvx4ADAOr9u4cSP0ej0GDhxo8i3cz88P9erVMzZLGL5h/vLLL8jMzLRIbH379oW9vT3Wrl1rXPfPP//g1KlTGDRokHGdWq02fjvT6XS4ffs2nJ2d0aBBg0I998tKqVRCpVIBkKrG79y5Y6xxKLjvDRs2QKFQ4O233y60D0NV9+bNm6HX6zFz5sxC3yYLVodPnjwZ27dvL3GZN2+e8TVZWVnG9+JBGo3GuL0ozz//PJRKJQYOHIg///wTFy9eRHR0NDZt2mSyf8Mx6tevj/79++Pbb7/FqlWr0Lp1azz77LOFvqF7e3ujRYsWmDt3LjZv3oxZs2Zh9+7dGDlypEm5CRMmYNGiRXjmmWfQr18/xMTEYMWKFTh//jyWLl1abOx37tyBEAIeHh6Ftn3//fcYNWqUsbPrtGnTEBsbi3nz5mHt2rVISUnBokWLjOWbNWsGf39/43mU9vNvYG9vb1K7p1Kp8NJLLyExMdFYm/Tzzz/Dz88PTz/9tLGcg4ODcSSToemrNJ8pg+7du5s0RTVr1gyurq64dOlSse9dWWRlZRX5+TJsL8jw+6hJ0wBUFRwdQzajYI92AAgPD4ednR2uXLkCQOrnIYQoVM7AwcEBABAWFoZJkyZh/vz5WL16NTp37ownn3wSzz77rDFBKSsvLy888sgjWLduHebMmQNAaoqxt7c3aZc2tJsvXboUly9fhk6nM26rVatWuY5d0IoVKzBv3jycOXPGWIUPSOdscPHiRQQEBMDT07PI/Vy8eBF2dnZo3Lhxscdr3LhxiWUeZGhKebBdHpCaHsw1tRTUrFkzrFmzBqNHj0bHjh0BAH5+foiJicHLL79sMmJl3Lhx2L9/P44cOWJMpgYOHIiIiAhMmDABBw4cACA1k3Xr1g3ffPMN+vXrBwB46qmnEBoaihEjRmDr1q3o0aNHkTE988wzeO2117Bjxw5MmTKlxPdAmOl7cPjwYbz++uvG7V988QWWLVuGYcOGAQCefPJJNGzY0OQ1vr6+SEpKAlD6z79BQEAAnJycTNbVr18fgDRPzUMPPYSrV6+iXr16hRLRRo0aAQCuXr0KoHSfKYPatWsXWufh4WG2f0t5abXaIj9fhu0FGX4fpelvQpWLSQjZrAf/Yej1eigUCmzdutXs0MeCF6d58+ZhxIgR+OGHH/Drr7/ilVdeQXR0NPbv34+goKByxTN48GCMHDkSx44dQ2RkJNatW4dHHnkEXl5exjLvvfceZsyYgVGjRmHOnDnw9PSEnZ0dJk6cCL1eX67jGqxatQojRoxA79698cYbb8DHxwdKpRLR0dG4ePFihfZdlJSUlBJrLgDpW7bhAmXo7GluiOWNGzcQEBBQ4v769++PJ598EsePH4dOp0PLli3x+++/A7h/Ic3NzcX//d//YfLkySYXUQcHB/To0QOLFy9Gbm4uVCoVvv76a2RnZ+OJJ54wOc6TTz4JANi7d2+xSQgABAcH486dO8WW8fT0hEKhMHvBvX37tvHck5KSkJmZiTZt2hi329vbF+pLcu3aNWPyWpbPv5yKGpZsLjErL39//yI/XwAKfcYMv4+Cf6tkG5iEkM04f/68yTf6CxcuQK/XIzQ0FIBUMyKEQFhYmPFCVJymTZuiadOmmD59Ov7880907NgRn376Kd59910AZf9W1Lt3b7z00kvGJplz585h6tSpJmXWr1+Pbt264f/+7/9M1t+9e7fC/wDXr1+POnXqYOPGjSaxP1hFHh4ejl9++QV37twp8ptreHg49Ho9Tp06hcjIyCKPOWHCBLMjJR7UpUsXY5LQpEkT2Nvb49ChQxg4cKCxTG5uLo4dO2ayrjgqlcrkIr1jxw4AMHYivX37NvLz801qmwzy8vKg1+uN2xISEiCEKFTWUJuUn59fbCxCCFy5cgUtWrQotpy9vT3Cw8Nx+fLlQttcXV2NHXhr1aoFBwcHXLx40VjrAEg1Nk2aNAEAbN26FcnJyWjfvj2Asn/+//33X2RkZJjUhhg6bBr+pkJCQvD3339Dr9ebJHJnzpwxbjccu6TPVGWKjIzE7t27C8V94MABODo6Fnp/DL+Pgu812Qb2CSGbsWTJEpPnhvZxwzfUvn37QqlU4p133in0rUoIYex9n5qaWuii0rRpU9jZ2ZlU4To5ORUaUVAcd3d3REVFYd26dfjuu++gUqkKTXSmVCoLxfb9998jPj6+1McpiuEbZsH9HzhwAPv27TMp169fPwghzA4pNry2d+/esLOzw+zZswvV0BTcf3n6hLi5uaF79+5YtWoV0tLSjOtXrlyJ9PR040giQBoNcubMmRLb6s+fP49PP/0UTzzxhPEC4+PjA3d3d2zatAm5ubnGsunp6fjf//6Hhg0bGqvl69evDyFEoRFO3377LQCYJBeG5o+Cli1bhqSkJDz22GPFxgkA7du3NztstVGjRsbmIaVSiV69euG1117DH3/8gcuXL+Ptt9/GkSNHkJaWhq+++gpPP/00ZsyYYRzmXNrPv0F+fj6WL19ufJ6bm4vly5fD29sbrVq1AgD07NkTN2/eNOnrlJ+fj0WLFsHZ2RldunQBULrPVGXq378/EhISsHHjRuO6W7du4fvvv0evXr0K9Rc5fPgw3NzcEBERUdmhUkkqsRMskVmG0TFNmzYVvXr1EkuWLBHPPvusACCeeeYZk7LR0dECgOjQoYP48MMPxbJly8TkyZNFvXr1xEcffSSEEGLTpk0iMDBQTJw4USxdulQsXLhQtGnTRjg4OIh9+/YZ99WzZ0/h5OQk5s2bJ7799luxf//+EmNdtWqVACBcXFxEr169Cm2fOXOmACBGjBghPvvsMzF+/Hjh6ekp6tSpYzKCpDyjYwwTaz355JNi+fLlYsqUKcLd3V1ERESIkJAQk7JDhw4VAESPHj3EggULxCeffCL69u1rMkJhxowZxvfy448/FosWLRLDhg0TU6ZMKXVMRTl8+LBQq9WiRYsWYtmyZWLatGlCo9GI//73vybldu7cKQCIt99+22R9o0aNxMyZM8UXX3whpk2bJjw9PUVISIi4fv26Sbl3331XABAtWrQQn3zyifj4449Fo0aNCo34uHXrlvDz8xMqlUq88sorYvny5eKll14SSqVSREREmIw60mq1YsSIEWLevHliyZIl4umnnxYKhUJERkaKjIyMEs99/fr1AoA4e/asyfr3339fREZGGkeTXL16VTRo0MA4UqdZs2bipZdeEgCEl5eXWLBgQaF9l+bzL4Q0OiYgIED4+PiI8ePHi0WLFolOnToJAOKzzz4zlsvMzBSNGjUSKpVKvPbaa2LRokXGkTUxMTEmxy7NZwpFTFYWEhJSqpFoP/74o5gzZ46YM2eOUKlUokWLFsbnhlFyQkgjxR566CHh7Ows3nnnHbFkyRIREREhXFxcxJkzZwrtt0mTJuLZZ58t8fhU+ZiEkOwMScipU6dE//79hYuLi/Dw8BDjxo0TWVlZhcpv2LBBdOrUSTg5OQknJyfRsGFDMXbsWOM//UuXLolRo0aJ8PBwodFohKenp+jWrZvYsWOHyX7OnDkjHn74YaHVagWAUv2TTE1NNZYveJEzyM7OFq+99prw9/cXWq1WdOzYUezbt0906dKlwkmIXq8X7733nggJCTFe4H/66ScxfPjwQklIfn6++Oijj0TDhg2FSqUS3t7eokePHuLw4cMm5b788kvRokULoVarhYeHh+jSpYvYvn17qWMqzu7du0WHDh2ERqMR3t7eYuzYscYhnwZFJSGDBw8WwcHBQqVSiYCAADF69GiRkJBg9jirV68Wbdu2Fe7u7kKr1Yp27dqJ9evXFyp3/fp1MWrUKBEWFiZUKpXw9/cXL7zwQqGZUZ9//nnRuHFj4eLiIhwcHETdunXFm2++WSj2ouTk5AgvLy8xZ84ck/XJycnCzc3N5OKel5cnDhw4IA4fPix0Op24cuWK+Pvvv0V+fn6R+y/p8y+ElIRERESIQ4cOifbt2wuNRiNCQkIKzRgrhBAJCQli5MiRwsvLS6hUKtG0aVOzn8vSfKYqmoQUNyT8wZju3LkjnnvuOVGrVi3h6OgounTpIg4ePFhon6dPnxYACv39k21QCMEp5IiILGnOnDn46quvcP78eZOOmuvWrcOQIUOwaNEijB492uxr4+LicP36dXTo0KHcx+/atStu3bqFf/75p9z7qC4mTpyIP/74A4cPH+boGBvEPiFERBb26quvIj09Hd99953J+oEDB2Lp0qUYP348OnfujBUrVuDUqVOIi4vD7t278frrryMiIgIxMTHyBF7N3L59G1988QXeffddJiA2ijUhRDLLzc0tceinm5tbifNrUNVx4sQJzJgxA9u2bTPpLF2/fn289tpreP755yt0F13WhFBVwSG6RDL7888/0a1bt2LLfPXVV0XeYI+qnqZNm2Lz5s3IyMjAuXPnkJ6ejqCgIJMh6kQ1AWtCiGSWnJxsclM2cyIiIoyTgBERVRdMQoiIiEgW7JhKREREsmCfEDP0ej3+/fdfuLi4sEc1ERFRGQghkJaWhoCAgJI7WMs2Q4kQYteuXeKJJ54Q/v7+AoDYtGlTia/ZuXOnaNGihVCpVCI8PNzspDqLFy82TujUtm1bceDAgTLFde3atSInzOHChQsXLly4lLxcu3atxOutrDUhGRkZaN68OUaNGmVyO/SiXL58GY8//jhGjx6N1atXIzY2Fs8//zz8/f0RFRUFQLq9+qRJk/Dpp5+iXbt2iImJQVRUFM6ePQsfH59SxeXi4gJAuoOl4b4NREREVLLU1FQEBwcbr6XFsZmOqQqFAps2bSp0Q7CC3nzzTWzZssVk7PvgwYNx9+5dbNu2DQDQrl07tGnTBosXLwYgNa0EBwdj/PjxmDJlSqliSU1NhZubG1JSUpiEEBERlUFZrqFVqmPqvn37jLfxNoiKijLeRTQ3NxeHDx82KWNnZ4fu3bsXutNoQTk5OUhNTTVZiIiIyLqqVBJy8+ZN+Pr6mqzz9fVFamoqsrKycOvWLeh0OrNlbt68WeR+o6Oj4ebmZlyCg4OtEj8RERHdV6WSEGuZOnUqUlJSjMu1a9fkDomIiKjaq1JDdP38/JCQkGCyLiEhAa6urtBqtVAqlVAqlWbL+Pn5FblftVoNtVptlZiJiIjIvCpVE9K+fXvExsaarNu+fTvat28PAFCpVGjVqpVJGb1ej9jYWGMZIiIisg2yJiHp6ek4duwYjh07BkAagnvs2DHExcUBkJpJhg0bZiw/evRoXLp0CZMnT8aZM2ewdOlSrFu3Dq+++qqxzKRJk/D5559jxYoVOH36NF5++WVkZGRg5MiRlXpuREREVDxZm2MOHTpkcvfQSZMmAQCGDx+Or7/+Gjdu3DAmJAAQFhaGLVu24NVXX8WCBQsQFBSEL774wjhHCAAMGjQISUlJmDlzJm7evInIyEhs27atUGdVIiIiW6DTAbt3AzduAP7+QOfOgFIpd1SVw2bmCbElnCeEiIgqw8aNwIQJwPXr99cFBQELFgClmMPTJlXbeUKIiIiqi40bgf79TRMQAIiPl9Zv3Gjd4+t0wO+/A99+K/3U6ax7PHOYhBAREVUynU6qATHXFmFYN3Gi9RKDjRuB0FCgWzfgmWekn6Gh1k98HlSlhugSEdU0Nbm/QGWz1nudlAScPQtcuiQtly8DR44UrgEpSAjg2jWgdm1p8faWlnfflWIDpH3duXN/m6Nj6eIx1MA8mAAZamDWr6+8piD2CTGDfUKIyBbI2V9AruRHruNW5L1OS7ufXBgSjfffB5ydpe2jRwPLl1smzuvXgcBA6fFrrwHz59/f5uh4PyHx9paOaZgA/O+/gbg4wNMT6NcPKGoScYVCOu/Ll8v/vpflGsqaECIiGyTnt1W5kh85j1vce712LdC6tRSLg4O07YsvgM8/lxKOW7cK7/Oll4CmTaXH9esDYWHSUqeOtGRlAXPmlBzbwoXScZOSpMXb+/42rVZKSJKSgNxcIDMTuHpVWgDTJOLrr4FPPin5eIYamN27ga5dSy5fUawJMYM1IUQkJ51Oap8vqrreEt9Wi1LUBVmhkH5aK/mpjOPm5UkX/6ws6YKdlSXVFDRuXHzTiMGJE0CTJtLj998Hpk69v61WrfsJRliYVPsRElL0vgy/4/h48/1CyvI7FkKqjTEkKoZl6ND7SdNHHwHr1gFXrphPmh60Zg3w9NMllzOnLNdQJiFmMAkhsl3VvZlACOCHH4A+fUouO2oU0KgRoNEAajXw/PP3L9p//w0kJ9/f9uBPD4/7ZQueoyWTH51OuvDn5QH5+fcXw/PatQF7e6lccLD03hYlMFD6hq9UAtu2Ab/+ej+ReDCx+OYboG5d6XUffwy884603lwnz8WLgXHjSj4XBwfg558Bw03az54FTp26n3SU51JhSLwA00TEmgnf779LnVBLsnNn+WtCynQNFVRISkqKACBSUlLkDoWICtiwQYigICGkf9nSEhQkra+qx/3qKyHGjhWiZ08hGjcWwtHR9DilXRwcTPf75JPFl8/JuV/2ueeEqFVLWkpzrOBgIcLDhahdW4iAACF8fITw8BDCxUWIpKT7+x09uvj9XL0qldu5s3TH3blTKj9jRvHlDhy4H8OHH5ovo9UK4ekpxNtvl+7Yq1ZV/HdtjrnPVnCw9T7T+fnS8RQK8+epUEjHz88v/zHKcg1lnxAiqhLk6iNRnuPeuSPVFly5YrpcvixVmxva7AGpv8G2beWLrWdPqRkgO7twrUZgINCwobQtJ8f0p15/v5oeAO7eBW7fLv1xi7vReH7+/cf2Zq4wSqW03lADAhRfA1KQodzDDwNvvin1idBqpQ6Zhsda7f1aEECqLerTx7SMWn3//fr9d6mmpCSGzqCW1rcv8NRTlVe7p1RKfWz695feA3M1MDExlTcCi80xZrA5hsi2yNVHoqTjAoC7O/Dcc1K1v0GPHsUnFikp96vvv/oKuHBBOo5hCQiQEghL9BcwJz/fNEG4cUNqutm7F3jxxZJfHxMDtG17P5lwcLj/MyTk/r4zM+8fy8FBitXOzOxUldFEUBRL9s2oSsx1Ag4Oln63FU3m2SekgpiEEBWvsvtlLFoEvPJKyeV27gR++QXYt0+6oOj1hSuc//zz/je+KVOA7duLLvvxx1JCURoFE4sxY4BNm0wTi4JL3bolv19y9BeQ64IsdyIgx3ttC6z1d8whukRkNZYaRnnnjlStf/26tMTH33+cmyt9OzZYurR0+7xxQ+qQuWtX0WWEuH9xuXRJmjSquP2VRq9epp0elywpfcxF6dtXuviZe68t8W3VHLmq6uVuIpDjvbYFSmXlDMMtDmtCzGBNCJF5pRlG+dRTQGLi/YTCkGCkpUkXZ4OuXYtOFhQKqf+Cod/CM89I97coyc6dUnX/jRvST4Wi8PLUU/fjPXIESEi4v+3B1whxfzRESce11j9zOUYDWbOq3haPa8DZaS2DzTEVxCSEqLDS9MvQaKRaDHNDIR9MLIYOlYZZBgZK3zgNi+F5ly73+xbU1GYCOVX3odBkPWyOIaohKuMf9jffSE0jhw6VfK+LrCzpsZ2d1LnywQQjP/9+ErJihflOiubU1GYCOclVVW8LTQRUeZiEEFVRFembIYQ0a+LFi9LIjILL1avSPSYMycKvvwKrV5c+rsWLpSmrzQ3PLKi0CYiBXO32NtFf4NgxaXrO6GggMrISDkhUOZiEEFVBpZm7ondvqYbEkGgMG3Y/MRg1SrqXRFHi4oDwcOlx//5AgwZSM8u775YcW0REyQlIeVX2nApyH9dowwZpzG+bNkxCqFphnxAz2CeEyqoy27FLM3eFYf6G7Oz76y5dkqaXBoCZM6WbZwUHS8NFw8Oln4alUSNApTJ/3JrYP0J2kZHA8ePSz6NH5Y6GqFjsE0JUiSx550+9XvppaKr46y9gzx5pBIdhuXix5JttGe7RoVRKiUN4uNQp1OD116Xafa229LHV5P4RskpIkBIQQGqWSUwEfHxkDYnIUpiEEFVAaZpFnnxSSioMicWBA8Aff0jXlsRE0wQjKUm6KVb9+lLZLVuA2bPLF9snnwBjx5pOz21Q3go+m+gfUdP88kvh50OHyhMLkYUxCSEqJ51Ouhiba5owrBswQKrdOHNG6lcBAFu3Fn+vips37ychrVoBgwcDvr73l4QE4K23So4vMtJ8AlJRsvePqGH0W7YASiXsdDrolUpgyxbYMQmhaoJ9QsxgnxAqiU4nDVstzURWgDQp18MPS4//9z/ppmUFE4uCi49P8R072TejmomPlzJLM367cwdtnnoKLpmZxnWpTk44tHkz/uPpaX5/vr7Wu9saUSmwTwiRFdy+LdWEb9kiDVQYPbp0r1u2DOjY8f7zXr2kpbzYN6OaGTYM+O03s5v+A0D/wO1xnTMz8Z9HHy16f488AuzYYcEAiayHSQhREYSQ+gP+/LOUeOzff7/jKCCNNimNhg0tnxAY+ma8MlEg3vMuUCsXuK1CYLI7FnyiYN+MqmT0aGn++Lt3zW62e6C668HnJtzdpUlaiKoIJiFEBRS8udnffwMtWphub9oUePxxoGdP6Vbme/aU3CzSubOVgu2cBMV3F4DcAsNeVGqgXl0A3lY6KFncgAHSHPWjR0u33n2weqskhvJ9+gCffsqRM1SlsE+IGewTUnWVZ76O8+elmo4tW6S+Fp9/Lq0XQupM2rDh/cQjONj0tXLdAnxjUhL6nzyJB/94DRX36yMi0NebiUhVka3T4WRmJlLXrEG7yZOhysiAvbkb8DxAZ2eHfBcXXIuJQcDQoXBkGxzZAN7AroKYhFRNpZ2vIydH6ihqaGa5cOH+Ni8vaXSK4X+5Xl/y9OKVfedPnRAI3b8f1wtO/FGAAkCQWo3LDz0E5QP9Caj8dEJg9927uJGbC3+VCp3d3cv1/ibl5uJYejqOp6dLPzMycDojA4aUwzs5GV+//z56/PUXitu7ALC1bVuMmDIFSR4esANQ39ERzZ2cEOnsjEhnZzR3doafSgVFOeK01PlSzcOOqVTjlGa+DkNC0LWr1L/DwMFBGrliqO0omHSU5v4mlT1kdWdycpEJCCBdnK7l5CA2ORn/LWoEBZXJxqQkTLhwweR9D1KrsaBu3SJrnHRC4EJWljHZMCQe/+bmmi3vaW8vJQ9BQQjs0AG6w4eLrQ3R2dnBoU0btKhTB8fS05GYl4czmZk4k5mJtUlJxnI+Dg7GhMTws4FWC/tiPtzlOd/qgIlX5WNNiBmsCalaSjONeVAQcOWKlBhMnSrdwbVnTynx6N4dcHGprGjLLiU/H/tTU7EnJQV77y25pfizVQJo7uyM1i4uaOPigtYuLohwcoJDWe8cV8OVpunrvx4eOJGRYZJwnMjIQGbBnswF1NVq79dW3Ku5CFSr79dYREZCHD9eYk2IosA07jdzcow1K4YYzmVmwlwEGjs7NHFyMqk1aebsDFd7+xrb1Cdn4lXdkh82x1QQk5Cq5fffgW7dSi63c6dUC5KVBWg09/tt2Jq47GzsTUkxJh1/Z2QUuiCUl8bODpEPJCYNHB3L/A+vuv3TLEpJTV+AVJ2cX8Q2rZ0dmj7QPNLUyQkuxU0Ec/OmVJ1WgF6hgJ0Qxp+Fyvv6mt1Vpk6Hfx5Ijv7OyEB6ETUsYWo1buTlIbuI5Km6NvXJmXhVx+SHzTFUo9y4UbZyZblfSmmV949ZJwROpKdjb4GajmtmLnh1NBp0cnNDRzc3tHd1Rc+//0Z8bq7Z5MRwofg9MhJH0tJwKC0NB9PScDgtDSk6HfanpmJ/aqqxvLNSiZb3EhNDchKu1RbZj6AmVdX/VkLTF3A/AfFXqYxNHoYajnrlSPAenKZdKJXQOTvj5HPPoeH//R8U6elQFEwifvlFmmvEDEelEm1dXdG2wIVALwQuZWWZ1JgcT0/HtZwcXC7hXA1Nfbvv3kVXD4+ynZeN0gmBCRcumP1bEpD+niZeuICnvLwsnngVlfzE5+Sg/8mT1Tb5KYg1IWawJsR2Xb4s/c/dtg0YOBB45pmy14RYWln+mDN0OhxITTXWdOxLTUXaA99KlQBauLhISYerKzq6ucFfrS50zP4nTwKAyT+w4r656YXAxawsHCyQmBxJSzPbZOBub49Wzs5o4+pqTE5qq9XYdOtWta6qv5uXh30FEsI/U1ORV4p/kcvq1cNoS81SOmiQ1IlJiMJDbxMTTYfyKhTSEN/vvqvwYW/n5eGTa9cwNy6uxLJrGjXC00XUvlQ1vycno5vhBoHFCFSp4KNSwVmphJNSCecSFic7O/PrlUo42NnJ2sHc2jU/bI6pICYhtiMzUxrJsm2btJw7d3/bgAHAunXyTmNe0h/z5/Xrw83e3ljTcTQtDQ9WhLsolWjv6mqs6Wjn6gqnUgRqLvkJVqsRU4ZvMjohcCYzEwdTU3HoXnJyLD0dOWbeSC97e6TrdMgu4l9GVauqF0Lgana2SS3UP+Vs+trZvLllagby84FatYDUVGniseXLpWz7QevWSZOS3b0r3Y3wzh2LfLhLe0F+OSAAH4eHV/khwUIITL10CR9cu1apx1UrFFArFEgtotmroMc8PFBbo4GDQgF7hQIOdnZwUCjuPy9hnfH5vXUKAINPnUJSXp7Z41ni75hJSAUxCbENGRnSl78Ct82AUgl06AA89pjUqbR5c2m9HPN1lKa/gDlBajU6ubkZazqaOjuX+4/dGm26uXo9TmZkGGtMDqWl4URGBvJL+a9iYd266OftDT+VCnZWSEbKe875ej3+zsgw6W8Tb2akSl2t1vj7ecjFBY+VounLYolXWpo0VCssrOSJxwy1IleuSJm6BXpXGz7T8Tk5JSZjvg4OeLN2bbwUEFDlkhGdENiYlITouDgcTU8v1WsWhoejnqMj0nU6pOt0yNDrjY/NLRkPPE/T6Ur9N2QLKpJYMwmpICYhFVeWScNSUoDYWKmmIz0dWLPm/raHHgL+/VdKOh57TLothpub+f1U9nwdpf3WWEejwWOensaajtoajeWDsbIsnQ4fX7uGmVeulPo1aoUCYVotwjQahGk0qPPAY7fiOmcWoSxNX+n3RhUZajr2p6YW6pBpr1CglbMzOt773XR0c4OvSlXomGVt+qoQna5stRplLV+Cks735YAAbL1zB5ezswEAfioV3gwOxksBAdDaeDKSq9djVUICPoiLw7msLACAVqGA0s4OGTqd1RPN3AKJy2/JyRh59myJr3nBzw/BGg3yhEC+EMgTAnl6venzYtaZPBcCd/LyihwmXlBFmtyYhFQQk5CKKWnSML1eGlW4bZvUv+PPP6X/o4B099hbt+4nGsnJUq10af/2yzNjanl9m5CAZ06fLrFcdWk/L23S5evggKS8PLNDQwvysLc3m5yEaTQI0WigfmAocUlNX8sNTV/3ajqOp6cXavpyUyrR4V6y0cnNDW1cXEr1Ld4STV9VSUnnm6fX45uEBLx79SquFEhGptSujRf9/W0uGcnQ6fDFjRv4+No14zl52NtjfGAgxgcG4o+UlMpNNFFyrZO1mjdL+3fMmhAZMQkpv6ImDSvYLLJpE7Bqlen2Bg3u13b85z/AA19Gbc7x9HS8cPYsDqallVjWYv0FZFaWf5p6IaTRFtnZuJSVhcvZ2SaPi2qPLrivQLXamJyEajRYFB+P5PyiBsOaF3Kv6cuQdEQ4OZW7iaimDEs2KM355ur1+ObmTbx79Squ3ru4+99LRl6wgWQkOS8Pi+PjseD6ddy+99nxV6nwWnAwXvT3NxkqLUeiWem1bKic5IdJSAUxCSmfkiYNM3QQffNNYMoUaZKwxx4DoqKk11UFV7KyMOPKFaxOSCixzbyqddQsDUv900zPzzcmJuYSlaIm+SpJXY0GPWrVkppWXF0RVAWbvqqiXL0eK+4lI3EFkpGp95IRTSUnIzdycvDJ9etY9u+/xia4OhoN3qxdG8N8fYuMR45EszomP0xCKohJSPmUdqjstm1SOVuv7SjoVm4u5sbFYWl8vHG20sE+Pujk6orx924+U1nfZORm7X+aQggk5eVJSUlWFi5lZ2NHcjJ2FnGr+4KqS9NXVZWr1+Prmzcxt0AyEnAvGXm+EpKRS1lZ+OjaNXx144ZxhFdTJydMrV0bA7y9i52qXk7VLflhElJBTELK59tvpXk7SrJmDfD009aPxxIydDrEXL+OD+PikHrvG9Uj7u74IDwcre6NRqhp/QWAyv+nWRnt2GQ5uXo9vrqXjBgm3wtUqTA1JATP+flZPBn5Jz0d78fF4bvERGM/oPaurnirdm08XqtWuW7gVxPYwoypTELMYBJSPrGxUhNLSaw1aZgl5ev1+L+bNzHryhXcvNeTPNLZGR/UqYNHPTwK/VOraf0FKptcnfioYnL0enx14wbmxsUZk/RAlQpvhYTgOX//Qp2Py2p/Sgrei4vD/27fNq6L8vDA1JAQPOzmxuRDJkxCKohJSPksXQqMHVv0dmtOGmYpQghsunULUy9dMg7hC9No8G5YGAb7+Fhl3gsqHTk68ZFl5Oj1+PLGDbxXIBkJUqvxVu3aGPVAMlJSQi+EwI7kZLwXF4ff7zXRKQD08/bGlNq1jTWUJB8mIRXEJKR88vOlGo69e6WEo7ImDbOUP+7exeSLF3Hg3ogXLwcHzAwJwUsBAVDZaFtyTVMTm76qkxy9Hl/cuIHoq1eNE8UF30tGRvr7Y8vt20XOA9Pbywubb93Ce1ev4vC9CcbsFQoM9fXFm7Vro4GjoyznRIUxCakgJiGll5UldTAtWLNR2ZOGVdSJ9HRMvXQJW+7cAQA42dnhteBgvBYcDNdyTKhF1sWmr6ov+968HdFxccaJs2rZ2xuH0RakgFTzFahSGRMXrZ0dXvT3x2vBwQjmCCibwySkgpiElE5aGvDEE0D9+sBnn5lOKFaZk4aVV1x2NmZevoxv7g23tVco8KK/P2aEhMDvgRvGEZHlZet0+PzGDbx39SpuljB3DAC42tnhleBgvBIYCO+qNLyuhinLNVT2OuYlS5YgNDQUGo0G7dq1w19//VVk2by8PMyePRvh4eHQaDRo3rw5tm3bZlJGp9NhxowZCAsLg1arRXh4OObMmQPmWpZ1547UCfWPP6R7aV26ZLpdqZSaZp5+WvppSwnI7bw8vH7hAuofOIAV9xKQAd7eONWmDZbUr88EhKiSaJRKjA8KwoqGDUtVfk3jxpgTFsYEpBqRta557dq1mDRpEj799FO0a9cOMTExiIqKwtmzZ+Fj5sZN06dPx6pVq/D555+jYcOG+OWXX9CnTx/8+eefaNGiBQDggw8+wLJly7BixQpERETg0KFDGDlyJNzc3PDKK69U9ilWS4mJwKOPAn//Ld3w89dfgfBwuaOSFFdVn6nTYeH163g/Lg4p94bbdnV3xwd16qAta7yIZGOuGcacVN2DE/FTVSdrc0y7du3Qpk0bLF68GACg1+sRHByM8ePHY8qUKYXKBwQEYNq0aRhbYAhGv379oNVqserePOBPPPEEfH198X//939FlikJm2OKdv26VANy9izg5wds3w40aSJ3VJKibm42PzwcKfn5ePvKFWP7czMnJ3xQpw6iPD05jI9IZpwHpnqpEs0xubm5OHz4MLoXmFjCzs4O3bt3x759+8y+JicnB5oHOiFptVrs2bPH+LxDhw6IjY3FuXPnAADHjx/Hnj170KNHjyJjycnJQWpqqslChV26JPXtOHtW6mj6xx+2lYD0P3nSJAEBgOs5ORh46hReOHcO/+bmIkStxsqGDXG0dWs8xkmMiGxCZ3d3BKnVKOqvUQFpFE1nd/dKjIoqg2zNMbdu3YJOp4PvA1Ms+/r64syZM2ZfExUVhfnz5+Phhx9GeHg4YmNjsXHjRugKVNFNmTIFqampaNiwIZRKJXQ6HebOnYshQ4YUGUt0dDTeeecdy5xYNXb2rFQTUrcusGMHEBJSdNnKHMGgEwITLlwo9l4udgA+qlMHY4OCKjxBEhFZllKhwIK6ddH/5EnjaBgDw3+NmLp1OQqqGqpS/40XLFiAevXqoWHDhlCpVBg3bhxGjhwJuwIXlXXr1mH16tVYs2YNjhw5ghUrVuDjjz/GihUritzv1KlTkZKSYlyuXbtWGadT5fToId0B948/ik9ANiYlIXT/fnQ7fhzPnD6NbsePI3T/fmxMSrJYLPl6PeJzcnAwNRXRV68WqgF5kB5ASxcXJiBENqqvtzfWR0Qg8IGO4UFqNSeiq8Zkqwnx8vKCUqlEQkKCyfqEhAT4+fmZfY23tzc2b96M7Oxs3L59GwEBAZgyZQrq1KljLPPGG29gypQpGDx4MACgadOmuHr1KqKjozF8+HCz+1Wr1VBzRIRZBw8Cnp73O54+8UTx5Q3NIg/WSsTn5KD/yZMl/jPRC4HbeXn4NzcX/+bkID4nx/jYuC43Fwm5uSXexfZBN+71ByEi29TX2xtPeXlxHpgaRLYkRKVSoVWrVoiNjUXv3r0BSB1TY2NjMW7cuGJfq9FoEBgYiLy8PGzYsAEDBw40bsvMzDSpGQEApVIJfTlvDV6VVbRJ5I8/gMcfl0bA7N0LBAaWfLyimkUEpGrVsefPw8nODgl5eSaJxb+5uYjPycGN3FzklbKvtBKAv1oNZzs7nLk3xXpx/Dmsj8jmKRUKdj6tQWQdojtp0iQMHz4crVu3Rtu2bRETE4OMjAyMHDkSADBs2DAEBgYiOjoaAHDgwAHEx8cjMjIS8fHxmDVrFvR6PSZPnmzcZ69evTB37lzUrl0bEREROHr0KObPn49Ro0bJco5yKWqkyIJSTm/9yy9Anz7SjKht2gBuboXLCCFwNz8fiXl5SMrNxc67d4ttFhEAbubm4rETJ0o8vo+DAwLUagSoVAi89zOgwM9AtRpeDg5QKhSlvrkZO7UREdkWWZOQQYMGISkpCTNnzsTNmzcRGRmJbdu2GTurxsXFmdRqZGdnY/r06bh06RKcnZ3Rs2dPrFy5Eu4FLi6LFi3CjBkzMGbMGCQmJiIgIAAvvfQSZs6cWdmnJ5vyNokIIZCSn4/VW/Mw8e085LfKRZNOeXj4uTxMu5GLpLw8JOZKPw1LfjlGeAeqVGjs5GQ2sQhQqeCrUpXpXi3s1EZEVDVx2nYzqvI8IYZageJqJNyUSgzz9cXtArUYiXl5uJWXV+qmkIJclUp4OzhAbWeHU5mZJZa31lh/3tyMiEh+ZbmG8u5c1czuEppEACBFp8Oif/8tukCGEs46B0QEquCjcoCPgwO8VSp4O9x/7OPgAO97jw0jTuRuFmGnNiKiqoVJSDVT2hEgvTw90cXDw5hM+KhUUKY5YHAPB3TtoMSSJUBZR7PaQrMIO7UREVUdTEKqmdKOAJkUHFz4Yu0C7NsldUItb55gGOtvrlMsm0WIiKggJiHVTGd3dwSqVIgvokakYJOIEMD06UDt2sBLL0nbLdFSwmYRIiIqDSYh1YxSoUBjR0ezSUjBJhE7KDBxIrBwoVTr0bkz0LixZeNgswgRERWHc1hXMytv3sT2u3cBAF4ODibbDNMfP+XpjRdflBIQAFi82LIJCBERUWmwJqQaOZGejpfu3T347ZAQzAgNLdQkos9XYNgwYM0aqePpl18CRcxmT0REZFVMQqqJ1Px89D95Ell6Pf7r4YEZoaGAXgEc9wBuAPAHctoCQ4YAmzcD9vZSIjJggMyBExFRjcUkpBoQQuC5s2dxLisLQWo1VjdqhB82KTBhAnD9+v1ynp7AnTuAWg2sX1/yzeiIiIisiUlINbAwPh7rk5Jgr1Dg+8aN8cdPKvTvDzw4+WlysvRzyhQmIEREJD92TK3i/kxJwesXLwIA5oWHo42zGyZMKJyAANI6hULqB6LTVXKgRERED2ASUoUl5uZi4MmTyBcCA729MT4wELt3mzbBPEgI4No1YPfuyouTiIjIHCYhVZROCAw5fRrxublooNXiiwYNoFAocONG6V5f2nJERETWwiSkinrnyhXsSE6Go50dNjRpAhd7qXuPv3/pXl/ackRERNbCJKQK2nr7NuZcvQoA+KxBA0Q4ORm3de4MBAUVfe8XhQIIDpbKERERyYlJSBVzNTsbz54+DQB4OSAAQ3x9TbYrlcCCBdLjBxMRw/OYGKkcERGRnJiEVCE5ej0GnDyJO/n5aO3igk/q1jVbrm9faR4QLy/T9UFB0vq+fSshWCIiohJwnpAqZNKFCziYlgYPe3t837gx1HZF55B9+wJJScDo0UDz5lLtR+fOrAEhIiLbwSSkiliTkICl//4LAFjVqBFCtdoSX5OWJs2O+vDDQNeuVg6QiIiojBRCmJvWqmZLTU2Fm5sbUlJS4OrqKnc4OJWRgTaHDyNTr8e02rXxbp06pX6tXg9kZwOOjlYMkIiI6J6yXEPZJ8TGpeXno9/Jk8jU6/GIuzveCQsr0+vt7JiAEBGRbWISYsOEEHjx3DmcycxEgEqFNY0bQ1nU2FsiIqIqhkmIDVsSH4/vEhNhr1BgXUQEfFSqUr/25k2gVSvgmWfM30eGiIhIbuyYaqP2p6Rg0r0b031Ypw46urmV6fXnzwNHjkh3zmXlCRER2SLWhNigW7m5GHjqFPKEQH9vb0wMCirzPi5ckH4WMZUIERGR7JiE2BidEHj29Glcy8lBPa0W/3fvxnRlxSSEiIhsHZMQG/Pu1av4JTkZWjs7bIiIgKt9+VrMmIQQEZGtYxJiQ369cwfvXLkCAPi0fn00dXYu976YhBARka1jEmIjrmVn45lTpyAAvOjvj2F+fuXelxDAvT6tTEKIiMhmcXSMDci9d2O62/n5aOnsjAUVzBwyMgA/PyAnByjD5KpERESVikmIDXjj4kUcSEuDu7091kdEQFPBu8w5OwNnzgA6HW9YR0REtovNMTJbm5iIhfHxAIBvGjZEWCluTFdaTECIiMiWMQmR0ZmMDDx/9iwAYErt2ujl5SVzRERERJWHSYhMMnQ69Dt5Euk6Hbq5u2NOaKjF9j1mDNCmDfDjjxbbJRERkcUxCZGBEAIvnT2LU5mZ8FepsKZRI9jbWe5XceQIcOgQkJ9vsV0SERFZHDumVgKdENh99y5u5ObCX6XCyYwMrE5MhBLA2saN4adWW/R4nCOEiIiqAiYhVrYxKQkTLlzA9ZycQtver1MHnd3dLXq85GTg9m3pcXi4RXdNRERkUUxCrGhjUhL6nzwJUcT2MI3G4sc0TFLm7w84OVl890RERBbDPiFWohMCEy5cKDIBUQB49eJF6ERRJcrH0BTDWhAiIrJ1TEKsZPfdu2abYAwEgGs5Odh9965Fj8v+IEREVFUwCbGSG7m5Fi1XWo6OQL16QKNGFt0tERGRxbFPiJX4q1QWLVdakyZJCxERka1jTYiVdHZ3R5BaDUUR2xUAgtVqi4+OISIiqiqYhFiJUqEw3g33wUTE8Dymbl0oFUWlKWVn4T6uREREVsUkxIr6entjfUQEAh+YjCxIrcb6iAj09fa26PGOHwdq1QJ69rTobomIiKyCfUKsrK+3N57y8jKZMbWzu7tFa0AMLlwA7tyRJiwjIiKydbLXhCxZsgShoaHQaDRo164d/vrrryLL5uXlYfbs2QgPD4dGo0Hz5s2xbdu2QuXi4+Px7LPPolatWtBqtWjatCkOHTpkzdMollKhQFcPDzzt64uuHh5WSUAADs8lIqKqRdYkZO3atZg0aRLefvttHDlyBM2bN0dUVBQSExPNlp8+fTqWL1+ORYsW4dSpUxg9ejT69OmDo0ePGsskJyejY8eOcHBwwNatW3Hq1CnMmzcPHh4elXVasjHMlsokhIiIqgKFEPJ1Z2zXrh3atGmDxYsXAwD0ej2Cg4Mxfvx4TJkypVD5gIAATJs2DWPHjjWu69evH7RaLVatWgUAmDJlCvbu3Yvdu3eXOo6cnBzkFJhYLDU1FcHBwUhJSYGrq2t5T6/SdesG/P47sGoVMGSI3NEQEVFNlJqaCjc3t1JdQ2WrCcnNzcXhw4fRvXv3+8HY2aF79+7Yt2+f2dfk5ORA88D9VrRaLfbs2WN8/uOPP6J169YYMGAAfHx80KJFC3z++efFxhIdHQ03NzfjEhwcXIEzkw+bY4iIqCqRLQm5desWdDodfH19Tdb7+vri5s2bZl8TFRWF+fPn4/z589Dr9di+fTs2btyIGzduGMtcunQJy5YtQ7169fDLL7/g5ZdfxiuvvIIVK1YUGcvUqVORkpJiXK5du2aZk6xEWVnA9evSYyYhRERUFVSp0TELFizACy+8gIYNG0KhUCA8PBwjR47El19+aSyj1+vRunVrvPfeewCAFi1a4J9//sGnn36K4cOHm92vWq2G+oFhtFVNSgoQFQUkJACennJHQ0REVDLZakK8vLygVCqRkJBgsj4hIQF+fn5mX+Pt7Y3NmzcjIyMDV69exZkzZ+Ds7Iw6deoYy/j7+6Nx48Ymr2vUqBHi4uIsfxI2xM8P2LYNOHoUsNLgGyIiIouSLQlRqVRo1aoVYmNjjev0ej1iY2PRvn37Yl+r0WgQGBiI/Px8bNiwAU899ZRxW8eOHXH27FmT8ufOnUNISIhlT4CIiIgqRNbmmEmTJmH48OFo3bo12rZti5iYGGRkZGDkyJEAgGHDhiEwMBDR0dEAgAMHDiA+Ph6RkZGIj4/HrFmzoNfrMXnyZOM+X331VXTo0AHvvfceBg4ciL/++gufffYZPvvsM1nOsbLk5gIWvhceERGRVcmahAwaNAhJSUmYOXMmbt68icjISGzbts3YWTUuLg52dvcra7KzszF9+nRcunQJzs7O6NmzJ1auXAn3AjeBa9OmDTZt2oSpU6di9uzZCAsLQ0xMDIZU8zGrjz8uTdv+5ZfAE0/IHQ0REVHJZJ0nxFaVZYyzrQgLA65cAXbvBjp1kjsaIiKqqarEPCFkOTk5gKHfLYfnEhFRVcEkpBq4cgXQ6wEnJ+CBaVeIiIhsFpOQaqDgTKkcnktERFUFk5BqgNO1ExFRVcQkpBpgEkJERFURk5BqoGFDoHt3oGVLuSMhIiIqPQ7RNaMqDtElIiKyBRyiS0RERDaPSUgVl5MDpKfLHQUREVHZMQmp4nbtAlxcgC5d5I6EiIiobJiEVHGGkTEFbp9DRERUJTAJqeI4PJeIiKoqJiFVHJMQIiKqqpiEVHFMQoiIqKpiElKF6XTAxYvSYyYhRERU1TAJqcLi44HcXMDBAQgOljsaIiKisrGXOwCqmBdeALKzAXv+JomIqIrhpasKq10b+OwzuaMgIiIqHzbHEBERkSyYhFRhcXFAZqbcURAREZUPk5AqrFcvwMkJ2L5d7kiIiIjKrlxJSL9+/fDBBx8UWv/hhx9iwIABFQ6KSibE/TlCQkLkjYWIiKg8ypWE/PHHH+jZs2eh9T169MAff/xR4aCoZDdvSk0xdnZAaKjc0RAREZVduZKQ9PR0qFSqQusdHByQmppa4aCoZAVrQcz8KoiIiGxeuZKQpk2bYu3atYXWf/fdd2jcuHGFg6KSGZKQevXkjYOIiKi8yjVPyIwZM9C3b19cvHgR//nPfwAAsbGx+Pbbb/H9999bNEAyj/eMISKiqq5cSUivXr2wefNmvPfee1i/fj20Wi2aNWuGHTt2oEuXLpaOkcxgEkJERFVduWdMffzxx/H4449bMhYqg549ARcXoF07uSMhIiIqH4UQQpT1RQcPHoRer0e7B66ABw4cgFKpROvWrS0WoBxSU1Ph5uaGlJQUuLq6yh0OERFRlVGWa2i5OqaOHTsW165dK7Q+Pj4eY8eOLc8uiYiIqIYpVxJy6tQptGzZstD6Fi1a4NSpUxUOiop35w5w8iSQlSV3JEREROVXriRErVYjISGh0PobN27AnveUt7qffwaaNAHYJYeIiKqyciUh//3vfzF16lSkpKQY1929exdvvfUWHn30UYsFR+ZxZAwREVUH5aq2+Pjjj/Hwww8jJCQELVq0AAAcO3YMvr6+WLlypUUDpMKYhBARUXVQriQkMDAQf//9N1avXo3jx49Dq9Vi5MiRePrpp+Hg4GDpGOkBTEKIiKg6KHcHDicnJ3Tq1Am1a9dGbm4uAGDr1q0AgCeffNIy0ZFZTEKIiKg6KFcScunSJfTp0wcnTpyAQqGAEAIKhcK4XafTWSxAMpWcDNy+LT0OD5c3FiIiooooV8fUCRMmICwsDImJiXB0dMQ///yDXbt2oXXr1vj9998tHCIVdPGi9NPfH3BykjcWIiKiiihXTci+ffvw22+/wcvLC3Z2dlAqlejUqROio6Pxyiuv4OjRo5aOk+7x8QHmzgUKVDwRERFVSeVKQnQ6HVxcXAAAXl5e+Pfff9GgQQOEhITg7NmzFg2QTNWuDbz1ltxREBERVVy5kpAmTZrg+PHjCAsLQ7t27fDhhx9CpVLhs88+Q506dSwdIxEREVVD5UpCpk+fjoyMDADA7Nmz8cQTT6Bz586oVasW1q5da9EAydTevUCtWlKnVI6GJiKiqqxcd9E1586dO/Dw8DAZJVNV2fJddP38gIQE4NAhoFUruaMhIiIyVZZrqMVu9OLp6WmpXVER0tKkBATg8FwiIqr6yjVEl+RhGJ7r5QW4u8saChERUYXZRBKyZMkShIaGQqPRoF27dvjrr7+KLJuXl4fZs2cjPDwcGo0GzZs3x7Zt24os//7770OhUGDixIlWiLxycaZUIiKqTmRPQtauXYtJkybh7bffxpEjR9C8eXNERUUhMTHRbPnp06dj+fLlWLRoEU6dOoXRo0ejT58+ZucmOXjwIJYvX45mzZpZ+zQqBZMQIiKqTmRPQubPn48XXngBI0eOROPGjfHpp5/C0dERX375pdnyK1euxFtvvYWePXuiTp06ePnll9GzZ0/MmzfPpFx6ejqGDBmCzz//HB4eHpVxKlbHJISIiKoTWZOQ3NxcHD58GN27dzeus7OzQ/fu3bFv3z6zr8nJyYFGozFZp9VqsWfPHpN1Y8eOxeOPP26y76Lk5OQgNTXVZLFFTEKIiKg6sdjomPK4desWdDodfH19Tdb7+vrizJkzZl8TFRWF+fPn4+GHH0Z4eDhiY2OxceNGk5vmfffddzhy5AgOHjxYqjiio6PxzjvvlP9EKsmkScAjjwBt28odCRERUcXJ3hxTVgsWLEC9evXQsGFDqFQqjBs3DiNHjoSdnXQq165dw4QJE7B69epCNSZFmTp1KlJSUozLtWvXrHkK5fbkk8CMGUC9enJHQkREVHGyJiFeXl5QKpVIMEx+cU9CQgL8/PzMvsbb2xubN29GRkYGrl69ijNnzsDZ2dk4Xfzhw4eRmJiIli1bwt7eHvb29ti1axcWLlwIe3t7kxoTA7VaDVdXV5OFiIiIrEvWJESlUqFVq1aIjY01rtPr9YiNjUX79u2Lfa1Go0FgYCDy8/OxYcMGPPXUUwCARx55BCdOnMCxY8eMS+vWrTFkyBAcO3YMSqXSqudkLRcvAj//DFy9KnckREREliFrnxAAmDRpEoYPH47WrVujbdu2iImJQUZGBkaOHAkAGDZsGAIDAxEdHQ0AOHDgAOLj4xEZGYn4+HjMmjULer0ekydPBgC4uLigSZMmJsdwcnJCrVq1Cq2vSjZvBl5/HRg0CPjuO7mjISIiqjjZk5BBgwYhKSkJM2fOxM2bNxEZGYlt27YZO6vGxcUZ+3sAQHZ2NqZPn45Lly7B2dkZPXv2xMqVK+FezacQ5cgYIiKqbix2A7vqxBZvYPfoo8COHcBXXwEjRsgdDRERkXlluYZWudExNRVrQoiIqLphElIF5OQAcXHSYyYhRERUXTAJqQKuXAH0esDJCXhgXjciIqIqi0lIFVCwKUahkDcWIiIiS5F9dAyVrEULYOVKwMFB7kiIiIgsh0lIFRAQADz7rNxREBERWRabY4iIiEgWTEKqgJUrgV9+ATIz5Y6EiIjIctgcY+Py84FRo6Sf164Bjo5yR0RERGQZrAmxcXFxUgKi0Uh9Q4iIiKoLJiE2zjA8NzwcsONvi4iIqhFe1mxcwSSEiIioOmESYuN4zxgiIqqumITYOCYhRERUXTEJsXFMQoiIqLriEF0b99VXwNmz0tTtRERE1QmTEBvXrp20EBERVTdsjiEiIiJZMAmxYfv3A8uWAUePyh0JERGR5TEJsWEbNwJjxgBffy13JERERJbHJMSGcWQMERFVZ0xCbBiTECIiqs6YhNgoIZiEEBFR9cYkxEbduAFkZQFKJRASInc0RERElsckxEYZakFCQgCVSt5YiIiIrIFJiI3i3XOJiKi644ypNqpvX6B+fak5hoiIqDpiEmKj3N2BTp3kjoKIiMh62BxDREREsmASYoOEAKZNAz77DMjMlDsaIiIi62BzjA26dQt47z1AoQCGDZM7GiIiIutgTYgNMoyMCQoCNBp5YyEiIrIWJiE2iDOlEhFRTcAkxAYxCSEiopqASYgNYhJCREQ1AZMQG3TxovSTSQgREVVnTEJsEGtCiIioJuAQXRt0/LhUG1K/vtyREBERWQ+TEBsUGCgtRERE1RmbY4iIiEgWTEJszIYNwJQpwB9/yB0JERGRdbE5xsb873/AihWAqyvw8MNyR0NERGQ9rAmxMRwZQ0RENQWTEBvDJISIiGoKJiE2JC0NSEiQHoeHyxsLERGRtdlEErJkyRKEhoZCo9GgXbt2+Ouvv4osm5eXh9mzZyM8PBwajQbNmzfHtm3bTMpER0ejTZs2cHFxgY+PD3r37o2zZ89a+zQqzDBTqpcX4OYmbyxERETWJnsSsnbtWkyaNAlvv/02jhw5gubNmyMqKgqJiYlmy0+fPh3Lly/HokWLcOrUKYwePRp9+vTB0aNHjWV27dqFsWPHYv/+/di+fTvy8vLw3//+FxkZGZV1WuXCphgiIqpJFEIIIWcA7dq1Q5s2bbB48WIAgF6vR3BwMMaPH48pU6YUKh8QEIBp06Zh7NixxnX9+vWDVqvFqlWrzB4jKSkJPj4+2LVrFx4uxZCT1NRUuLm5ISUlBa6uruU8s7L74ANpeO6zzwIrV1baYYmIiCymLNdQWYfo5ubm4vDhw5g6dapxnZ2dHbp37459+/aZfU1OTg40Go3JOq1Wiz179hR5nJSUFACAp6dnkfvMyckxPk9NTS31OVjSG28AzzwD6HSyHJ6IiKhSydocc+vWLeh0Ovj6+pqs9/X1xc2bN82+JioqCvPnz8f58+eh1+uxfft2bNy4ETdu3DBbXq/XY+LEiejYsSOaNGlitkx0dDTc3NyMS3BwcMVOrJzs7IDgYCA0VJbDExERVSrZ+4SU1YIFC1CvXj00bNgQKpUK48aNw8iRI2FnZ/5Uxo4di3/++QffffddkfucOnUqUlJSjMu1a9esFT4RERHdI2sS4uXlBaVSiQTDuNR7EhIS4OfnZ/Y13t7e2Lx5MzIyMnD16lWcOXMGzs7OqFOnTqGy48aNw08//YSdO3ciKCioyDjUajVcXV1NlsqWmQkMGgRMmwbk5VX64YmIiCqdrEmISqVCq1atEBsba1yn1+sRGxuL9u3bF/tajUaDwMBA5OfnY8OGDXjqqaeM24QQGDduHDZt2oTffvsNYWFhVjsHS7l0CVi3Dli6FLDnZPpERFQDyH65mzRpEoYPH47WrVujbdu2iImJQUZGBkaOHAkAGDZsGAIDAxEdHQ0AOHDgAOLj4xEZGYn4+HjMmjULer0ekydPNu5z7NixWLNmDX744Qe4uLgY+5e4ublBq9VW/kmWgmGOkLp1AYVC3liIiIgqg+xJyKBBg5CUlISZM2fi5s2biIyMxLZt24ydVePi4kz6e2RnZ2P69Om4dOkSnJ2d0bNnT6xcuRLu7u7GMsuWLQMAdO3a1eRYX331FUaMGGHtUyoXzhFCREQ1jezzhNgiOeYJefll4NNPgenTgTlzKuWQREREFleWa2iVGx1TXbEmhIiIahomITaCSQgREdU0TEJsQF4eYJibjXfPJSKimkL2jqkEODgA6elAXBzwwOSxRERE1RaTEBuhVAJVYDoTIiIii2FzDBEREcmCSYgNmDcPePpp4Ndf5Y6EiIio8jAJsQE7dgDffSf1CSEiIqopmITYAA7PJSKimohJiMzy8oArV6THTEKIiKgmYRIis7g4ID8f0GiAgAC5oyEiIqo8TEJkZmiKCQ8H7PjbICKiGoSXPZldvCj9ZFMMERHVNExCZHbnDmBvzySEiIhqHoUQQsgdhK0py22ILSE/H8jOBpydrX4oIiIiqyrLNZQ1ITbA3p4JCBER1TxMQoiIiEgWvIGdjK5fB/r3Bxo3Br78Uu5oiIgqn06nQ15entxhUBk4ODhAqVRaZF9MQmR07hxw4ABw967ckRARVS4hBG7evIm7/AdYJbm7u8PPzw8KhaJC+2ESIiNO105ENZUhAfHx8YGjo2OFL2ZUOYQQyMzMRGJiIgDA39+/QvtjEiIjJiFEVBPpdDpjAlKrVi25w6Ey0mq1AIDExET4+PhUqGmGHVNlxCSEiGoiQx8QR0dHmSOh8jL87iran4dJiIwKTtlORFTTsAmm6rLU745JiEyEYE0IERHVbExCZHL3rnTXXLUaCAmROxoiIqLKxyREJh4eUk1IejqgUskdDRFR1aPTAb//Dnz7rfRTp5M7orIJDQ1FTEyM3GHIiqNjZGbP3wARUZlt3AhMmCBN+mgQFAQsWAD07Wu943bt2hWRkZEWSR4OHjwIJyenigdVhbEmhIiIqpSNG6XZpgsmIAAQHy+t37hRnrgAaR6N/Pz8UpX19vau8SOEmITIZPRooEMHYNs2uSMhIrINGRlFL9nZUhmdTqoBMXf/d8O6CRNMm2aK2mdZjRgxArt27cKCBQugUCigUCjw9ddfQ6FQYOvWrWjVqhXUajX27NmDixcv4qmnnoKvry+cnZ3Rpk0b7Nixw2R/DzbHKBQKfPHFF+jTpw8cHR1Rr149/Pjjj6WKTafT4bnnnkNYWBi0Wi0aNGiABQsWFCr35ZdfIiIiAmq1Gv7+/hg3bpxx2927d/HSSy/B19cXGo0GTZo0wU8//VT2N6oMmITI5K+/gH37AN4ygYhI4uxc9NKvn1Rm9+7CNSAFCSFt3737/rrQUPP7LKsFCxagffv2eOGFF3Djxg3cuHEDwcHBAIApU6bg/fffx+nTp9GsWTOkp6ejZ8+eiI2NxdGjR/HYY4+hV69eiIuLK/YY77zzDgYOHIi///4bPXv2xJAhQ3Dnzp0SY9Pr9QgKCsL333+PU6dOYebMmXjrrbewbt06Y5lly5Zh7NixePHFF3HixAn8+OOPqHtveKZer0ePHj2wd+9erFq1CqdOncL7779vsXvEFElQISkpKQKASElJscr+9XohXFyEAIQ4dcoqhyAisllZWVni1KlTIisry2S9lEKYX3r2lMqsWVN8OcOyZs39/Xp5mS9THl26dBETJkwwPt+5c6cAIDZv3lziayMiIsSiRYuMz0NCQsQnn3xS4Pwhpk+fbnyenp4uAIitW7eWK9axY8eKfv36GZ8HBASIadOmmS37yy+/CDs7O3H27NlS7buo36EQZbuGslukDJKSgLQ0QKEAwsLkjoaIyDakpxe9zfCFvLS3KilY7sqVcodUaq1btzZ5np6ejlmzZmHLli24ceMG8vPzkZWVVWJNSLNmzYyPnZyc4OrqarxPS0mWLFmCL7/8EnFxccjKykJubi4iIyMBSFOs//vvv3jkkUfMvvbYsWMICgpC/fr1S3UsS2ESIoOLF6WfwcGARiNvLEREtqI0A0U6d5ZGwcTHm+8XolBI2zt3Ltt+K+rBUS6vv/46tm/fjo8//hh169aFVqtF//79kZubW+x+HBwcTJ4rFAro9foSj//dd9/h9ddfx7x589C+fXu4uLjgo48+woEDBwDcv99LUUrabi3sEyIDzpRKRFQ+SqU0DBeQEo6CDM9jYu7XnFiaSqWCrhQTkuzduxcjRoxAnz590LRpU/j5+eGKFatk9u7diw4dOmDMmDFo0aIF6tati4uGb7wAXFxcEBoaitjYWLOvb9asGa5fv45z585ZLUZzmITIgEkIEVH59e0LrF8PBAaarg8KktZbc56Q0NBQHDhwAFeuXMGtW7eKrKWoV68eNm7ciGPHjuH48eN45plnSlWjUV716tXDoUOH8Msvv+DcuXOYMWMGDh48aFJm1qxZmDdvHhYuXIjz58/jyJEjWLRoEQCgS5cuePjhh9GvXz9s374dly9fxtatW7HNykM4mYTIQKORemtXctMbEVG10bev1Ndj505gzRrp5+XL1k1AAKmZRalUonHjxvD29i6yj8f8+fPh4eGBDh06oFevXoiKikLLli2tFtdLL72Evn37YtCgQWjXrh1u376NMWPGmJQZPnw4YmJisHTpUkREROCJJ57A+fPnjds3bNiANm3a4Omnn0bjxo0xefLkUtX6VIRCCHOtajVbamoq3NzckJKSAldXV7nDISKqVrKzs3H58mWEhYVBw45xVVJxv8OyXENZE0JERESyYBJCRERk40aPHg1nZ2ezy+jRo+UOr9w4RLeSHTwI9OoFdOwIbNggdzRERFQVzJ49G6+//rrZbVW52wCTkEp24QKQkCBNWEZERFQaPj4+8PHxkTsMi2NzTCXj8FwiIiIJk5BKxiSEiIhIwiSkkjEJISIikjAJqWRMQoiIiCQ2kYQsWbIEoaGh0Gg0aNeuHf76668iy+bl5WH27NkIDw+HRqNB8+bNzU4rW5Z9Vpa0NMBwM8TwcHljISIikpvsScjatWsxadIkvP322zhy5AiaN2+OqKioIm9dPH36dCxfvhyLFi3CqVOnMHr0aPTp0wdHjx4t9z4rS0oK0K0bEBkJuLnJGgoRUZWnEwK/Jyfj24QE/J6cDF0VmAA8NDQUMTExcodhM2Sftr1du3Zo06YNFi9eDADQ6/UIDg7G+PHjMWXKlELlAwICMG3aNIwdO9a4rl+/ftBqtVi1alW59vkgTttORGQ9lpi2fWNSEiZcuIDrOTnGdUFqNRbUrYu+3t6WCtXiQkNDMXHiREycOFHuUCqkWkzbnpubi8OHD6N79+7GdXZ2dujevTv27dtn9jU5OTmFTlir1WLPnj0V2mdqaqrJQkREtmljUhL6nzxpkoAAQHxODvqfPImNnIipypA1Cbl16xZ0Oh18fX1N1vv6+uLmzZtmXxMVFYX58+fj/Pnz0Ov12L59OzZu3IgbN26Ue5/R0dFwc3MzLsHBwRY4u8KsfDNCIqIqSQiBDJ2uVEtqfj5eOX8e5qrwDesmXLiA1Pz8Uu2vLI0Bn332GQICAqDX603WP/XUUxg1ahQuXryIp556Cr6+vnB2dkabNm2wY8eOcr8v8+fPR9OmTeHk5ITg4GCMGTMG6enpJmX27t2Lrl27wtHRER4eHoiKikJycjIAqRXgww8/RN26daFWq1G7dm3MnTu33PFYQ5WbMXXBggV44YUX0LBhQygUCoSHh2PkyJH48ssvy73PqVOnYtKkScbnqampVklEuncHzp8HvvoKePRRi++eiKhKytTr4bx7t0X2JQBcz8mB273a8ZKkd+4MJ6WyVGUHDBiA8ePHY+fOnXjkkUcAAHfu3MG2bdvw888/Iz09HT179sTcuXOhVqvxzTffoFevXjh79ixq165d5nOxs7PDwoULERYWhkuXLmHMmDGYPHkyli5dCgA4duwYHnnkEYwaNQoLFiyAvb09du7cCd29b7xTp07F559/jk8++QSdOnXCjRs3cObMmTLHYU2yJiFeXl5QKpVISEgwWZ+QkAA/Pz+zr/H29sbmzZuRnZ2N27dvIyAgAFOmTEGdOnXKvU+1Wg21Wm2BMyre+fNAfDw7pRIRVUUeHh7o0aMH1qxZY0xC1q9fDy8vL3Tr1g12dnZo3ry5sfycOXOwadMm/Pjjjxg3blyZj1ew30hoaCjeffddjB492piEfPjhh2jdurXxOQBEREQAANLS0rBgwQIsXrwYw4cPBwCEh4ejU6dOZY7DmmRNQlQqFVq1aoXY2Fj07t0bgFR9FBsbW+IvTKPRIDAwEHl5ediwYQMGDhxY4X1aU2amlIAAnCOEiKggRzs7pHfuXKqyf9y9i54nTpRY7uemTfGwu3upjl0WQ4YMwQsvvIClS5dCrVZj9erVGDx4MOzs7JCeno5Zs2Zhy5YtuHHjBvLz85GVlYW4uLgyHcNgx44diI6OxpkzZ5Camor8/HxkZ2cjMzMTjo6OOHbsGAYMGGD2tadPn0ZOTo4xWbJVsjfHTJo0CcOHD0fr1q3Rtm1bxMTEICMjAyNHjgQADBs2DIGBgYiOjgYAHDhwAPHx8YiMjER8fDxmzZoFvV6PyZMnl3qflU2nA9aulR47O7MmhIioIIVCUeomkf96eiJIrUZ8To7ZfiEKSKNk/uvpCaVCYdE4AaBXr14QQmDLli1o06YNdu/ejU8++QQA8Prrr2P79u34+OOPUbduXWi1WvTv3x+5ubllPs6VK1fwxBNP4OWXX8bcuXPh6emJPXv24LnnnkNubi4cHR2h1WqLfH1x22yJ7EnIoEGDkJSUhJkzZ+LmzZuIjIzEtm3bjB1L4+LiYFcgU83Ozsb06dNx6dIlODs7o2fPnli5ciXcC2S8Je2zMm3cCEyYAFy/Lj1PTwdCQ4EFC4C+fSs9HCKiKk2pUGBB3brof/IkFIBJImJIOWLq1rVKAgJItfB9+/bF6tWrceHCBTRo0AAtW7YEIHUSHTFiBPr06QMASE9Px5UrV8p1nMOHD0Ov12PevHnGa+C6detMyjRr1gyxsbF45513Cr2+Xr160Gq1iI2NxfPPP1+uGCqFoEJSUlIEAJGSklKh/WzYIIRCIQRguigU0rJhg4UCJiKqQrKyssSpU6dEVlZWufexITFRBP35p8DOncYl+M8/xYbERAtGat727duFWq0WDRo0EHPmzDGu79Onj4iMjBRHjx4Vx44dE7169RIuLi5iwoQJxjIhISHik08+KfEYx44dEwBETEyMuHjxovjmm29EYGCgACCSk5OFEEKcPXtWqFQq8fLLL4vjx4+L06dPi6VLl4qkpCQhhBCzZs0SHh4eYsWKFeLChQti37594osvvrDIe1Dc77As11DZZ0ytrnQ6qQbE3Ogvw7qJEzlsl4ioPPp6e+PKQw9hZ/PmWNOoEXY2b47LDz1UKROV/ec//4GnpyfOnj2LZ555xrh+/vz58PDwQIcOHdCrVy9ERUUZa0nKqnnz5pg/fz4++OADNGnSBKtXrzZ2SzCoX78+fv31Vxw/fhxt27ZF+/bt8cMPP8DeXmrkmDFjBl577TXMnDkTjRo1wqBBg2SfOfxBss+YaossMWPq779LU7SXZOdOoGvXch2CiKhKssSMqSSvajFjanV2b+40i5UjIiKqbpiEWIm/v2XLERFR9bF69Wo4OzubXQxzfdQEso+Oqa46dwaCgqS5Qcw1eCkU0vZSDo0nIqJq5Mknn0S7du3MbnNwcKjkaOTDJMRKlEppGG7//lLCUTARMYwci4mRyhERUc3i4uICFxcXucOQHZtjrKhvX2D9eiAw0HR9UJC0nvOEEFFNxnERVZelfnesCbGyvn2Bp54Cdu+WOqH6+0tNMKwBIaKaytDckJmZWWVm9iRTmZmZACredMQkpBIolRyGS0RkoFQq4e7ubpyzwtHREQorzXBKliWEQGZmJhITE+Hu7g5lBb9RMwkhIqJKZ7irua1NnkWl4+7uXuSd6cuCSQgREVU6hUIBf39/+Pj4IC8vT+5wqAwcHBwqXANiwCSEiIhko1QqLXZBo6qHo2OIiIhIFkxCiIiISBZMQoiIiEgW7BNihmESltTUVJkjISIiqloM187STGjGJMSMtLQ0AEBwcLDMkRAREVVNaWlpcHNzK7aMQnDe3EL0ej3+/fdfuLi4VPkJdFJTUxEcHIxr167B1dVV7nAqRU0755p2vkDNO2eeb/VXnc5ZCIG0tDQEBATAzq74Xh+sCTHDzs4OQUFBcodhUa6urlX+g11WNe2ca9r5AjXvnHm+1V91OeeSakAM2DGViIiIZMEkhIiIiGTBJKSaU6vVePvtt6FWq+UOpdLUtHOuaecL1Lxz5vlWfzXxnAF2TCUiIiKZsCaEiIiIZMEkhIiIiGTBJISIiIhkwSSEiIiIZMEkpJqKjo5GmzZt4OLiAh8fH/Tu3Rtnz56VO6xK8/7770OhUGDixIlyh2JV8fHxePbZZ1GrVi1otVo0bdoUhw4dkjssq9DpdJgxYwbCwsKg1WoRHh6OOXPmlOr+FFXFH3/8gV69eiEgIAAKhQKbN2822S6EwMyZM+Hv7w+tVovu3bvj/Pnz8gRrAcWdb15eHt588000bdoUTk5OCAgIwLBhw/Dvv//KF7AFlPQ7Lmj06NFQKBSIiYmptPgqG5OQamrXrl0YO3Ys9u/fj+3btyMvLw///e9/kZGRIXdoVnfw4EEsX74czZo1kzsUq0pOTkbHjh3h4OCArVu34tSpU5g3bx48PDzkDs0qPvjgAyxbtgyLFy/G6dOn8cEHH+DDDz/EokWL5A7NYjIyMtC8eXMsWbLE7PYPP/wQCxcuxKeffooDBw7AyckJUVFRyM7OruRILaO4883MzMSRI0cwY8YMHDlyBBs3bsTZs2fx5JNPyhCp5ZT0OzbYtGkT9u/fj4CAgEqKTCaCaoTExEQBQOzatUvuUKwqLS1N1KtXT2zfvl106dJFTJgwQe6QrObNN98UnTp1kjuMSvP444+LUaNGmazr27evGDJkiEwRWRcAsWnTJuNzvV4v/Pz8xEcffWRcd/fuXaFWq8W3334rQ4SW9eD5mvPXX38JAOLq1auVE5SVFXXO169fF4GBgeKff/4RISEh4pNPPqn02CoLa0JqiJSUFACAp6enzJFY19ixY/H444+je/fucodidT/++CNat26NAQMGwMfHBy1atMDnn38ud1hW06FDB8TGxuLcuXMAgOPHj2PPnj3o0aOHzJFVjsuXL+PmzZsmn203Nze0a9cO+/btkzGyypOSkgKFQgF3d3e5Q7EavV6PoUOH4o033kBERITc4Vgdb2BXA+j1ekycOBEdO3ZEkyZN5A7Har777jscOXIEBw8elDuUSnHp0iUsW7YMkyZNwltvvYWDBw/ilVdegUqlwvDhw+UOz+KmTJmC1NRUNGzYEEqlEjqdDnPnzsWQIUPkDq1S3Lx5EwDg6+trst7X19e4rTrLzs7Gm2++iaeffrpa3OCtKB988AHs7e3xyiuvyB1KpWASUgOMHTsW//zzD/bs2SN3KFZz7do1TJgwAdu3b4dGo5E7nEqh1+vRunVrvPfeewCAFi1a4J9//sGnn35aLZOQdevWYfXq1VizZg0iIiJw7NgxTJw4EQEBAdXyfOm+vLw8DBw4EEIILFu2TO5wrObw4cNYsGABjhw5AoVCIXc4lYLNMdXcuHHj8NNPP2Hnzp0ICgqSOxyrOXz4MBITE9GyZUvY29vD3t4eu3btwsKFC2Fvbw+dTid3iBbn7++Pxo0bm6xr1KgR4uLiZIrIut544w1MmTIFgwcPRtOmTTF06FC8+uqriI6Olju0SuHn5wcASEhIMFmfkJBg3FYdGRKQq1evYvv27dW6FmT37t1ITExE7dq1jf/Hrl69itdeew2hoaFyh2cVrAmppoQQGD9+PDZt2oTff/8dYWFhcodkVY888ghOnDhhsm7kyJFo2LAh3nzzTSiVSpkis56OHTsWGnZ97tw5hISEyBSRdWVmZsLOzvR7k1KphF6vlymiyhUWFgY/Pz/ExsYiMjISAJCamooDBw7g5Zdfljc4KzEkIOfPn8fOnTtRq1YtuUOyqqFDhxbqzxYVFYWhQ4di5MiRMkVlXUxCqqmxY8dizZo1+OGHH+Di4mJsM3Zzc4NWq5U5OstzcXEp1N/FyckJtWrVqrb9YF599VV06NAB7733HgYOHIi//voLn332GT777DO5Q7OKXr16Ye7cuahduzYiIiJw9OhRzJ8/H6NGjZI7NItJT0/HhQsXjM8vX76MY8eOwdPTE7Vr18bEiRPx7rvvol69eggLC8OMGTMQEBCA3r17yxd0BRR3vv7+/ujfvz+OHDmCn376CTqdzvh/zNPTEyqVSq6wK6Sk3/GDiZaDgwP8/PzQoEGDyg61csg9PIesA4DZ5auvvpI7tEpT3YfoCiHE//73P9GkSROhVqtFw4YNxWeffSZ3SFaTmpoqJkyYIGrXri00Go2oU6eOmDZtmsjJyZE7NIvZuXOn2b/b4cOHCyGkYbozZswQvr6+Qq1Wi0ceeUScPXtW3qAroLjzvXz5cpH/x3bu3Cl36OVW0u/4QdV9iK5CiGo03SARERFVGeyYSkRERLJgEkJERESyYBJCREREsmASQkRERLJgEkJERESyYBJCREREsmASQkRERLJgEkJERESyYBJCRDXC77//DoVCgbt378odChHdwySEiIiIZMEkhIiIiGTBJISIKoVer0d0dDTCwsKg1WrRvHlzrF+/HsD9ppItW7agWbNm0Gg0eOihh/DPP/+Y7GPDhg2IiIiAWq1GaGgo5s2bZ7I9JycHb775JoKDg6FWq1G3bl383//9n0mZw4cPo3Xr1nB0dESHDh1w9uxZ6544ERWJSQgRVYro6Gh88803+PTTT3Hy5Em8+uqrePbZZ7Fr1y5jmTfeeAPz5s3DwYMH4e3tjV69eiEvLw+AlDwMHDgQgwcPxokTJzBr1izMmDEDX3/9tfH1w4YNw7fffouFCxfi9OnTWL58OZydnU3imDZtGubNm4dDhw7B3t4eo0aNqpTzJyIz5L6NLxFVf9nZ2cLR0VH8+eefJuufe+458fTTTxtvb/7dd98Zt92+fVtotVqxdu1aIYQQzzzzjHj00UdNXv/GG2+Ixo0bCyGEOHv2rAAgtm/fbjYGwzF27NhhXLdlyxYBQGRlZVnkPImobFgTQkRWd+HCBWRmZuLRRx+Fs7Ozcfnmm29w8eJFY7n27dsbH3t6eqJBgwY4ffo0AOD06dPo2LGjyX47duyI8+fPQ6fT4dixY1AqlejSpUuxsTRr1sz42N/fHwCQmJhY4XMkorKzlzsAIqr+0tPTAQBbtmxBYGCgyTa1Wm2SiJSXVqstVTkHBwfjY4VCAUDqr0JElY81IURkdY0bN4ZarUZcXBzq1q1rsgQHBxvL7d+/3/g4OTkZ586dQ6NGjQAAjRo1wt69e032u3fvXtSvXx9KpRJNmzaFXq836WNCRLaNNSFEZHUuLi54/fXX8eqrr0Kv16NTp05ISUnB3r174erqipCQEADA7NmzUatWLfj6+mLatGnw8vJC7969AQCvvfYa2rRpgzlz5mDQoEHYt28fFi9ejKVLlwIAQkNDMXz4cIwaNQoLFy5E8+bNcfXqVSQmJmLgwIFynToRFYNJCBFVijlz5sDb2xvR0dG4dOkS3N3d0bJlS7z11lvG5pD3338fEyZMwPnz5xEZGYn//e9/UKlUAICWLVti3bp1mDlzJubMmQN/f3/Mnj0bI0aMMB5j2bJleOuttzBmzBjcvn0btWvXxltvvSXH6RJRKSiEEELuIIioZvv999/RrVs3JCcnw93dXe5wiKiSsE8IERERyYJJCBEREcmCzTFEREQkC9aEEBERkSyYhBAREZEsmIQQERGRLJiEEBERkSyYhBAREZEsmIQQERGRLJiEEBERkSyYhBAREZEs/h9knCM6+GUWHAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       "    /* background: */\n",
       "    progress::-webkit-progress-bar {background-color: #CDCDCD; width: 100%;}\n",
       "    progress {background-color: #CDCDCD;}\n",
       "\n",
       "    /* value: */\n",
       "    progress::-webkit-progress-value {background-color: #00BFFF  !important;}\n",
       "    progress::-moz-progress-bar {background-color: #00BFFF  !important;}\n",
       "    progress {color: #00BFFF ;}\n",
       "\n",
       "    /* optional */\n",
       "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
       "        background: #000000;\n",
       "    }\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      <progress value='15' class='' max='15' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      100% [15/15] [09:07]\n",
       "      <br>\n",
       "      ████████████████████100.00% [79/79] [val_loss=0.0696, val_acc=0.9855]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100%|███████████████████████████████| 79/79 [00:02<00:00, 35.97it/s, val_acc=0.989, val_loss=0.0499]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'val_loss': 0.049907402562307135, 'val_acc': 0.9885000586509705}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np \n",
    "import pandas as pd \n",
    "from matplotlib import pyplot as plt\n",
    "import torch\n",
    "from torch import nn\n",
    "import torch.nn.functional as F\n",
    "from torch.utils.data import Dataset,DataLoader\n",
    "import torchkeras #Attention this line \n",
    "\n",
    "\n",
    "#================================================================================\n",
    "# 一，准备数据\n",
    "#================================================================================\n",
    "\n",
    "import torchvision \n",
    "from torchvision import transforms\n",
    "\n",
    "transform = transforms.Compose([transforms.ToTensor()])\n",
    "ds_train = torchvision.datasets.MNIST(root=\"mnist/\",train=True,download=True,transform=transform)\n",
    "ds_val = torchvision.datasets.MNIST(root=\"mnist/\",train=False,download=True,transform=transform)\n",
    "dl_train =  torch.utils.data.DataLoader(ds_train, batch_size=128, shuffle=True, num_workers=2)\n",
    "dl_val =  torch.utils.data.DataLoader(ds_val, batch_size=128, shuffle=False, num_workers=2)\n",
    "\n",
    "for features,labels in dl_train:\n",
    "    break \n",
    "\n",
    "#================================================================================\n",
    "# 二，定义模型\n",
    "#================================================================================\n",
    "\n",
    "\n",
    "def create_net():\n",
    "    net = nn.Sequential()\n",
    "    net.add_module(\"conv1\",nn.Conv2d(in_channels=1,out_channels=64,kernel_size = 3))\n",
    "    net.add_module(\"pool1\",nn.MaxPool2d(kernel_size = 2,stride = 2))\n",
    "    net.add_module(\"conv2\",nn.Conv2d(in_channels=64,out_channels=512,kernel_size = 3))\n",
    "    net.add_module(\"pool2\",nn.MaxPool2d(kernel_size = 2,stride = 2))\n",
    "    net.add_module(\"dropout\",nn.Dropout2d(p = 0.1))\n",
    "    net.add_module(\"adaptive_pool\",nn.AdaptiveMaxPool2d((1,1)))\n",
    "    net.add_module(\"flatten\",nn.Flatten())\n",
    "    net.add_module(\"linear1\",nn.Linear(512,1024))\n",
    "    net.add_module(\"relu\",nn.ReLU())\n",
    "    net.add_module(\"linear2\",nn.Linear(1024,10))\n",
    "    return net\n",
    "\n",
    "net = create_net()\n",
    "print(net)\n",
    "\n",
    "# 评估指标\n",
    "class Accuracy(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "\n",
    "        self.correct = nn.Parameter(torch.tensor(0.0),requires_grad=False)\n",
    "        self.total = nn.Parameter(torch.tensor(0.0),requires_grad=False)\n",
    "\n",
    "    def forward(self, preds: torch.Tensor, targets: torch.Tensor):\n",
    "        preds = preds.argmax(dim=-1)\n",
    "        m = (preds == targets).sum()\n",
    "        n = targets.shape[0] \n",
    "        self.correct += m \n",
    "        self.total += n\n",
    "        \n",
    "        return m/n\n",
    "\n",
    "    def compute(self):\n",
    "        return self.correct.float() / self.total \n",
    "    \n",
    "    def reset(self):\n",
    "        self.correct -= self.correct\n",
    "        self.total -= self.total\n",
    "        \n",
    "\n",
    "\n",
    "#================================================================================\n",
    "# 三，训练模型\n",
    "#================================================================================\n",
    "\n",
    "model = torchkeras.KerasModel(net,\n",
    "      loss_fn = nn.CrossEntropyLoss(),\n",
    "      optimizer= torch.optim.Adam(net.parameters(),lr=0.001),\n",
    "      metrics_dict = {\"acc\":Accuracy()}\n",
    "    )\n",
    "\n",
    "from torchkeras import summary\n",
    "summary(model,input_data=features);\n",
    "\n",
    "\n",
    "# if gpu/mps is available, will auto use it, otherwise cpu will be used.\n",
    "\n",
    "dfhistory=model.fit(train_data=dl_train, \n",
    "                    val_data=dl_val, \n",
    "                    epochs=15, \n",
    "                    patience=5, \n",
    "                    monitor=\"val_acc\",\n",
    "                    mode=\"max\",\n",
    "                    ckpt_path='checkpoint')\n",
    "\n",
    "#================================================================================\n",
    "# 四，评估模型\n",
    "#================================================================================\n",
    "\n",
    "model.evaluate(dl_val)\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d27c94a1-8df1-4c2e-a131-578588792967",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba9f63aa-2dc8-4004-bbbf-062b42aab800",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "ddbfb751",
   "metadata": {},
   "source": [
    "## 五，M1芯片与CPU和Nvidia GPU速度对比"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b77f25cd-53a2-44a3-b308-9cafa045ee60",
   "metadata": {},
   "source": [
    "使用以上代码作为范例，分别在CPU, mac m1芯片，以及Nvidia GPU上 运行。\n",
    "\n",
    "得到的运行速度截图如下：\n",
    "\n",
    "纯CPU跑效果\n",
    "![](https://tva1.sinaimg.cn/large/008vxvgGgy1h8pu8qudibj318i0d0n06.jpg)\n",
    "\n",
    "Mac M1 芯片加速效果\n",
    "![](https://tva1.sinaimg.cn/large/008vxvgGgy1h8pubdxbrkj318u0eywhq.jpg)\n",
    "\n",
    "\n",
    "Tesla P100 GPU加速效果\n",
    "![](https://tva1.sinaimg.cn/large/008vxvgGgy1h8pu9epg15j319i0dcn0c.jpg)\n",
    "\n",
    "\n",
    "纯CPU跑一个epoch大约是3min 18s。\n",
    "\n",
    "使用mac m1芯片加速，一个epoch大约是33 s，相比CPU跑，加速约6倍。\n",
    "\n",
    "使用Nvidia Tesla P100 GPU加速，一个epoch大约是 8s，相比CPU跑，加速约25倍。\n",
    "\n",
    "这和pytorch官网显示的训练过程平均加速7倍相当。\n",
    "\n",
    "![](https://tva1.sinaimg.cn/large/008vxvgGgy1h8putb28ivj30zk0lwq4j.jpg)\n",
    "\n",
    "整体来说Mac M1芯片对 深度学习训练过程的加速还是非常显著的，通常达到5到7倍左右。\n",
    "\n",
    "不过目前看和企业中最常使用的高端的Tesla P100 GPU相比，还是有2到4倍的训练速度差异，可以视做一个mini版的GPU吧。\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "formats": "ipynb,md"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}