swift_tf.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "swift_tf",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": []
    },
    "language_info": {
      "file_extension": ".swift",
      "mimetype": "text/x-swift",
      "name": "swift",
      "version": ""
    },
    "kernelspec": {
      "display_name": "Swift",
      "language": "swift",
      "name": "swift"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "JtEZ1pCPn--z"
      },
      "source": [
        "# Model training walkthrough"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "LDrzLFXE8T1l"
      },
      "source": [
        "machine learning model that categorizes iris flowers by species. \n",
        "\n",
        "1. Build a model,\n",
        "2. Train this model on example data, and\n",
        "3. Use the model to make predictions about unknown data.\n",
        "\n",
        "## TensorFlow programming\n",
        "\n",
        "uses these high-level Swift for TensorFlow concepts:\n",
        "\n",
        "* Import data with the Datasets API.\n",
        "* Build models using Swift abstractions.\n",
        "* Use Python libraries using Swift's Python interoperability when pure Swift libraries are not available.\n",
        "\n",
        "structured like many TensorFlow programs:\n",
        "\n",
        "1. Import and parse the data sets.\n",
        "2. Select the type of model.\n",
        "3. Train the model.\n",
        "4. Evaluate the model's effectiveness.\n",
        "5. Use the trained model to make predictions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "g4Wzg69bnwK2",
        "scrolled": true,
        "colab": {}
      },
      "source": [
        "import TensorFlow\n",
        "\n",
        "import Python\n",
        "%include \"EnableIPythonDisplay.swift\"\n",
        "IPythonDisplay.shell.enable_matplotlib(\"inline\")\n",
        "let plt = Python.import(\"matplotlib.pyplot\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DgXtRpoamN3-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "// Download a helper file that helps us work around some temporary limitations\n",
        "// in the dataset API.\n",
        "import Foundation\n",
        "import FoundationNetworking\n",
        "func download(from sourceString: String, to destinationString: String) {\n",
        "    let source = URL(string: sourceString)!\n",
        "    let destination = URL(fileURLWithPath: destinationString)\n",
        "    let data = try! Data.init(contentsOf: source)\n",
        "    try! data.write(to: destination)\n",
        "}\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nWTok4IEBbL9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "download(\n",
        "    from: \"https://raw.githubusercontent.com/tensorflow/swift/master/docs/site/tutorials/TutorialDatasetCSVAPI.swift\",\n",
        "    to: \"TutorialDatasetCSVAPI.swift\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Zx7wc0LuuxaJ"
      },
      "source": [
        "## The iris classification problem\n",
        "* Iris setosa\n",
        "* Iris virginica\n",
        "* Iris versicolor\n",
        "\n",
        "<table>\n",
        "  <tr><td>\n",
        "    <img src=\"https://www.tensorflow.org/images/iris_three_species.jpg\"\n",
        "         alt=\"Petal geometry compared for three iris species: Iris setosa, Iris virginica, and Iris versicolor\">\n",
        "  </td></tr>\n",
        "  <tr><td align=\"center\">\n",
        "    <b>Figure 1.</b> <a href=\"https://commons.wikimedia.org/w/index.php?curid=170298\">Iris setosa</a> (by <a href=\"https://commons.wikimedia.org/wiki/User:Radomil\">Radomil</a>, CC BY-SA 3.0), <a href=\"https://commons.wikimedia.org/w/index.php?curid=248095\">Iris versicolor</a>, (by <a href=\"https://commons.wikimedia.org/wiki/User:Dlanglois\">Dlanglois</a>, CC BY-SA 3.0), and <a href=\"https://www.flickr.com/photos/33397993@N05/3352169862\">Iris virginica</a> (by <a href=\"https://www.flickr.com/photos/33397993@N05\">Frank Mayfield</a>, CC BY-SA 2.0).<br/>&nbsp;\n",
        "  </td></tr>\n",
        "</table>\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "DKkgac4WO0mP",
        "scrolled": true,
        "colab": {}
      },
      "source": [
        "// Import and parse the training dataset\n",
        "\n",
        "// Download the dataset file and convert it into a structure that can be used by this Swift program.\n",
        "\n",
        "// from http://download.tensorflow.org/data/iris_training.csv.\n",
        "\n",
        "let trainDataFilename = \"iris_training.csv\"\n",
        "download(from: \"http://download.tensorflow.org/data/iris_training.csv\", to: trainDataFilename)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "FQvb_JYdrpPm",
        "outputId": "178d2836-ce85-49a2-f7af-4c3f1dc08e23",
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        }
      },
      "source": [
        "// This dataset, `iris_training.csv`, is a plain text file that stores tabular data formatted as comma-separated values (CSV). Let's look a the first 5 entries.\n",
        "let f = Python.open(trainDataFilename)\n",
        "for _ in 0..<5 {\n",
        "    print(Python.next(f).strip())\n",
        "}\n",
        "f.close()\n",
        "// The first line is a header containing information about the dataset:\n",
        "//   * There are 120 total examples. Each example has four features and one of three possible label names. "
      ],
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "120,4,setosa,versicolor,virginica\r\n",
            "6.4,2.8,5.6,2.2,2\r\n",
            "5.0,2.3,3.3,1.0,1\r\n",
            "4.9,2.5,4.5,1.7,2\r\n",
            "4.9,3.1,1.5,0.1,0\r\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "None\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "9Edhevw7exl6",
        "outputId": "17607bfe-a2b7-46d4-cbba-ea23bfb19bc5",
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "let featureNames = [\"sepal_length\", \"sepal_width\", \"petal_length\", \"petal_width\"]\n",
        "let labelName = \"species\"\n",
        "let columnNames = featureNames + [labelName]\n",
        "\n",
        "print(\"Features: \\(featureNames)\")\n",
        "print(\"Label: \\(labelName)\")"
      ],
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Features: [\"sepal_length\", \"sepal_width\", \"petal_length\", \"petal_width\"]\r\n",
            "Label: species\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "CCtwLoJhhDNc"
      },
      "source": [
        "* `0`: Iris setosa\n",
        "* `1`: Iris versicolor\n",
        "* `2`: Iris virginica\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "sVNlJlUOhkoX",
        "scrolled": true,
        "colab": {}
      },
      "source": [
        "let classNames = [\"Iris setosa\", \"Iris versicolor\", \"Iris virginica\"]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "dqPkQExM2Pwt"
      },
      "source": [
        "### Create a Dataset\n",
        "\n",
        "Swift for TensorFlow's Dataset API is a high-level API for reading data and transforming it into a form used for training.\n",
        "\n",
        "Eventually, the Dataset API will be able to load data from many file formats. The Dataset API is currently very incomplete, so we include some glue code from \"TutorialDatasetCSVAPI.swift\" that helps the Dataset API load data from the CSV file into the `IrisBatch` struct."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "bBx_C6UWO0mc",
        "scrolled": true,
        "colab": {}
      },
      "source": [
        "let batchSize = 32\n",
        "\n",
        "/// A batch of examples from the iris dataset.\n",
        "struct IrisBatch {\n",
        "    /// [batchSize, featureCount] tensor of features.\n",
        "    let features: Tensor<Float>\n",
        "\n",
        "    /// [batchSize] tensor of labels.\n",
        "    let labels: Tensor<Int32>\n",
        "}\n",
        "\n",
        "%include \"TutorialDatasetCSVAPI.swift\"\n",
        "\n",
        "let trainDataset: Dataset<IrisBatch> = Dataset(\n",
        "    contentsOfCSVFile: trainDataFilename, hasHeader: true,\n",
        "    featureColumns: [0, 1, 2, 3], labelColumns: [4]\n",
        ").batched(batchSize)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "gB_RSn62c-3G"
      },
      "source": [
        "The `Dataset<IrisBatch>` is a sequence of `IrisBatch`es. Let's look at the first element of the dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "iDuG94H-C122",
        "outputId": "27f4ab30-19c1-4581-b5f6-ff691332ab2e",
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 578
        }
      },
      "source": [
        "let firstTrainExamples = trainDataset.first!\n",
        "let firstTrainFeatures = firstTrainExamples.features\n",
        "let firstTrainLabels = firstTrainExamples.labels\n",
        "print(\"First batch of features: \\(firstTrainFeatures)\")\n",
        "print(\"First batch of labels: \\(firstTrainLabels)\")"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "First batch of features: [[6.4, 2.8, 5.6, 2.2],\r\n",
            " [5.0, 2.3, 3.3, 1.0],\r\n",
            " [4.9, 2.5, 4.5, 1.7],\r\n",
            " [4.9, 3.1, 1.5, 0.1],\r\n",
            " [5.7, 3.8, 1.7, 0.3],\r\n",
            " [4.4, 3.2, 1.3, 0.2],\r\n",
            " [5.4, 3.4, 1.5, 0.4],\r\n",
            " [6.9, 3.1, 5.1, 2.3],\r\n",
            " [6.7, 3.1, 4.4, 1.4],\r\n",
            " [5.1, 3.7, 1.5, 0.4],\r\n",
            " [5.2, 2.7, 3.9, 1.4],\r\n",
            " [6.9, 3.1, 4.9, 1.5],\r\n",
            " [5.8, 4.0, 1.2, 0.2],\r\n",
            " [5.4, 3.9, 1.7, 0.4],\r\n",
            " [7.7, 3.8, 6.7, 2.2],\r\n",
            " [6.3, 3.3, 4.7, 1.6],\r\n",
            " [6.8, 3.2, 5.9, 2.3],\r\n",
            " [7.6, 3.0, 6.6, 2.1],\r\n",
            " [6.4, 3.2, 5.3, 2.3],\r\n",
            " [5.7, 4.4, 1.5, 0.4],\r\n",
            " [6.7, 3.3, 5.7, 2.1],\r\n",
            " [6.4, 2.8, 5.6, 2.1],\r\n",
            " [5.4, 3.9, 1.3, 0.4],\r\n",
            " [6.1, 2.6, 5.6, 1.4],\r\n",
            " [7.2, 3.0, 5.8, 1.6],\r\n",
            " [5.2, 3.5, 1.5, 0.2],\r\n",
            " [5.8, 2.6, 4.0, 1.2],\r\n",
            " [5.9, 3.0, 5.1, 1.8],\r\n",
            " [5.4, 3.0, 4.5, 1.5],\r\n",
            " [6.7, 3.0, 5.0, 1.7],\r\n",
            " [6.3, 2.3, 4.4, 1.3],\r\n",
            " [5.1, 2.5, 3.0, 1.1]]\r\n",
            "First batch of labels: [2, 1, 2, 0, 0, 0, 0, 2, 1, 0, 1, 1, 0, 0, 2, 1, 2, 2, 2, 0, 2, 2, 0, 2, 2, 0, 1, 2, 1, 1, 1, 1]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "E63mArnQaAGz"
      },
      "source": [
        "Notice that the features for the first `batchSize` examples are grouped together (or *batched*) into `firstTrainFeatures`, and that the labels for the first `batchSize` examples are batched into `firstTrainLabels`.\n",
        "\n",
        "You can start to see some clusters by plotting a few features from the batch, using Python's matplotlib:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "me5Wn-9FcyyO",
        "outputId": "143926b4-c980-42d8-bad1-7e2c2a42a847",
        "scrolled": false,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 300
        }
      },
      "source": [
        "let firstTrainFeaturesTransposed = firstTrainFeatures.transposed()\n",
        "let petalLengths = firstTrainFeaturesTransposed[2].scalars\n",
        "let sepalLengths = firstTrainFeaturesTransposed[0].scalars\n",
        "\n",
        "plt.scatter(petalLengths, sepalLengths, c: firstTrainLabels.array.scalars)\n",
        "plt.xlabel(\"Petal length\")\n",
        "plt.ylabel(\"Sepal length\")\n",
        "plt.show()"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xmcm2W5//HPlUkyW6ctbQcKdIXW\nggUKdFjKJlBBKFgUkLJUZBNRBCoe/bmgIoogB0UED8uBg3jYobKoLHIQBIQiUyhradkKbYHudJs1\nyfX7IyHMkplJO0meSeb7fr3m1eR+7nnyTSlzTZ77fu7b3B0RERGAUNABRESk71BREBGRNBUFERFJ\nU1EQEZE0FQUREUlTURARkTQVBRERSVNREBGRNBUFERFJCwcdYFMNGzbMx4wZE3QMEZGiMnfu3JXu\nXttTv6IrCmPGjKG+vj7oGCIiRcXM3sumny4fiYhImoqCiIikqSiIiEiaioKIiKSpKIiISFrRzT4S\nEelP3Juh+XGIr4BoHRbZMa+vp6IgItJHeeub+OqZQAt4DDC8/CBs8G8xK8vLa+rykYhIH+Tu+Mff\nBF8DvhFoBpqg+QlovDdvr6uiICLSF8XfSV4y6qQRb7g9by+roiAi0hd5DMy6OBjL28uqKIiI9EXh\n8WDVGQ5UQMX0vL2sioKISB/g7njLPHzDNXjDHeDrscFXgFUB5clOVgWRCVj1zLzl0OwjEZGAucfx\nj2dB85MkB5TLYf2l2Bb/jQ17FG+8DxLLsOjeUH4gZvn70a2iICIStKa/QcuTQGOqoREcfM23sS3/\nRWjA1wsWRZePREQC5o2zwRszHGmB1lcKmkVFQUSkT/OCvpqKgohIwKzyaLDKDEciENmloFnyVhTM\nbIKZzWvztc7MZnXoc6CZrW3T56f5yiMi0mdVHAnR/YFKkj+WK8CqsC2uzuugciZ5ezV3XwDsCmDJ\nRTqWApnuzX7K3Y/MVw4Rkb7OrAwGXwWtL0LLHAgNhoppWGhwwbMUqgRNBd5296z2CBUR6W/MDKK7\nJ78CVKgxheOBrhbrmGJmL5nZQ2Y2sUB5REQkg7wXBTOLAtOBuzMcfgEY7e6TgKuA+7o4x5lmVm9m\n9StWZFogSkREcqEQnxQOB15w92UdD7j7OnffkHr8IBAxs2EZ+l3v7nXuXldbW5v/xCIi/VQhisIJ\ndHHpyMyGmyWXATSzPVN5VhUgk4iIZJDXgWYzqwYOAb7Rpu0sAHe/FjgW+KaZxUje3328uxf2Tg0R\nEUnLa1Fw943A0A5t17Z5fDVwdT4ziIhI9nRHs4iIpKkoiIhImoqCiIikqSiIiEiaioKIiKSpKIiI\nSJqKgoiIpKkoiIhImoqCiIikFXZLHxGRAvH4SrzhZmieA+GRWPXpWESr8/dERUFESo7HP8JXHgW+\nEWiB2Ct40//B4CuwiqlBx+vTdPlIREqOb/g9+DqgJdWSAJrwdT/BPRFgsr5PRUFESk/zU0C8c3ti\nA8Q/KHicYqKiICKlp8sN7xMQqilolGKjoiAiJceqTwMqO7RGoHw/LDQoiEhFQ0VBREpPxZegaiZQ\nDlaT/DOyGzbosqCT9XmafSQiJcfMsIHfwwecAa0LoWw4Fh4ddKyioKIgIiXLQltA+V5Bxygqunwk\nIiJpKgoiIpKmoiAiIml5KwpmNsHM5rX5Wmdmszr0MTP7vZm9ZWYvm9nu+cojIiI9y9tAs7svAHYF\nMLMyYClwb4duhwPjU197Adek/hQRkQAU6vLRVOBtd3+vQ/tRwJ88aQ4w2My2LlAmERHpoFBF4Xjg\n9gzt2wKL2zxfkmoTEZEA5L0omFkUmA7c3YtznGlm9WZWv2LFityFExGRdgrxSeFw4AV3X5bh2FJg\nZJvnI1Jt7bj79e5e5+51tbW1eYopIiKFuKP5BDJfOgJ4APi2md1BcoB5rbt/WIBMItKF9c3NXDHn\nGR5YOB+A6Z/Zke/svQ815eWd+npiI77hKmi6HzwBFdOwmlladK6I5bUomFk1cAjwjTZtZwG4+7XA\ng8A04C2gATg1n3lEpHvxRIIZ99zBO2tW05JIbkZz2yvzeHbJ+/z1hK9SFvr04oJ7Al/9VYgtJL2Z\nTeNdeMszMOyvmEUCeAfSW3ktCu6+ERjaoe3aNo8dODufGUQke0+89y6L161NFwSAlkSCJevW8sR7\n7zJ17Pafdm55FuLv8OnuZgCtkFgGzY9BxWEFyy25ozuaRSRt/ooVNLS2dmpvaG1lfsdJHrE3wFs6\n9cUb8NbX8pRQ8k1FQUTSRgwcRGWk82WfqkiEEQM7jBOUjQDrPM4AVVjZqPwElLxTURCRtMPGjaMq\nEiFklm4LmVEZjnDYuHHtO5cfBDaA9j9GLFkoKqYVJK/knoqCiKRVhCPc85UTmLz1NoRDIcKhEJO3\n3oZ7jjuBinD7TxBmUWzonRDdg+TwZBgik7Chd2Ch6kDy98S9mcS6S0ksqyPx0U4kVp+BxxYFHatP\nseRYb/Goq6vz+vr6oGOIlLyNLcnxgupotMe+ntgIOBYakOdUvZNYfQa0PAc0p1oMrAYb9ghWNrS7\nby16ZjbX3et66qdPCiKSUXU0mlVBALBQdZ8vCB57C1r+zacFAcDBm/HGrm6l6n9UFESkf4i9BZZp\nFn4ztL5S8Dh9lYqCiPQPZWPBYxkORCG8Y8Hj9FUqCiJFwt1pjceDjkFrPE5fHIt0T+AZf+gnWWQC\nRHYBOlwSsyhWdWJ+wxWRQqx9JCK94O7c+OJc/qv+OdY2NbH1gBp+sN/nOPIzEwqa4+7XXuHyZ//F\nyoaNDKmsYtbe+3DSzpMKmiETT6zH110ETQ8CcTyyGzboF1h4XKe+tsV1+PpfQeP9QCtEdsMG/hwr\n27LgufsqzT4S6eOurf83V/37WRpjn/4WXBEOc/XhX+TgsdsVJMO9b7zOBf94tF2GynCYnxxwEMfv\ntEtBMmTi7viqryTvrk4vt2FgA7DaR7HQkC6/DxIkN4XsHzT7SKQExBMJrql/rt0PY4CmWIzfPPt0\nwXJc8ey/OmVojMW4Ys4zBcuQUevLEHuT9usvOXgL3nBnl99mZv2qIGwKFQWRPmxDSwtNsczXyd9f\nu7ZgOT7csD5j+4qGjSSCvNoQXwSW6cdYM8QWFDpNSVBREOnDasrLu7xXYNyQzJdG8mHkoMz7I2xT\nU9NuSYyCC49P7uPQSQWEdy54nFKgoiDSh4XMmLXXPlSG288JqQiH+d4++xcsxw/2PYCKDBm+X8AM\nmVjksxCZBLRdmC8EVolVHRtUrKKmoiDSx508aTcuOvDzjKgZSLSsjAlDh3HdEUexz8jCrUR66Pbj\nufILR7D9FkOIhEKMHbwFlx9yGNMnBD+/34ZcD1UngA0EyqH8IGzobO3+tpk0+0hEpB/Q7CMREdlk\nKgoiIpKmoiAiImkqCiIikpbXomBmg83sHjN7w8zmm9mUDscPNLO1ZjYv9fXTfOYREZHu9bggnpnt\nC1wIjE71N8DdPZtFV64EHnb3Y80sClRl6POUux+ZfWQREcmXbFZJvRH4DjAXyHrdXjMbBBwAnALg\n7i20X6BERET6mGwuH61194fcfbm7r/rkK4vvGwusAG4ysxfN7AYzy7Sb9xQze8nMHjKziZlOZGZn\nmlm9mdWvWLEii5cWEZHN0WVRMLPdzWx34HEz+08zm/JJW6q9J2Fgd+Aad98N2Aj8oEOfF4DR7j4J\nuAq4L9OJ3P16d69z97ra2tps3peIiGyG7i4f/abD87Z3wjlwcA/nXgIscffnUs/voUNRcPd1bR4/\naGb/ZWbD3H1lD+cWkSLk7hB/J7ktZng8lnGF0+4lYsugdQ6EdyQU+UweUvZvXRYFdz8IwMy2c/d3\n2h4zsx4Hmd39IzNbbGYT3H0BMBV4vcN5hgPL3N3NbE+Sn1yyuTQlIkXGW9/EP/4WxJeDGVg1DL4C\ni+6Z1fcnEglYMxNaP13mJhHaCob9hVBocL5i9zvZlOl7MrTdneX5zwFuNbOXgV2BX5nZWWZ2Vur4\nscCrZvYS8HvgeC+2xZhEpEfuzfjqmRB/H2gEb4DECnzN1/F4luOE637SriAAkFgGK7Uaai51+UnB\nzHYAJgKDzOzoNocGAhXZnNzd59H+shPAtW2OXw1cnXVaESlOzY+TnHzY4Xc+T+CN92EDvt7zOZoy\nDjlC4n0SidWEuth6UzZNd2MKE4AjgcHAF9u0rwey+C8oIpISX5kcR+ikGeLLsjxJ5h3okudfDioK\nOdHdmML9wP1mNsXdny1gJhEpNdHJJO977cCqsPK9sjtHqBYSyzMdgLJxvUknbWRz89qJZnZCh7a1\nQH2qcIhIASxeu5bF69YyfshQaqsz3fLTvdZ4nJeWfUSZGbtsNZyyUOGWPrPIjnj5QdD8BNCYaq1I\nbqdZflB2Jxl4MXyc4SJF5emEQtn8KJNsZPM3WQ7swKeDy8cA7wKTzOwgd5+Vr3AiAg2trZz94APM\nWbKEaFmI5nicY3acyC8O+nzW+yM//f57nPPQX4i74w7l4TKuO/IoJm+9bZ7Tf8oG/xYa78Ubbgdi\nUDEdqz4Js+x+oIcqPkdii1tg3U8hvhhCA6F6FqHqGfkN3s/0uPOamc0B9nX3eOp5GHgK2A94xd0/\nm/eUbWjnNelvvvv3h3jwzQU0xz9dZaYyHOb8Kfty+m49bqTFioaNHPjHG2iMtb8mXx2J8Mxp36Cm\nvLyL75RSksud17YABrR5Xg0MSRWJ5s3MJyJZaI7F+FuHggDQGItx07wXsjrHXxa8QSLDL38OPPL2\nm7mIKSUkm89tlwHzzOwJkiNFB5C836Aa+L88ZhPp95rj8Yw/0AHWN2f3O9nqxsZORQWgNZ7g46am\nXuWT0tPjJwV3vxHYh+S6RPcC+7n7De6+0d2/l++AIv1ZTTTKiIEDO7WHzJgyYlRW59h35CiqIpFO\n7eGQMWXEyF5nlNKS7fSDEMkVT9cA48zsgPxFEpFPmBm/OvhQKsNhylKDypFQiOpIlP+3X3b/G+49\nYiR7bTuCqvCnhaEqHOHQ7cczccut8pJbilc2A82/BmYArwGJVLO7+/Q8Z8tIA83SH725ahU3vFjP\n26tXsdvW23D6bpMZPqAm6++PJRI8sGA+s+e/RpmFOG7iTkwbPyHr2UtS/LIdaM6mKCwAdnH3PjGo\nrKIgIrLpcjn76B2g8wVJEREpOdnMPmogOfvoMdpMQXX3c/OWSkREApFNUXgg9SUiIiWux6Lg7jeb\nWSUwKrVZjohIYLx1AbS+CmXbQnTPzdq9TbrWY1Ewsy8ClwNRYKyZ7QpcFNTsIxHpn9xb8I/PgeZn\n4ZNCENoShtyKlWnv9lzJpsReCOwJfAzpjXN63I5TRCSXfOONyYJAU3LnNm+A+GJ87X8EHa2kZFMU\nWt19bYe2RMaeIiL50nAX0HFZjji01OOJ9UEkKknZDDS/ZmYnAmVmNh44F3gmv7FERDrq7lap1oKl\nKHXZfFI4h+Rezc3A7cA6QHsoiEhhlR9Cxt9jy0Zh2oozZ7JZEK/B3X/s7nu4e13qcVZLK5rZYDO7\nx8zeMLP5Zjalw3Ezs9+b2Vtm9rKZ7b65b0RESpvVnJccWKYy1RIFq8YGXxZkrJLT5eUjM/sLySXX\nM8py9tGVwMPufqyZRYGqDscPB8anvvYCrkn9KSLSjoWGwLAH8ca/QGs9hMdilV/RzKMc625M4fLe\nnNjMBpHce+EUAHdvAVo6dDsK+JMnF2Cak/pksbW7f9ib1xaR0mShKqx6Bsk1OiUfuiwK7v7PXp57\nLMnltm8ys0nAXOA8d9/Yps+2wOI2z5ek2lQUREQCkM9bAcPA7sA17r4bsBH4weacyMzONLN6M6tf\nsWJFLjOKiEgb+SwKS4Al7v5c6vk9JItEW0uBtls/jUi1tePu16cGuetqa3X9UEQkX/JWFNz9I2Cx\nmU1INU0FXu/Q7QHg5NQspL2BtRpPECkuHnsf33gDvuE6PPZ20HGkl/I9++gc4NbUzKN3gFPN7KzU\n918LPAhMA94iuUT3qdlHF5GgJTbeAut/TXKRA4cNf8AHnEVowLeCjiabqcud18zsc919Yw4GojeL\ndl4T6Rs8/iG+4lA632lcgQ2djUXGBxFLupDtzmv5nH0kIqWs6f+6ONCKNz2iolCkslk6ezxwCfBZ\noOKTdnfXSqki/Zqlvro6JsUom4Hmm0jeaRwDDgL+BNySz1AiUgQqPk/mYccIVvmFQqeRHMlmldRK\nd3/MzMzd3wMuNLO5wE/znC1n3J3X/vUGT9/7b6IVEaaetD+jPzuy528U2USrGxv48/zXWbxuLXXb\nbMsXth9PtKws6Fh5YWXD8Zofw/qLSRYHB0Iw4NtYeFzA6WRzdTnQnO5g9gywH8n7DP5B8j6CS919\nQrffmCebOtDs7lxx5nU8fsfTNDe0ECozyiJhzrxsJkedfXgek0p/8/Kyj5j557uJeYKmWIyqSIRt\nBtQw+7gTqSkvDzpe3nj8A2j6OxCH8kOw8KigI0kG2Q40Z3P56DySC9mdC0wGvgp8rXfxCuflJ1/n\n8TuepmljM+5OPJagpbGF67/3v6z+aE3Q8aREuDvfeeRBNrS20BSLAdDQ2sr769ZyTf1zPXx3cbOy\nbbDqU7Dq01UQSkA2S2c/7+4bSO6jcK67H+3uc/IfLTeemj2H5obOm3OEwiGef3heAImkFC3buIEP\n1q/r1N4Sj/OXhQsCSCSyeXosCmZWZ2avAC8Dr5jZS2Y2Of/RciMSDWPWeSaEYUSi2QypiPSsLBTq\n8k7PSCifq8mI5FY2/1r/B/iWu49x9zHA2SRnJBWFqTMPIFIe6dSeSCTY6wjt6SO5UVtVzY7Dagl1\n+AWkIhxmxsSdA0olsumyKQpxd3/qkyfu/jTJ6alFYdyuY/nqhccRrYhQXhWlckAF5ZVRLrjjfKoH\nVWd9njXL13Lbr2bzy+Ov4O7fPMD6NRu67Lt25Tru+PV9/HLGb7nzsvtYt0qbivcHvz/sSGqrqqmO\nRCkvC1MZDrPntiM4dbei+WAtktXso9+R3P/udpJzzmYATaTuVXD3F/KcsZ3NXeZixZJV/PvBF4hW\nRJkyvY4Bg7MvCIteW8ys/S6gtbmVlqZWyiujVFRX8IfnL2Wr0e1XbV2y8APOmfIjWppaaWlsIVoZ\npbwyylVzfsW247be5NxSXFrjcZ58bxEfbFjPpK2Gs8tWw4OOJAJkP/som6LweDeH3d0P3tRwvRHE\n2kfn7ftjXn92Ybu2UMiYctQeXDj7e+3av3/IRcz7x6u0/Xu1kDH5kElc8tCPC5JXRKSjXq999Al3\nPyg3kYpTrDXGG8+92ak9kXDqO8xecndeeuI1OhZaTzgvPvZKXnOKiORCNrOPtjKzG83sodTzz5rZ\n6fmP1jdYyAiFM9+RGqnoPIAd7mJGU6RcM51EpO/LZqD5j8AjwDap5wuBWfkK1NeUlZVxwLFTOk1f\njVZEOOy09lfOzIypJ+3fqQBEyiMccvKB+Y4qItJr2RSFYe5+F8ldNHD3GBDPa6o+5tw/nM7YSaOp\nqC5Pzl6qKmfivjtwykUzOvU96zdfY/zk7dr1nbDn9nz9spkBJBcR2TTZXNPYaGZDSS2H+Mm2mXlN\n1cdUD6rm6jmXsOD5t1iy8EPG7jyK7SeNydi3qqaSK5++mIVz3+b9+UsZ/dkRjN9dq4yLSHHIZvbR\n7sBVwE7Aq0AtcKy7v5z/eJ1p5zURkU2Xy9lHL6S25pxAcueMBe7emoOMIiLSx3Q5pmBme5jZcEiP\nI0wGLgZ+Y2ZDCpRPREQKqLuB5uuAFgAzOwC4lOSua2uB6/MfTURECq27y0dl7r469XgGcL27zwZm\nm1lWa06b2SJgPcnZSrGO17PM7EDgfuDdVNOf3f2i7OOLiEgudVsUzCycunQ0FTgzy+/r6CB3X9nN\n8afc/chNOJ+IiORJdz/cbwf+aWYrgUbgKQAzG0c/m5IqItJfdDmm4O4XA98leUfzfv7p3NUQcE6W\n53fg72Y218zO7KLPlNTGPQ+Z2cQszysiInnQ7WWgTNtuuvvCTH27sJ+7LzWzLYFHzewNd3+yzfEX\ngNHuvsHMpgH3AeM7niRVUM4EGDVKe8CKiORLXvcJdPelqT+XA/cCe3Y4vi61/zPu/iAQMbNhGc5z\nvbvXuXtdbW1tx8MiIpIjeVu608yqgZC7r089PhS4qEOf4cAyd3cz25NkkVqVr0wdvT5nIbddPJsl\nCz9gwh7jOOmCYxm1w7YZ+y54/i1uvXg2789fwrjdxnLSBccydqfMn1reevFdbvnlPSx65X3G7jKa\nmT85tstlMUQ2V2NrKzfNe4H7F8wnEgpxwk67MGOnXQhrT2jphR6XudjsE5ttR/LTASSLz23ufrGZ\nnQXg7tea2beBb5Lc3rMRON/dn+nuvLla5uK5v83lFzN+S3NDC5DcNCdaWc7vnv5Fpx/gcx99iZ99\n+TJaGltwTy6nXV4R5fLHL2TCHuPa9X35ydf50bRfpfo6Zka0Msqlj1zATvvu0OvcIgCxRIKj77qN\nN1etojme3B23Mhxm/9FjuPaIowJOJ31Rtstc5O1XCnd/x90npb4mpgaucfdr3f3a1OOrU8cmufve\nPRWEHGbjqm/fmC4IkNw0p2ljE9d/73879b/6nGTfT+qnJ5ymhmauOf/mTn3/cN7/0NzQnN5ox91p\nbmjmv2bdlJ83I/3SY+++zbtrVqcLAkBjLMZT7y3iteXLAkwmxa5ffs5sWN/IyqWrMx6b32GXtdaW\nVpa+9VHGvgvr32733N155+X3MvZ9e96iTQ8q0oXnlixmY2vnJcgS7sz98IMAEkmp6JdFobwySjiS\neTe1QcNq2j0PR8JUVJVn7DtwaPu+ZsaAQdUZ+w4YnLldZHMMH1BDeVnnf8PhUBm11fq3JpuvXxaF\ncCTMYacfTHlltF17eVU5M/7fl9q1mRlf/OahlFd17nvc96Z3OvfRs6ZR3qGIlFeVc+z5umlbcufL\nO36Wsg4DygZUhMs4eIz275DN1y+LAsA3Lj+Zzx23D9GKCFUDKymvjHLMrCM44uuf79T31F+ewNST\nDkj3jVZEmf6tL/Dlc6d16nvij4/hC6ce2KZvhGlnTO1UbER6o7aqmj8edQzDBwygMhyhIhxmuy2G\ncMcxMygPaz9w2Xx5m32UL7neZGfd6vWsWrqa4WO3pHJAZbd916/ZwIrFqxg+dkuqarrvu3HtRpa9\nt5KtRg+juotLSiK95e68s2Y1kbIyRg0aHHQc6cNytslOqRs4pIaBQ2p67gjUbDGAmi0GZNW3elA1\n2+2iYiD5ZWZsP2Ro0DGkhPTby0ciItKZioKIiKSpKIiISFq/H1PI1jP3P88ff3oHH727nJE7bMvp\nl5zE7lN3DjqWBGTp+nVc+vSTPPneu1SGI5y48y58s24vIhnuHRApJv1+9lE2/nHH0/z2jGvaLYtR\nXhnl5/d9n8mHTCpoFgnemsZGDrnlJj5ubCJB8v+finCYz40ewzVad0j6qMDXPioV7s5/f/9/2xUE\ngObGFq7/fud1kqT03fHayzS0tqYLAkBTLMYTixbxzprMy6eIFAsVhR60tsRY9cGajMcWv6E1Zvqj\nFz78kKZYrFN7pCzEglXdbUcu0vepKPQgEg0zYFBVxmPDth1S4DTSF4wfMoRIqPPYQTzhjBo4KIBE\nIrmjotADM+OEHx1NRXXn9YxOvvC4gFJJkGbusiuRsvb/60RCIT4zdCgTt9wqoFQiuaGikIVjz/8i\ne07bHTMDkhvyHHLyAXx+5gEBJys+zbEYlzz9TyZdexWfufoKTpx9Z9FdctmmZiC3Hn0cOw6rpcxC\nREIhDt1+HDd/6Zigo4n0mmYfZeGhGx/jD+fdRHNDc7qtvCrKj26bxT7T9yholmJ31l/v58n3FtHU\nZnOYAdEoj5x0ClvXZLfcSF+yoaWFSCikReikz9Psoxxxd2664PZ2BQGguaGFG394W0CpitP7az/m\nnx0KAkBzLM4fX3ohoFS9MyAaVUGQkqKi0IPWlhgfr1iX8diH72jbw03x9prVRMs6/5NrTcR5RVtI\nivQJKgo9iETDnXZY+8TwMbUFTlPcths8hJZ4olN7JBRiYu2WASQSkY5UFHpgZnzt58dl2E0tyqkX\nnxhQquI0evBg9h05qtM2ktGyMk7ZdfdNPl9zLEaiyMbERPq6vF4MNbNFwHogDsQ6DnJYcjrPlcA0\noAE4xd373MXlI79xKKGyEH/62V2s/uhjthw9jK9fOpP9j94r6GhF5w/TvsglTz/J3a+/SlOsld2G\nb81FB32ebWsGZn2Ofy9dwgWPP8o7a9YQCZUxY+JO/HC/z+navkgO5HX2Uaoo1Ll7xjmHZjYNOIdk\nUdgLuNLdu/1JG8Tso7bcPT01VXpnc/4uF65ayZfvvJXGNncUl5eFmTp2O66e9sVcRxQpGcUy++go\n4E+eNAcYbGZbB5ypWyoIubM5f5fXzX2elni8XVtzPMZj777Nsg0bchVNpN/Kd1Fw4O9mNtfMzsxw\nfFtgcZvnS1Jt7ZjZmWZWb2b1K1asyFNUKQYLV60knuHTbbQszPvrPg4gkUhpyXdR2M/ddwcOB842\ns826Bdjdr3f3Onevq63VjJ/+bJethhPO8AmjJR5ju8Fai0qkt/JaFNx9aerP5cC9wJ4duiwFRrZ5\nPiLV1ie1trSy+qM1xGPxnjtLXnxj8h6dBpQrw2GO3nEiQ6syL1xY6jzxMZ5YH3QMKRF5KwpmVm1m\nNZ88Bg4FXu3Q7QHgZEvaG1jr7h/mK9PmSiQS3HzhXRw97DS+ut3ZHFN7GrN/91eKbYmQUjBq0GDu\n/soJ7DNyFBXhMFtWVXPOnlO46MCpQUcrOG9dSGLlUfjyffHle5NYNROP97n/faTI5HMO31bAvanB\nxDBwm7s/bGZnAbj7tcCDJGcevUVySuqpecyz2e789X3cffkD6aUuWppauemCO6geVMVhpx4ccLr+\nZ4dhtdzy5a8EHSNQnliLrz4BvM0nhNa5+KrjofYxzDQ9VzZP3v7luPs7QKe9KlPF4JPHDpydrwy5\n4O7c+Z/3Z1j7qJlbfnGPioIEwhvvB2/t0BoHXwfNT0HFQYHkkuIX9JTUPq+1JUbDusaMx1Z/qNku\nEpDYe0BT53aPQXxJweNI6VCzcAf8AAAIa0lEQVRR6EEkGqZ2xNCMx8ZMHJmxXSTfLLorWIaBdSuD\nyE6FDyQlQ0WhB2bGWb/5GuWV0Xbt5ZVRvnH5yQGlkn6v4gsQ2hKItGksh/BOENk1qFRSAjQalYX9\nj9mbigEV3PzTO/ng7Y8YM3Ekp118Ajvtt2PQ0aSfMovC0Lvw9VdB84NABCqPwQacpbvupVe085qI\nSD9QLGsfiYhIH6KiICIiaSoKIiKSpqIgIiJpKgoiIpKmoiAiImkqCiIikqaiICIiaSoKIiKSpqIg\nIiJpKgoiIpKmoiAiImkqCiIikqaiICIiaSoKIiKSlveiYGZlZvaimf01w7FTzGyFmc1LfZ2R7zwi\nItK1Quy8dh4wHxjYxfE73f3bBcghIiI9yOsnBTMbARwB3JDP1xERkdzI9+Wj3wHfBxLd9DnGzF42\ns3vMbGSe84iISDfyVhTM7EhgubvP7abbX4Ax7r4L8ChwcxfnOtPM6s2sfsWKFXlIKyIikN9PCvsC\n081sEXAHcLCZ3dK2g7uvcvfm1NMbgMmZTuTu17t7nbvX1dbW5jGyiEj/lrei4O4/dPcR7j4GOB74\nh7vPbNvHzLZu83Q6yQHpgkkkEsx7/FUevOExFtS/XciXFhHpkwox+6gdM7sIqHf3B4BzzWw6EANW\nA6cUKsea5Wv57oE/Y+WSVbg77rDDnuO4+G8/pLyyvFAxRET6FHP3oDNskrq6Oq+vr+/1eX4y/VKe\nf3ge8Vg83RatiPDl847gjEtO6vX5RUT6EjOb6+51PfXrl3c0tzS1UP9I+4KQbG/lkZseDyiViEjw\n+mVRiMcTdPUJKdYSK3AaEZG+o18WhcrqCsbttl2n9rJwGVOm9/jpSkSkZPXLogDw3Ru/SfWgKqKV\nUQAqqsvZYqtBnHGpxhNEpP8q+OyjvmLsTqP401tX8/BNj7P4jaVM2GMcU2fuT2V1RdDRREQC02+L\nAsDAoTUc9x/Tg44hItJn9NvLRyIi0pmKgoiIpKkoiIhImoqCiIikqSiIiEiaioKIiKQV3YJ4ZrYC\neK9D8zBgZQBxCkHvrTiV8nuD0n5/pfreRrt7jxvSFF1RyMTM6rNZ/a8Y6b0Vp1J+b1Da76+U31s2\ndPlIRETSVBRERCStVIrC9UEHyCO9t+JUyu8NSvv9lfJ761FJjCmIiEhulMonBRERyYGiLgpm9j9m\nttzMXg06S66Z2Ugze9zMXjez18zsvKAz5YqZVZjZv83spdR7+3nQmXLNzMrM7EUz+2vQWXLJzBaZ\n2StmNs/Mer9Zeh9iZoPN7B4ze8PM5pvZlKAzBaGoLx+Z2QHABuBP7r5T0Hlyycy2BrZ29xfMrAaY\nC3zJ3V8POFqvmZkB1e6+wcwiwNPAee4+J+BoOWNm5wN1wEB3PzLoPLliZouAOncvuXn8ZnYz8JS7\n32BmUaDK3T8OOlehFfUnBXd/ElgddI58cPcP3f2F1OP1wHxg22BT5YYnbUg9jaS+ive3kw7MbARw\nBHBD0FkkO2Y2CDgAuBHA3Vv6Y0GAIi8K/YWZjQF2A54LNknupC6vzAOWA4+6e8m8N+B3wPeBRNBB\n8sCBv5vZXDM7M+gwOTQWWAHclLrsd4OZVQcdKggqCn2cmQ0AZgOz3H1d0Hlyxd3j7r4rMALY08xK\n4vKfmR0JLHf3uUFnyZP93H134HDg7NQl3FIQBnYHrnH33YCNwA+CjRQMFYU+LHW9fTZwq7v/Oeg8\n+ZD6iP44cFjQWXJkX2B66tr7HcDBZnZLsJFyx92Xpv5cDtwL7BlsopxZAixp84n1HpJFot9RUeij\nUoOxNwLz3f23QefJJTOrNbPBqceVwCHAG8Gmyg13/6G7j3D3McDxwD/cfWbAsXLCzKpTkx5IXVo5\nFCiJmX/u/hGw2MwmpJqmAkU/qWNzhIMO0BtmdjtwIDDMzJYAP3P3G4NNlTP7Al8FXkldewf4kbs/\nGGCmXNkauNnMykj+YnKXu5fU1M0StRVwb/L3FcLAbe7+cLCRcuoc4NbUzKN3gFMDzhOIop6SKiIi\nuaXLRyIikqaiICIiaSoKIiKSpqIgIiJpKgoiIpKmoiAly8ziqdU8XzWzu82sqof+P8ryvIvMbFi2\n7b1hZmPM7MQ2z08xs6tz+RoibakoSClrdPddUyvotgBn9dA/q6JQYGOAE3vqJJIrKgrSXzwFjAMw\ns5mp/Rzmmdl1qcX5LgUqU223pvrdl1r47bVNXfwt02uk2jeY2cWpvSTmmNlWqfbtU89fMbNfmtkn\nq8heCuyfOs93Um3bmNnDZvammV2Wg78bkTQVBSl5ZhYmuYDbK2a2IzAD2De1IF8cOMndf8CnnyxO\nSn3rae4+meS+COea2dAsXy/ja6QOVwNz3H0S8CTw9VT7lcCV7r4zyXV4PvEDkmv87+ruV6Tadk2d\nf2dghpmN3KS/EJFuFPUyFyI9qGyzRMhTJNeSOhOYDDyfWq6hkuTy3Zmca2ZfTj0eCYwHVmXxulO7\neY0W4JMlPeaSXPcJYArwpdTj24DLuzn/Y+6+FsDMXgdGA4uzyCXSIxUFKWWNqd/U01ILDd7s7j/s\n7hvN7EDg88AUd28wsyeAiixft7vXaPVP15aJs3n/Dza3eby55xDJSJePpL95DDjWzLYEMLMhZjY6\ndaw1tVw5wCBgTaog7ADsnaPX6Moc4JjU4+PbtK8HajbhtUV6RUVB+pXUHtcXkNw97GXgUZKrtgJc\nD7ycGmh+GAib2XySg71Z7x/dw2t0ZRZwfqr/OGBtqv1lIJ4amP5Ol98tkiNaJVWkD0jdQ9Ho7m5m\nxwMnuPtRQeeS/kfXIkX6hsnA1akxj4+B0wLOI/2UPimIiEiaxhRERCRNRUFERNJUFEREJE1FQURE\n0lQUREQkTUVBRETS/j9xaEVFAbAuXgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "None\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "W23DIMVPQEBt"
      },
      "source": [
        "A model is a `struct` that conforms to [`Layer`](https://www.tensorflow.org/swift/api_docs/Protocols/Layer), defines a [`call(_:)`](https://www.tensorflow.org/swift/api_docs/Protocols/Layer#call_:) method that maps input `Tensor`s to output `Tensor`s. The `call(_:)` method often simply sequences the input through sublayers. \n",
        "`IrisModel` sequences the input through three [`Dense`](https://www.tensorflow.org/swift/api_docs/Structs/Dense) sublayers."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "wr5A5WvthvZ0",
        "scrolled": true,
        "colab": {}
      },
      "source": [
        "let hiddenSize: Int = 20\n",
        "struct IrisModel: Layer {\n",
        "    var layer1 = Dense<Float>(inputSize: 4, outputSize: hiddenSize, activation: relu)\n",
        "    var layer2 = Dense<Float>(inputSize: hiddenSize, outputSize: hiddenSize, activation: relu)\n",
        "    var layer3 = Dense<Float>(inputSize: hiddenSize, outputSize: 3)\n",
        "    \n",
        "    @differentiable\n",
        "    func callAsFunction(_ input: Tensor<Float>) -> Tensor<Float> {\n",
        "        return input.sequenced(through: layer1, layer2, layer3)\n",
        "    }\n",
        "}\n",
        "\n",
        "var model = IrisModel()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "fK0vrIRv_tcc"
      },
      "source": [
        "The activation function determines the output shape of each node in the layer.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "sKjJGIYzO0mr",
        "outputId": "af5645b5-cc75-4c2f-df0f-9979ca1bc9df",
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "// Apply the model to a batch of features.\n",
        "let firstTrainPredictions = model(firstTrainFeatures)\n",
        "firstTrainPredictions[0..<5]"
      ],
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[[ -1.7329772, -0.13151097,  0.19777158],\n",
              " [ -1.1974344, -0.06770605,   0.1761316],\n",
              " [  -1.349941, -0.14748853,  0.13317056],\n",
              " [ -1.0228678, 0.017132223,   0.1877974],\n",
              " [ -1.1936471, 0.013257623,  0.21275392]]\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "wxyXOhwVr5S3"
      },
      "source": [
        "Here, each example returns a [logit](https://developers.google.com/machine-learning/crash-course/glossary#logits) for each class. \n",
        "\n",
        "To convert these logits to a probability for each class, use the [softmax](https://developers.google.com/machine-learning/crash-course/glossary#softmax) function:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "w9DHYDrvhIvK",
        "colab_type": "code",
        "outputId": "d1a83517-5aa0-488c-e3ba-b519cb6b0dd7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 340
        }
      },
      "source": [
        "model.layer1"
      ],
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "▿ Dense<Float>\n",
              "  - weight : [[  -0.14492011,   -0.37954563,   -0.16574049,     0.3085397,   0.014497697,   -0.28915244,\n",
              "    0.051759124,    0.20387083,    0.43077147,   -0.37560666,  0.0128573775,    0.14746654,\n",
              "    -0.46427745,     -0.347071,   -0.27823567,   -0.44490415,    0.21269625,    0.16966563,\n",
              "     -0.2250281,    0.08960563],\n",
              " [    0.4164847,   -0.42250705,    0.37574083,   -0.26941198,    0.12186271,    0.03589827,\n",
              "   -0.023880959,   -0.20080477,  -0.111144066,   -0.24596316,   -0.44131857,    0.25859576,\n",
              "   -0.041369915,     0.3730446,     0.3214333,  -0.009039223,   0.106165886,     0.4246788,\n",
              "    -0.08701587,   -0.38448274],\n",
              " [  -0.10086638,   -0.16406566,    0.36039305,    0.40359855,  -0.040762603,   -0.25001335,\n",
              "     -0.2762763,    -0.3644892,   -0.12599999,   -0.10050154,    -0.3062815,    0.44099373,\n",
              "     0.32428092,   -0.18351877,  -0.052918136,    0.49296975,  -0.100485384,   -0.18916929,\n",
              "      0.2956305,   -0.44767243],\n",
              " [   0.22463435,    0.27754343,    0.24517715,  -0.048776627, -0.0005801916,     0.4823004,\n",
              "     0.13744432,   -0.47332567,   -0.14139831,   -0.41945088,   -0.47918856,  -0.008722544,\n",
              "    -0.16316718,    0.15953654,   -0.28508234,   -0.16690987,   -0.18278545,    0.14870715,\n",
              "    -0.47366238,    0.08225161]]\n",
              "  - bias : [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]\n",
              "  - activation : (Function)\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 47
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p66IaSeQg9Qc",
        "colab_type": "code",
        "outputId": "d01ce89e-08c1-4cd6-8499-c9bb970036e4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 561
        }
      },
      "source": [
        "firstTrainPredictions"
      ],
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[[    -1.7329772,    -0.13151097,     0.19777158],\n",
              " [    -1.1974344,    -0.06770605,      0.1761316],\n",
              " [     -1.349941,    -0.14748853,     0.13317056],\n",
              " [    -1.0228678,    0.017132223,      0.1877974],\n",
              " [    -1.1936471,    0.013257623,     0.21275392],\n",
              " [    -0.9195043, -0.00034573674,      0.1541571],\n",
              " [     -1.129523,    0.028615057,     0.21262708],\n",
              " [    -1.7541509,    -0.10661441,     0.23848464],\n",
              " [     -1.606487,    -0.08647609,     0.23940669],\n",
              " [     -1.071094,  -0.0008304119,      0.1797471],\n",
              " [    -1.3189634,    -0.11956555,     0.16512263],\n",
              " [     -1.700394,    -0.10663462,     0.23425575],\n",
              " [    -1.1852164,    0.047620267,      0.2252725],\n",
              " [    -1.1387551,   -0.008177459,     0.18791792],\n",
              " [    -2.0848308,    -0.20082211,     0.22726771],\n",
              " [    -1.5921623,    -0.14633834,      0.1995019],\n",
              " [    -1.8481008,    -0.16160625,      0.2072714],\n",
              " [    -2.0289984,    -0.14850974,       0.228187],\n",
              " [    -1.7058232,     -0.1600421,     0.19565877],\n",
              " [    -1.1867101,   0.0008380711,     0.19526908],\n",
              " [    -1.8003541,     -0.1685304,     0.20181924],\n",
              " [    -1.7357035,    -0.13149834,     0.19857927],\n",
              " [    -1.1193354,    0.021390319,      0.1986916],\n",
              " [    -1.6586404,      -0.148471,      0.1686955],\n",
              " [     -1.855031,    -0.14251322,     0.21979311],\n",
              " [     -1.083974,    0.014972538,      0.1939413],\n",
              " [    -1.4097486,    -0.08591962,     0.19838478],\n",
              " [    -1.5944822,    -0.16179568,      0.1723036],\n",
              " [    -1.4306463,    -0.16595113,     0.15313365],\n",
              " [    -1.6969911,    -0.10479659,     0.22825569],\n",
              " [    -1.5250158,    -0.06648445,      0.2139497],\n",
              " [    -1.1757054,    -0.06192237,     0.18795238]]\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "_tRwHZmTNTX2",
        "outputId": "37b6a9e9-6d70-4759-8155-be77d771dfa3",
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "softmax(firstTrainPredictions[0..<5])"
      ],
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[[0.077790916,   0.3858663,   0.5363428],\n",
              " [ 0.12431279,  0.38472518,  0.49096212],\n",
              " [ 0.11448321,  0.38103098,  0.50448585],\n",
              " [ 0.13918012,  0.39377084,  0.46704903],\n",
              " [ 0.11870335,  0.39683965,    0.484457]]\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "uRZmchElo481"
      },
      "source": [
        "Taking the `argmax` across classes gives us the predicted class index. But, the model hasn't been trained yet, so these aren't good predictions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "-Jzm_GoErz8B",
        "outputId": "e0f84e82-6c69-43e2-80d8-d26431b6d653",
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "print(\"Prediction: \\(firstTrainPredictions.argmax(squeezingAxis: 1))\")\n",
        "print(\"    Labels: \\(firstTrainLabels)\")"
      ],
      "execution_count": 50,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Prediction: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]\r\n",
            "    Labels: [2, 1, 2, 0, 0, 0, 0, 2, 1, 0, 1, 1, 0, 0, 2, 1, 2, 2, 2, 0, 2, 2, 0, 2, 2, 0, 1, 2, 1, 1, 1, 1]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "RaKp8aEjKX6B"
      },
      "source": [
        " [`softmaxCrossEntropy(logits:labels:)`](https://www.tensorflow.org/swift/api_docs/Functions#/s:10TensorFlow19softmaxCrossEntropy6logits6labelsAA0A0VyxGAG_AFys5Int32VGtAA0aB13FloatingPointRzlF) function which takes the model's class probability predictions and the desired label, and returns the average loss across the examples."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-owA627Nit4Q",
        "colab_type": "code",
        "outputId": "8414bb9c-1faa-4fc1-8ccc-7ef7cbaa22af",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 266
        }
      },
      "source": [
        "TensorFlow.softmaxCrossEntropy"
      ],
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "error",
          "ename": "",
          "evalue": "ignored",
          "traceback": [
            "error: <Cell 51>:1:1: error: ambiguous reference to member 'softmaxCrossEntropy(logits:labels:)'\nTensorFlow.softmaxCrossEntropy\n^~~~~~~~~~\n\nTensorFlow.softmaxCrossEntropy:2:13: note: found this candidate\npublic func softmaxCrossEntropy<Scalar>(logits: Tensor<Scalar>, labels: Tensor<Int32>) -> Tensor<Scalar> where Scalar : TensorFlowFloatingPoint\n            ^\n\nTensorFlow.softmaxCrossEntropy:2:13: note: found this candidate\npublic func softmaxCrossEntropy<Scalar>(logits: Tensor<Scalar>, probabilities: Tensor<Scalar>) -> Tensor<Scalar> where Scalar : TensorFlowFloatingPoint\n            ^\n\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "tMAT4DcMPwI-",
        "outputId": "36f7e42d-1c5f-4c83-9a5f-b522627bf65c",
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "let untrainedLogits = model(firstTrainFeatures)\n",
        "let untrainedLoss = softmaxCrossEntropy(logits: untrainedLogits, labels: firstTrainLabels)\n",
        "print(\"Loss test: \\(untrainedLoss)\")"
      ],
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Loss test: 1.169491\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "lOxFimtlKruu"
      },
      "source": [
        "### Create an optimizer\n",
        "\n",
        "An *[optimizer](https://developers.google.com/machine-learning/crash-course/glossary#optimizer)* applies the computed gradients to the model's variables to minimize the `loss` function. You can think of the loss function as a curved surface (see Figure 3) and we want to find its lowest point by walking around. The gradients point in the direction of steepest ascent—so we'll travel the opposite way and move down the hill. By iteratively calculating the loss and gradient for each batch, we'll adjust the model during training. Gradually, the model will find the best combination of weights and bias to minimize loss. And the lower the loss, the better the model's predictions.\n",
        "\n",
        "<table>\n",
        "  <tr><td>\n",
        "    <img src=\"https://cs231n.github.io/assets/nn3/opt1.gif\" width=\"70%\"\n",
        "         alt=\"Optimization algorithms visualized over time in 3D space.\">\n",
        "  </td></tr>\n",
        "  <tr><td align=\"center\">\n",
        "    <b>Figure 3.</b> Optimization algorithms visualized over time in 3D space.<br/>(Source: <a href=\"http://cs231n.github.io/neural-networks-3/\">Stanford class CS231n</a>, MIT License, Image credit: <a href=\"https://twitter.com/alecrad\">Alec Radford</a>)\n",
        "  </td></tr>\n",
        "</table>\n",
        "\n",
        "Swift for TensorFlow has many [optimization algorithms](https://github.com/rxwei/DeepLearning/blob/master/Sources/DeepLearning/Optimizer.swift) available for training. This model uses the SGD optimizer that implements the *[stochastic gradient descent](https://developers.google.com/machine-learning/crash-course/glossary#gradient_descent)* (SGD) algorithm. The `learningRate` sets the step size to take for each iteration down the hill. This is a *hyperparameter* that you'll commonly adjust to achieve better results."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "8xxi2NNGKwG_",
        "scrolled": true,
        "colab": {}
      },
      "source": [
        "let optimizer = SGD(for: model, learningRate: 0.01)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eNYIAUcEkBl6",
        "colab_type": "text"
      },
      "source": [
        "Differentiate the loss with respect to the model's parameters"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "rxRNTFVe56RG",
        "outputId": "51f5f452-3dcd-491c-8eff-72695248e72e",
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "let (loss, grads) = model.valueWithGradient { model -> Tensor<Float> in\n",
        "    let logits = model(firstTrainFeatures)\n",
        "    return softmaxCrossEntropy(logits: logits, labels: firstTrainLabels)\n",
        "}\n",
        "print(\"Current loss: \\(loss)\")"
      ],
      "execution_count": 54,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Current loss: 1.169491\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "5B27cIT0O0nE"
      },
      "source": [
        "Next, we pass the gradient that we just calculated to the optimizer, which updates the model's differentiable variables accordingly:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "icyvh-o6O0nF",
        "scrolled": true,
        "colab": {}
      },
      "source": [
        "optimizer.update(&model.allDifferentiableVariables, along: grads)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "nhpgM7UpO0nG"
      },
      "source": [
        "If we calculate the loss again, it should be smaller, because gradient descent steps (usually) decrease the loss:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "aw0OzyojAa39",
        "outputId": "0a1450c0-edb4-4510-8692-3045a0860aa1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "let logitsAfterOneStep = model(firstTrainFeatures)\n",
        "let lossAfterOneStep = softmaxCrossEntropy(logits: logitsAfterOneStep, labels: firstTrainLabels)\n",
        "print(\"Next loss: \\(lossAfterOneStep)\")"
      ],
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Next loss: 1.1405698\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "7Y2VSELvwAvW"
      },
      "source": [
        "### Training loop\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "AIgulGRUhpto",
        "scrolled": true,
        "colab": {}
      },
      "source": [
        "let epochCount = 500\n",
        "var trainAccuracyResults: [Float] = []\n",
        "var trainLossResults: [Float] = []"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "066kVZQFO0nL",
        "outputId": "b3f8b075-008a-4dbd-c980-6b5375bf5a46",
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        }
      },
      "source": [
        "func accuracy(predictions: Tensor<Int32>, truths: Tensor<Int32>) -> Float {\n",
        "    return Tensor<Float>(predictions .== truths).mean().scalarized()\n",
        "}\n",
        "\n",
        "for epoch in 1...epochCount {\n",
        "    var epochLoss: Float = 0\n",
        "    var epochAccuracy: Float = 0\n",
        "    var batchCount: Int = 0\n",
        "    for batch in trainDataset {\n",
        "        let (loss, grad) = model.valueWithGradient { (model: IrisModel) -> Tensor<Float> in\n",
        "            let logits = model(batch.features)\n",
        "            return softmaxCrossEntropy(logits: logits, labels: batch.labels)\n",
        "        }\n",
        "        optimizer.update(&model.allDifferentiableVariables, along: grad)\n",
        "        \n",
        "        let logits = model(batch.features)\n",
        "        epochAccuracy += accuracy(predictions: logits.argmax(squeezingAxis: 1), truths: batch.labels)\n",
        "        epochLoss += loss.scalarized()\n",
        "        batchCount += 1\n",
        "    }\n",
        "    epochAccuracy /= Float(batchCount)\n",
        "    epochLoss /= Float(batchCount)\n",
        "    trainAccuracyResults.append(epochAccuracy)\n",
        "    trainLossResults.append(epochLoss)\n",
        "    if epoch % 50 == 0 {\n",
        "        print(\"Epoch \\(epoch): Loss: \\(epochLoss), Accuracy: \\(epochAccuracy)\")\n",
        "    }\n",
        "}"
      ],
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 50: Loss: 0.5034555, Accuracy: 0.6979167\n",
            "Epoch 100: Loss: 0.36751425, Accuracy: 0.9114583\n",
            "Epoch 150: Loss: 0.26342344, Accuracy: 0.984375\n",
            "Epoch 200: Loss: 0.19525252, Accuracy: 0.984375\n",
            "Epoch 250: Loss: 0.1575833, Accuracy: 0.984375\n",
            "Epoch 300: Loss: 0.13529453, Accuracy: 0.984375\n",
            "Epoch 350: Loss: 0.12082361, Accuracy: 0.984375\n",
            "Epoch 400: Loss: 0.110710636, Accuracy: 0.9921875\n",
            "Epoch 450: Loss: 0.103334695, Accuracy: 0.9921875\n",
            "Epoch 500: Loss: 0.09789996, Accuracy: 0.9921875\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "j3wdbmtLVTyr"
      },
      "source": [
        "see the *loss* go down and the *accuracy* go up."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "agjvNd2iUGFn",
        "outputId": "cfb27b64-4de2-4e7d-fd76-ff7d541fd90e",
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 517
        }
      },
      "source": [
        "plt.figure(figsize: [12, 8])\n",
        "\n",
        "let accuracyAxes = plt.subplot(2, 1, 1)\n",
        "accuracyAxes.set_ylabel(\"Accuracy\")\n",
        "accuracyAxes.plot(trainAccuracyResults)\n",
        "\n",
        "let lossAxes = plt.subplot(2, 1, 2)\n",
        "lossAxes.set_ylabel(\"Loss\")\n",
        "lossAxes.set_xlabel(\"Epoch\")\n",
        "lossAxes.plot(trainLossResults)\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtQAAAHjCAYAAAADuoh4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xl8XXWd//H3J3vSrG3SNelGA7QF\nWiBA2RQBsYBS1BkEBwF/OvycEdFxdH7ozG9UHEedRX86MigyCC7DIg5QFQcZ9p2mtAXa0jZdk25J\nmqVJs97k8/vj3pZLaJubJicnuff1fDzu495z7jf3vul5EN6cfs/3mLsLAAAAwLFJCzsAAAAAMJ5R\nqAEAAIBhoFADAAAAw0ChBgAAAIaBQg0AAAAMA4UaAAAAGAYKNQAAADAMFGoAAABgGCjUAAAAwDBk\nhB1gqEpLS3327NlhxwAAAECSW7lyZaO7lw02btwV6tmzZ6u6ujrsGAAAAEhyZrY9kXFM+QAAAACG\nIbBCbWZ3mVm9mb15hPfNzH5oZjVm9rqZnRZUFgAAACAoQZ6hvlvS0qO8f6mkytjjRkm3B5gFAAAA\nCERghdrdn5XUdJQhyyT93KNellRsZtOCygMAAAAEIcw51DMk1cZt18X2vYuZ3Whm1WZW3dDQMCrh\nAAAAgESMi4sS3f0Od69y96qyskFXLgEAAABGTZjL5u2UVBG3XR7bBwAAxpFv/2G9Hl+3N+wYSFJp\nZvqfL7437BhHFWahXi7pJjO7T9JZklrdfXeIeQAAwBDtaunUnc9t1cLphZo5MS/sOEhCZhZ2hEEF\nVqjN7F5JF0gqNbM6SV+TlClJ7v5jSY9KukxSjaQOSZ8MKgsAAAjGr17ZLnfXbR8/TRUUaqSowAq1\nu18zyPsu6bNBfT+A8ee6u17Vq1v3hR0DwBB0R/p18fwplGmktHF363EAyWlr4wE9u7FBF544WZWT\n88OOAyBBZqaPnVEx+EAgiVGoAYwJj63dI0m6ddlClZdwpgsAMH5QqIEEHeiOqKWzN+wYSesPb+7R\nyTOKKNMAgHGHQg0koKu3Txf96zPas78r7ChJ7csfOCHsCAAADBmFGkjAo2/s1p79Xfr8RZWaUZwb\ndpyklJFu+sDCqWHHAABgyCjUSHpv7mxVZ2/fsD7jrhe26riyCfrCxZXjYj1MAAAweijUSGrPbGzQ\n9Xe9OiKf9c1lCynTAADgXSjUSGo/e2Grygqy9b2rFsl07GU4M91UNXviCCYDAADJgkKNpPPmzlZt\nbmjXge4+Pb2hQV+4uFLnV5aFHQsAACQpCjWSSltXr66+42W1d0ckSTmZafr4mTNDTgUAAJIZhRpJ\n5b9e26n27ojuvK5Kc8smqCg3U5Pys8OOBQAAkhiFGkmhvTui+17doXte2qZFFcW6eMGUsCMBAIAU\nQaFGUrjr+a363uMbZSZ95dL5YccBAAAphEKNca+3r1+/emW7zq8s1X9cf4ayMtLCjgQAAFIIhRrj\n2mNr9+jhVTu1d3+3/vHDJ1OmAQDAqKNQY9zq6Inoy79eo94+15K5E3XBCZPDjgQAAFIQhRrj1sOr\ndml/V0QPfuZsbroCAABCQ6HGuHDbUzVaub35Hfter2vVgmmFOn1WSUipAAAAKNQYB7bvO6B/+eMG\nlZfkqjg369D+GcU5+sLFx8vs2G8pDgAAMFwUaox5v3hpu9LN9OBnztGUwpyw4wAAALwDSyJgTOvo\nieiB6lotPWkqZRoAAIxJFGqMaY+sjl54eP05s8OOAgAAcFgUaoxZ7q57Xtym+dMKVcWFhwAAYIxi\nDjVC9aMnN+knz245/JsutXVH9N2PnsyFhwAAYMyiUCM0B7oj+skzWzS7dIKqZh/+DHRBdoauPHXG\nKCcDAABIHIUagenoiaiv34/4/oMr69TWHdHXr1jIWtIAAGDcolAjEL97fZdu+s9Vg447aUahTptZ\nPAqJAAAAgkGhxohzd93x7BbNmpSnTyyZddSxF5xQxvxoAAAwrgVaqM1sqaQfSEqXdKe7f2fA+7Mk\n3SWpTFKTpGvdvS7ITHhbpK9fWxoPyI88K+OYbG1s1+t1rfrmsoX6xNmzR/bDAQAAxpjACrWZpUu6\nTdL7JdVJWmFmy919Xdywf5H0c3e/x8wulPRtSZ8IKhPe6V/+uFE/fmZzIJ9dkJ2hD59WHshnAwAA\njCVBnqE+U1KNu2+RJDO7T9IySfGFeoGkL8ZePyXp4QDzYIBXtu7T/GmF+tyF80b8s+eWTVB+NjOK\nAABA8guy8cyQVBu3XSfprAFj1kj6iKLTQj4sqcDMJrn7vvhBZnajpBslaebMmYEFTiU9kX6t3bVf\nN5wzW5edPC3sOAAAAONW2HdK/JKk95rZKknvlbRTUt/AQe5+h7tXuXtVWVnZaGdMShv2tKkn0q9F\n5aywAQAAMBxBnqHeKakibrs8tu8Qd9+l6BlqmVm+pI+6e0uAmRCzui76x7yooijkJAAAAONbkGeo\nV0iqNLM5ZpYl6WpJy+MHmFmpmR3M8BVFV/xAwJ7d2KDfrdml0vwszSjODTsOAADAuBZYoXb3iKSb\nJD0mab2kB9x9rZndamZXxIZdIGmDmW2UNEXSt4LKg6jWzl7d8LNX9crWJp07r5Q1oAEAAIYp0GUY\n3P1RSY8O2Pf3ca8flPRgkBnwTq/XtajfpX//s9P0gYVTw44DAAAw7oV9USJG2Zra6Nzpc+eVKj2N\ns9MAAADDRaFOMatrWzW3bIKKcjPDjgIAAJAUKNQpxN21urZFi1kqDwAAYMRQqFPI7tYuNbZ3a1EF\nhRoAAGCkUKhTyMH504sp1AAAACOGQp1CVte1KCs9TSdOKwg7CgAAQNKgUKeQNbUtmj+9UNkZ6WFH\nAQAASBoU6hTR1+96o65Vi8u51TgAAMBIolCniJr6dh3o6eOCRAAAgBEW6J0SMTb8+JnNuvO5LZK4\nIBEAAGCkUaiTXHt3RD96skblJbm65syZmlM6IexIAAAASYVCneT+67U6tXdH9O2PnKxTZ5aEHQcA\nACDpUKiT1P6uXl38r8+ovq1bi8qLKNMAAAABoVAnqVU7WlTf1q2rz6jQdWfPDjsOAABA0qJQJ6k1\ntS0yk/728vkqyMkMOw4AAEDSYtm8JLWmtkXzyvIp0wAAAAGjUCchd9fq2hbWnAYAABgFTPlIMl29\nfXp+U6P2HehhzWkAAIBRQKFOMt/83Tr96pUdkqTTZ7GyBwAAQNAo1EmktaNXv3mtTksXTtVnLjhO\n86cVhh0JAAAg6VGox7GOnoie3tCgSL9Lkl7Zsk9dvf363EXztHB6UcjpAAAAUgOFehz70ZM1+ven\nN79j35K5EynTAAAAo4hCPU519fbp3ld36MITJ+url80/tH9GcW6IqQAAAFIPhXocen5To5av2anm\njl59+vw5mjc5P+xIAAAAKYtCPc60dvbqz39erc7ePi0qL9LZcyeFHQkAACClUajHmV9X16qzt08P\n/eU5WlReLDMLOxIAAEBKo1CPI/39rl+8vF1Vs0p06kzWmAYAABgLuPX4OLKpvl3b93XoqqqKsKMA\nAAAgJtBCbWZLzWyDmdWY2S2HeX+mmT1lZqvM7HUzuyzIPOPdmtoWSdLpszk7DQAAMFYMWqjN7HNm\nNuQGZ2bpkm6TdKmkBZKuMbMFA4b9naQH3P1USVdL+vehfk8qWV3XooKcDM2ZNCHsKAAAAIhJ5Az1\nFEkrzOyB2BnnRK+CO1NSjbtvcfceSfdJWjZgjEs6eH/sIkm7EvzslLSmtkWLyouVlsaFiAAAAGPF\noIXa3f9OUqWk/5B0g6RNZvaPZnbcID86Q1Jt3HZdbF+8r0u61szqJD0q6XOH+yAzu9HMqs2suqGh\nYbDISamrt09v7WnTogrugggAADCWJLTKh7u7me2RtEdSRFKJpAfN7HF3/5thfP81ku529381s7Ml\n/cLMTnL3/gHff4ekOySpqqrKh/F9485tT9Xod6/vVk+kT339rkXlxWFHAgAAQJxBC7WZfV7SdZIa\nJd0p6cvu3mtmaZI2STpSod4pKX45ivLYvnifkrRUktz9JTPLkVQqqX4o/xDJal97t37wxCbNmTRB\nc8vytai8WOfOKw07FgAAAOIkcoZ6oqSPuPv2+J3u3m9mHzzKz62QVGlmcxQt0ldL+viAMTskXSTp\nbjObLylHUmrO6TiM+6tr1RPp148+fqoqpxSEHQcAAACHkUih/oOkpoMbZlYoab67v+Lu64/0Q+4e\nMbObJD0mKV3SXe6+1sxulVTt7ssl/bWkn5rZXyl6geIN7p5SUzoOZ1dLp676yUva09qlc46bRJkG\nAAAYwxIp1LdLOi1uu/0w+w7L3R9V9GLD+H1/H/d6naRzE0qaQu55aZt2t3bpE0tm6WNncBMXAACA\nsSyRQm3xZ41jUz24ZfkI6+93dUX61BPp1/0ranXJgin6+hULw44FAACAQSRSjLeY2c2KnpWWpL+U\ntCW4SKnp6jte1qvbDs2s0XVnzw4vDAAAABKWSKH+jKQfKnpXQ5f0hKQbgwyVanbs69Cr25p0+SnT\ndMqMIpXmZ2vJ3IlhxwIAAEACBi3U7l6v6AodCMhja/dIkm5ZeqIqJuaFnAYAAABDkcg61DmKrhe9\nUNFl7SRJ7v6/AsyVUv577R4tmFZImQYAABiHBr31uKRfSJoq6QOSnlH0Bi1tQYZKJXXNHXptR7OW\nnjQ17CgAAAA4BokU6nnu/n8lHXD3eyRdLumsYGOljl++vEMm6U9OLw87CgAAAI5BIhcl9saeW8zs\nJEl7JE0OLlLy29XSqdfrWiW57l+xQ5csmKrpxblhxwIAAMAxSKRQ32FmJYqu8rFcUr6k/xtoqiT3\nuXtXaeX25kPbnzx3dnhhAAAAMCxHLdRmliZpv7s3S3pW0txRSZXE9rR2aeX2Zn36vDn6yGnlystK\n1+zSCWHHAgAAwDE66hxqd++X9DejlCUlPL4uukTe1WdWaMH0Qso0AADAOJfIlI//MbMvSbpf0oGD\nO9296cg/kpr2tHbpoVU71f/2ndrf5bdrdum4sgmaN7lgFJMBAAAgKIkU6o/Fnj8bt8/F9I93+c4f\n1uvh1bsGHXfLpSeOQhoAAACMhkTulDhnNIKMd/VtXfr9G7t1/dmz9NXL5x9xnMmUlZHIaoUAAAAY\nDxK5U+J1h9vv7j8f+Tjj132v1qq3z3X9ObOVnZEedhwAAACMkkSmfJwR9zpH0kWSXpNEoY7z6tYm\nnTyjSHPL8sOOAgAAgFGUyJSPz8Vvm1mxpPsCSzRONbZ3q7wkL+wYAAAAGGXHMpn3gCTmVQ+w70CP\nSvOzwo4BAACAUZbIHOrfKrqqhxQt4AskPRBkqPGmv9/VdKBHkyjUAAAAKSeROdT/Evc6Imm7u9cF\nlGdc2t/Vq75+16QJ2WFHAQAAwChLpFDvkLTb3bskycxyzWy2u28LNNk40tjeI0mcoQYAAEhBicyh\n/rWk/rjtvtg+xOxr75YkzlADAACkoEQKdYa79xzciL3mVGycpgOcoQYAAEhViRTqBjO74uCGmS2T\n1BhcpPGn8WChnkChBgAASDWJzKH+jKRfmdmPYtt1kg5798RUdXDKRwmFGgAAIOUkcmOXzZKWmFl+\nbLs98FTjTNOBHhXnZSoz/ViW9QYAAMB4NmgDNLN/NLNid29393YzKzGzfxiNcOPFvvYeTeTsNAAA\nQEpK5JTqpe7ecnDD3ZslXRZcpPGnsb1bpazwAQAAkJISKdTpZnaoLZpZrqSE2qOZLTWzDWZWY2a3\nHOb975vZ6thjo5m1HO5zxjrukggAAJC6Erko8VeSnjCzn0kySTdIumewHzKzdEm3SXq/ohcyrjCz\n5e6+7uAYd/+ruPGfk3TqkNKPAV29fdrV0qmzj5sUdhQAAACEIJGLEr9rZmskXSzJJT0maVYCn32m\npBp33yJJZnafpGWS1h1h/DWSvpZI6LFk+ZpdOtDTp0tPmhZ2FAAAAIQg0WUp9ipapv9U0oWS1ifw\nMzMk1cZt18X2vYuZzZI0R9KTR3j/RjOrNrPqhoaGBCMHz911z4vbdPyUfC2ZOzHsOAAAAAjBEQu1\nmR1vZl8zs7ck/ZukHZLM3d/n7j860s8do6slPejufYd7093vcPcqd68qKysb4a8+dq/taNHaXft1\n3dmzZWZhxwEAAEAIjjbl4y1Jz0n6oLvXSJKZ/dVRxg+0U1JF3HZ5bN/hXC3ps0P47DHhnhe3qSAn\nQx8+9bAn3gEAAJACjjbl4yOSdkt6ysx+amYXKXpRYqJWSKo0szlmlqVoaV4+cJCZnSipRNJLQ/js\n0NXv79Kjb+zWn55eoQnZiVzbCQAAgGR0xELt7g+7+9WSTpT0lKQvSJpsZreb2SWDfbC7RyTdpOhF\njOslPeDua83sVjO7Im7o1ZLuc3cfzj/IaLv31VpF+l2fODuR6zMBAACQrGwoPdbMShS9MPFj7n5R\nYKmOoqqqyqurq8P46kN6Iv0677tPasH0Qt39yTNDzQIAAIBgmNlKd68abFyiq3xIit4lMXaBYChl\neqx4bO0e1bd16/qzZ4cdBQAAACEbUqFG1IubG1Wcl6n3Hj92VhwBAABAOCjUx6D5QK/K8rOVlsZS\neQAAAKmOQn0MWjp7VJyXGXYMAAAAjAEU6mPQ0tGr4ryssGMAAABgDKBQH4PWzl4V53KGGgAAABTq\nYxI9Q02hBgAAAIV6yLp6+9TZ28eUDwAAAEiiUA/Z/s5eSVIRUz4AAAAgCvWQtcQKNVM+AAAAIFGo\nh6ylI1aoc5nyAQAAAAr1kLV09EjiDDUAAACiKNRD1MIcagAAAMShUA9RawdzqAEAAPA2CvUQtXT2\nKD3NlJ+dEXYUAAAAjAEU6iFq6YjeJdHMwo4CAACAMYBCPUQtnb0qYroHAAAAYijUQ9QaO0MNAAAA\nSBTqIWvt7GWFDwAAABxCoR6i7kifcjLTw44BAACAMYJCPUQ9kX5lZfDHBgAAgCia4RD1RPqVlc4f\nGwAAAKJohkPU08cZagAAALyNZjhE3Uz5AAAAQBya4RBRqAEAABCPZjgE7q6eSL+ymUMNAACAGJrh\nEPT2uSRxhhoAAACHBNoMzWypmW0wsxozu+UIY64ys3VmttbM/jPIPMPV09cviUINAACAt2UE9cFm\nli7pNknvl1QnaYWZLXf3dXFjKiV9RdK57t5sZpODyjMSeiKxQs2UDwAAAMQE2QzPlFTj7lvcvUfS\nfZKWDRjz55Juc/dmSXL3+gDzDNuhQp3BnRIBAAAQFWShniGpNm67LrYv3vGSjjezF8zsZTNbGmCe\nYXu7UHOGGgAAAFGBTfkYwvdXSrpAUrmkZ83sZHdviR9kZjdKulGSZs6cOdoZD+np65NEoQYAAMDb\ngmyGOyVVxG2Xx/bFq5O03N173X2rpI2KFux3cPc73L3K3avKysoCCzyYbuZQAwAAYIAgm+EKSZVm\nNsfMsiRdLWn5gDEPK3p2WmZWqugUkC0BZhqWg1M+sjMp1AAAAIgKrBm6e0TSTZIek7Re0gPuvtbM\nbjWzK2LDHpO0z8zWSXpK0pfdfV9QmYbrUKHmDDUAAABiAp1D7e6PSnp0wL6/j3vtkr4Ye4x5rEMN\nAACAgWiGQ8AqHwAAABiIZjgE3RRqAAAADEAzHALulAgAAICBaIZDwJQPAAAADEQzHIJuLkoEAADA\nADTDIXh72bz0kJMAAABgrKBQDwFTPgAAADAQzXAIKNQAAAAYiGY4BD19fUpPM6WnWdhRAAAAMEZQ\nqIegJ9LPknkAAAB4B9rhEPRE+pnuAQAAgHegHQ5BTx+FGgAAAO9EOxyCbqZ8AAAAYADa4RD0RPqV\nnckfGQAAAN5GOxwCzlADAABgINrhEPRE+pXNHGoAAADEoR0OAat8AAAAYCDa4RCwygcAAAAGoh0O\nATd2AQAAwEC0wyFgygcAAAAGoh0OQXTKR3rYMQAAADCGUKiHgCkfAAAAGIh2OATdTPkAAADAALTD\nIeiJ9LEONQAAAN6BdjgELJsHAACAgWiHCXJ35lADAADgXWiHCWrvjqjfpZxM/sgAAADwNtphgh5e\ntVOSdM680pCTAAAAYCwJtFCb2VIz22BmNWZ2y2Hev8HMGsxsdezx6SDzHCt31z0vbdcp5UU6taI4\n7DgAAAAYQzKC+mAzS5d0m6T3S6qTtMLMlrv7ugFD73f3m4LKMRJe3LxPNfXt+pc/XSQzCzsOAAAA\nxpAgz1CfKanG3be4e4+k+yQtC/D7ArOoolj/cOVJ+uAp08KOAgAAgDEmyEI9Q1Jt3HZdbN9AHzWz\n183sQTOrCDDPMcvPztC1S2YpJ5PbjgMAAOCdwr4o8beSZrv7KZIel3TP4QaZ2Y1mVm1m1Q0NDaMa\nEAAAADiaIAv1TknxZ5zLY/sOcfd97t4d27xT0umH+yB3v8Pdq9y9qqysLJCwAAAAwLEIslCvkFRp\nZnPMLEvS1ZKWxw8ws/hJyVdIWh9gHgAAAGDEBbbKh7tHzOwmSY9JSpd0l7uvNbNbJVW7+3JJN5vZ\nFZIikpok3RBUHgAAACAI5u5hZxiSqqoqr66uDjsGAAAAkpyZrXT3qsHGhX1RIgAAADCuUagBAACA\nYRh3Uz7MrEHS9pC+vlRSY0jfjdHDcU5+HOPUwHFODRzn1BDWcZ7l7oMuMTfuCnWYzKw6kXk0GN84\nzsmPY5waOM6pgeOcGsb6cWbKBwAAADAMFGoAAABgGCjUQ3NH2AEwKjjOyY9jnBo4zqmB45waxvRx\nZg41AAAAMAycoQYAAACGgUKdADNbamYbzKzGzG4JOw+OnZndZWb1ZvZm3L6JZva4mW2KPZfE9puZ\n/TB23F83s9PCS46hMLMKM3vKzNaZ2Voz+3xsP8c6iZhZjpm9amZrYsf5G7H9c8zsldjxvN/MsmL7\ns2PbNbH3Z4eZH0NjZulmtsrMfhfb5jgnGTPbZmZvmNlqM6uO7RsXv7cp1IMws3RJt0m6VNICSdeY\n2YJwU2EY7pa0dMC+WyQ94e6Vkp6IbUvRY14Ze9wo6fZRyojhi0j6a3dfIGmJpM/G/r3lWCeXbkkX\nuvsiSYslLTWzJZK+K+n77j5PUrOkT8XGf0pSc2z/92PjMH58XtL6uG2Oc3J6n7svjlsib1z83qZQ\nD+5MSTXuvsXdeyTdJ2lZyJlwjNz9WUlNA3Yvk3RP7PU9kq6M2/9zj3pZUrGZTRudpBgOd9/t7q/F\nXrcp+h/hGeJYJ5XY8WqPbWbGHi7pQkkPxvYPPM4Hj/+Dki4yMxuluBgGMyuXdLmkO2PbJo5zqhgX\nv7cp1IObIak2brsutg/JY4q774693iNpSuw1xz4JxP6691RJr4hjnXRi0wBWS6qX9LikzZJa3D0S\nGxJ/LA8d59j7rZImjW5iHKP/J+lvJPXHtieJ45yMXNIfzWylmd0Y2zcufm9nhPXFwFjk7m5mLH2T\nJMwsX9JvJH3B3ffHn6TiWCcHd++TtNjMiiU9JOnEkCNhhJnZByXVu/tKM7sg7DwI1HnuvtPMJkt6\n3Mzein9zLP/e5gz14HZKqojbLo/tQ/LYe/CviWLP9bH9HPtxzMwyFS3Tv3L3/4rt5lgnKXdvkfSU\npLMV/avfgyeM4o/loeMce79I0r5RjoqhO1fSFWa2TdFplxdK+oE4zknH3XfGnusV/R/kMzVOfm9T\nqAe3QlJl7GriLElXS1oeciaMrOWSro+9vl7SI3H7r4tdSbxEUmvcXzthDIvNl/wPSevd/Xtxb3Gs\nk4iZlcXOTMvMciW9X9H58k9J+pPYsIHH+eDx/xNJTzo3Yxjz3P0r7l7u7rMV/W/wk+7+Z+I4JxUz\nm2BmBQdfS7pE0psaJ7+3ubFLAszsMkXnb6VLusvdvxVyJBwjM7tX0gWSSiXtlfQ1SQ9LekDSTEnb\nJV3l7k2xUvYjRVcF6ZD0SXevDiM3hsbMzpP0nKQ39Pacy68qOo+aY50kzOwURS9SSlf0BNED7n6r\nmc1V9EzmREmrJF3r7t1mliPpF4rOqW+SdLW7bwknPY5FbMrHl9z9gxzn5BI7ng/FNjMk/ae7f8vM\nJmkc/N6mUAMAAADDwJQPAAAAYBgo1AAAAMAwUKgBAACAYaBQAwAAAMNAoQYAAACGgUINAAAADAOF\nGgAAABgGCjUAAAAwDBRqAAAAYBgo1AAAAMAwUKgBAACAYaBQAwAAAMNAoQYAAACGgUINAAAADAOF\nGgAAABgGCjUAAAAwDBRqAAAAYBgo1AAAAMAwUKgBAACAYaBQAwAAAMNAoQYAAACGgUINAAAADAOF\nGgAAABiGjLADDFVpaanPnj077BgAAABIcitXrmx097LBxo27Qj179mxVV1eHHQMAAABJzsy2JzKO\nKR8AAADAMFCoAQAAgGGgUAMAAADDQKEGAAAAhoFCDQAAAAwDhRoAAAAYBgp1AjbsadPN967S1sYD\nYUcBAADAGEOhTkCkv1/L1+zSul37w44CAACAMYZCnYDjyvJlJm2qbws7CgAAAMYYCnUCcjLTVVGS\np5r69rCjAAAAYIyhUCeocnI+hRoAAADvQqFO0Lwp+drScECRvv6wowAAAGAMoVAnqHJygXr6+rWj\nqSPsKAAAABhDKNQJmjc5X5K0iWkfAAAAiEOhTtDBQs08agAAAMSjUCcoPztD04tyKNQAAAB4Bwr1\nEMybUsBa1AAAAHgHCvUQHFw6r7/fw44CAACAMYJCPQSVk/PV1duvnS2dYUcBAADAGEGhHoK3V/pg\n2gcAAACiKNRDwEofAAAAGIhCPQTFeVkqK8jWxr0UagAAAERRqIdo/rRCvbmzNewYAAAAGCMo1EN0\nakWxNu5tU3t3JOwoAAAAGAMo1EN02qwS9bu0prYl7CgAAAAYAyjUQ7S4oliS9Nr25pCTAAAAYCyg\nUA9RUW6mKifn67UdFGoAAAAEWKjN7C4zqzezN4/wvpnZD82sxsxeN7PTgsoy0k6fVaKV25vVxx0T\nAQAAUl6QZ6jvlrT0KO9fKqky9rhR0u0BZhlRZx83Sfu7InqD1T4AAABSXmCF2t2fldR0lCHLJP3c\no16WVGxm04LKM5LOm1cqSXp+U0PISQAAABC2MOdQz5BUG7ddF9v3LmZ2o5lVm1l1Q0P4JXZSfrZO\nmlGoZzc1hh0FAAAAIRsXFyUS48VwAAAgAElEQVS6+x3uXuXuVWVlZWHHkSSdX1mm17Y3sx41AABA\niguzUO+UVBG3XR7bNy6cP69UkX7Xy5v3hR0FAAAAIQqzUC+XdF1stY8lklrdfXeIeYbk9NklyslM\n0/M1TPsAAABIZRlBfbCZ3SvpAkmlZlYn6WuSMiXJ3X8s6VFJl0mqkdQh6ZNBZQlCdka6lsydpGe5\nMBEAACClBVao3f2aQd53SZ8N6vtHw/mVZfrm79aprrlD5SV5YccBAABACMbFRYlj1Xsqo8vnPbuR\naR8AAACpikI9DPMm52tGca6efKs+7CgAAAAICYV6GMxMF8+frOdrGtTV2xd2HAAAAISAQj1MF86f\noq7efr3E8nkAAAApiUI9TEvmTlReVrqeeGtv2FEAAAAQAgr1MGVnpOv8ylI9ub5e0YVLAAAAkEoo\n1CPgovlTtKu1S+t3t4UdBQAAAKOMQj0C3nfCZJlJT6xn2gcAAECqoVCPgLKCbC0qL9b/sHweAABA\nyqFQj5BLFk7RmtoW7W7tDDsKAAAARhGFeoQsXThVkvTYm3tCTgIAAIDRRKEeIXPL8nX8lHz991oK\nNQAAQCqhUI+gpQun6tWtTdrX3h12FAAAAIwSCvUI+sBJU9Xv0uPrWO0DAAAgVVCoR9CCaYWqmJjL\ntA8AAIAUQqEeQWampQun6oWaRrV29oYdBwAAAKOAQj3Clp40Tb19zrQPAACAFEGhHmGnzSxWeUmu\nlq/ZFXYUAAAAjAIK9QgzM31o0XS9UNOoRlb7AAAASHoU6gAsWzxdff2uR9/YHXYUAAAABIxCHYAT\npxbq+Cn5Wr6aaR8AAADJjkIdkGWLZ6h6e7PqmjvCjgIAAIAAUagD8qFTpkuSfruGaR8AAADJjEId\nkJmT8nTqzGI9snpn2FEAAAAQIAp1gK5YNF1v7WnTxr1tYUcBAABAQAIt1Ga21Mw2mFmNmd1ymPdn\nmtlTZrbKzF43s8uCzDPaLj9lmtJMXJwIAACQxAIr1GaWLuk2SZdKWiDpGjNbMGDY30l6wN1PlXS1\npH8PKk8YJhfk6JzjSrV8zS65e9hxAAAAEIAgz1CfKanG3be4e4+k+yQtGzDGJRXGXhdJSrpTuVcs\nnq4dTR1aXdsSdhQAAAAEIMhCPUNSbdx2XWxfvK9LutbM6iQ9Kulzh/sgM7vRzKrNrLqhoSGIrIH5\nwMKpykpP41bkAAAASSrsixKvkXS3u5dLukzSL8zsXZnc/Q53r3L3qrKyslEPORxFuZl634ll+u2a\n3errZ9oHAABAsgmyUO+UVBG3XR7bF+9Tkh6QJHd/SVKOpNIAM4XiysUz1Njerec2ja+z6wAAABhc\nkIV6haRKM5tjZlmKXnS4fMCYHZIukiQzm69ooU661nnh/MkqycvUA9W1gw8GAADAuBJYoXb3iKSb\nJD0mab2iq3msNbNbzeyK2LC/lvTnZrZG0r2SbvAkXA4jOyNdHz61XI+v26umAz1hxwEAAMAIygjy\nw939UUUvNozf9/dxr9dJOjfIDGPFVWeU664XtuqhVTv1qfPmhB0HAAAAIyTsixJTxolTC7WovEgP\nrKhlTWoAAIAkQqEeRVedUaENe9v0el1r2FEAAAAwQijUo+hDi6YrJzNN93NxIgAAQNKgUI+iwpxM\nXXbSNP129S519vSFHQcAAAAjgEI9yq46o0Jt3RH94c3dYUcBAADACKBQj7Kz5kzUrEl5um8F0z4A\nAACSAYV6lJmZrjlzpl7d2qSNe9vCjgMAAIBholCH4KqqCmVlpOkXL20POwoAAACGiUIdgokTsvTB\nU6bpv16rU3t3JOw4AAAAGAYKdUg+sWSWDvT06aFVO8OOAgAAgGGgUIdkcUWxTppRqF++tJ07JwIA\nAIxjFOqQmJmuWzJbG/a26ZWtTWHHAQAAwDGiUIfoQ4umqyg3k4sTAQAAxjEKdYhys9L1sTMq9N9r\n92hPa1fYcQAAAHAMKNQh+8SSWXJ3/ezFrWFHAQAAwDGgUIesYmKePnjKdP3ype1qPtATdhwAAAAM\nEYV6DLjpwnk60NOnu17gLDUAAMB4Q6EeA46fUqBLT5qqu1/YptbO3rDjAAAAYAgo1GPETRfOU1t3\nRHe/sC3sKAAAABiChAq1mR1nZtmx1xeY2c1mVhxstNSycHqRLp4/RXe9sFVtXZylBgAAGC8SPUP9\nG0l9ZjZP0h2SKiT9Z2CpUtTNF81Ta2evfs661AAAAONGooW6390jkj4s6d/c/cuSpgUXKzWdUl6s\nC04o053PbVF7dyTsOAAAAEhAooW618yukXS9pN/F9mUGEym1feHi49Xc0aufPrsl7CgAAABIQKKF\n+pOSzpb0LXffamZzJP0iuFipa3FFsS47eap++twW1bdx90QAAICxLqFC7e7r3P1md7/XzEokFbj7\ndwPOlrK+/IET1RPp1w/+Z1PYUQAAADCIRFf5eNrMCs1soqTXJP3UzL4XbLTUNad0gj5+1kzdt6JW\nmxvaw44DAACAo0h0ykeRu++X9BFJP3f3syRdPNgPmdlSM9tgZjVmdssRxlxlZuvMbK2ZsXJIzM0X\nVSo3M13f+v36sKMAAADgKBIt1BlmNk3SVXr7osSjMrN0SbdJulTSAknXmNmCAWMqJX1F0rnuvlDS\nFxINnuxK87N180Xz9ORb9Xpi/d6w4wAAAOAIEi3Ut0p6TNJmd19hZnMlDTbB90xJNe6+xd17JN0n\nadmAMX8u6TZ3b5Ykd69PPHry++S5czRvcr6+8dt16urtCzsOAAAADiPRixJ/7e6nuPtfxLa3uPtH\nB/mxGZJq47brYvviHS/peDN7wcxeNrOlh/sgM7vRzKrNrLqhoSGRyEkhMz1N37hioXY0dbCMHgAA\nwBiV6EWJ5Wb2kJnVxx6/MbPyEfj+DEmVki6QdI2iFzu+65bm7n6Hu1e5e1VZWdkIfO34ce68Ul1+\n8jT96KkaLlAEAAAYgxKd8vEzScslTY89fhvbdzQ7Fb1F+UHlsX3x6iQtd/ded98qaaOiBRtxvvah\nBcrNStcXH1ijSF9/2HEAAAAQJ9FCXebuP3P3SOxxt6TBThWvkFRpZnPMLEvS1YqW8ngPK3p2WmZW\nqugUEOY2DDC5MEf/cOVJWlPbotuf3hx2HAAAAMRJtFDvM7NrzSw99rhW0r6j/YC7RyTdpOjFjOsl\nPeDua83sVjO7Ijbssdhnr5P0lKQvu/tRPzdVffCU6bpi0XT94IlNenNna9hxAAAAEGPuPvggs1mS\n/k3R24+7pBclfc7da4/6gwGoqqry6urq0f7aMaGlo0eXfP9ZFedlavlN5yknMz3sSAAAAEnLzFa6\ne9Vg4xJd5WO7u1/h7mXuPtndr5Q02CofGGHFeVn67p+coo172/Xd/34r7DgAAABQ4lM+DueLI5YC\nCXvfCZN1wzmz9bMXtumR1QOv8QQAAMBoG06hthFLgSH528vn68zZE/V/fvO61u3aH3YcAACAlDac\nQj345GsEIjM9Tbf92Wkqzs3S//5ltZoP9IQdCQAAIGUdtVCbWZuZ7T/Mo03R9agRkrKCbN1+7Wna\n29qtm+9bpb5+/v8GAAAgDEct1O5e4O6Fh3kUuHvGaIXE4Z06s0TfvHKhntvUqL97+E0lsmILAAAA\nRhaleJz72BkztaOpQ7c9tVnFeZn6P0tPDDsSAABASqFQJ4EvXXKCWjp6dfvTm1WUm6nPvPe4sCMB\nAACkDAp1EjAz3brsJLV29uo7f3hLRbmZuubMmWHHAgAASAkU6iSRnmb63lWL1dYV0VcfekPu0sfP\nolQDAAAEbTjL5mGMycpI008+cbouOL5MX33oDd31/NawIwEAACQ9CnWSyclM108+UaWlC6fq1t+t\n021P1YQdCQAAIKlRqJNQVkaafvTxU3Xl4un658c26Dt/eEv9rFMNAAAQCOZQJ6mM9DT961WLlZed\noR8/s1lbG9v1/Y8tVl4WhxwAAGAkcYY6iaWnmb515Un6u8vn6/F1e/Unt7+k2qaOsGMBAAAkFQp1\nkjMzffr8ufqP689QbXOHLv/hc3pi/d6wYwEAACQNCnWKeN+Jk/X7z52viol5+tQ91frOH95Sb19/\n2LEAAADGPQp1Cpk5KU+/+YtzdM2ZM/XjZzbr6jte1q6WzrBjAQAAjGsU6hSTk5mub3/kZP3g6sV6\na/d+XfqD5/Twqp1yZxUQAACAY0GhTlHLFs/Q724+X8eVTdAX7l+t//2LlWpo6w47FgAAwLhDoU5h\nc0on6NefOUdfvexEPb2xQZd8/xk9spqz1QAAAENBoU5x6WmmG99znB69+TzNmjRBn79vta7/2QqW\n1wMAAEgQhRqSpHmTC/SbvzhHX//QAq3c1qT3f/8Z/eSZzeqJsBIIAADA0VCocUh6mumGc+fo8S++\nV+fNK9O3//CWLvn+M3r0jd1MAwEAADgCCjXeZXpxrn563en62Q1nKDsjXX/5q9f0kdtf1IptTWFH\nAwAAGHMCLdRmttTMNphZjZndcpRxHzUzN7OqIPMgcWam9504WY9+/nz900dP0a6WTv3pj1/Sn/+8\nWjX17WHHAwAAGDMCK9Rmli7pNkmXSlog6RozW3CYcQWSPi/plaCy4Nilp5muOqNCT3/pffryB07Q\nS5v36QP/71n97UNvqH5/V9jxAAAAQhfkGeozJdW4+xZ375F0n6Rlhxn3TUnflUQ7G8Nys9L12ffN\n09NfvkDXnjVT96+o1XnffUpfe+RNNR/oCTseAABAaIIs1DMk1cZt18X2HWJmp0mqcPffB5gDI6g0\nP1vfWHaSnvzrC/TR02fol6/s0Hv++Sn96x83qIliDQAAUlBoFyWaWZqk70n66wTG3mhm1WZW3dDQ\nEHw4DGrmpDx9+yOn6Pc3n6dzjyvVvz1Zo3O/86S+8du12tXSGXY8AACAUWNBLYdmZmdL+rq7fyC2\n/RVJcvdvx7aLJG2WdPAKt6mSmiRd4e7VR/rcqqoqr64+4tsISU19m25/eoseWb1TUvTW5p9571xV\nTikIORkAAMCxMbOV7j7oohlBFuoMSRslXSRpp6QVkj7u7muPMP5pSV86WpmWKNRjXV1zh+58bqvu\nX1Grzt4+XXTiZH36/LlaMneizCzseAAAAAlLtFAHNuXD3SOSbpL0mKT1kh5w97VmdquZXRHU9yJc\n5SV5+voVC/XiLRfqCxdX6rUdzbrmpy/r8h8+rwdX1qk70hd2RAAAgBEV2BnqoHCGenzp6u3Tw6t2\n6q4Xtmrj3naV5mfrE0tm6c+WzFRpfnbY8QAAAI4o9CkfQaFQj0/urhdq9uk/nt+ipzY0KCs9TcsW\nT9cnz52jBdMLw44HAADwLokW6ozRCAOYmc6rLNV5laXa3NCuu1/YpgdX1unXK+u0ZO5EXbtkli5Z\nMFVZGaEtPAMAAHBMOEON0LR29OreFTv0y5e3q665U6X5WbqqqkLXnDlTFRPzwo4HAABSHFM+MG70\n97ue3dSgX72yQ0+s3yuX9N7jy/TxM2fqfSdOVmY6Z60BAMDoo1BjXNrd2qn7Xq3VfSt2aO/+bpXm\nZ+ujp8/QVVUVOq4sP+x4AAAghVCoMa5F+vr19IYG3V9dqyffqldfv6tqVomuOqNCl588TROymf4P\nAACCRaFG0qhv69JDr+3U/dW12tJwQHlZ6bpkwRQtO3WGzptXypQQAAAQCAo1ko6767UdzXpwZZ0e\nfWOPWjt7NWlCli4/ZZqWLZ6h02YWczdGAAAwYijUSGrdkT49s6FBj6zZpf9Zt1fdkX5VTMzVskUz\ndOWp0zVvckHYEQEAwDhHoUbKaOvq1WNr9+qR1Tv1Qk2j+l1aMK1QV546XR9aNF3TinLDjggAAMYh\nCjVSUn1bl37/+m49vHqX1tS2yEw6a85EXXrSNL1/wRRNL6ZcAwCAxFCokfK2NR7QI6t36bev71JN\nfbsk6aQZhbpkwVRdsnCKTphSwJxrAABwRBRqIM6WhnY9vm6v/rhur17b0Sx3qWJibrRcL5ii02eV\nKIPVQgAAQBwKNXAE9W1demJ9vR5ft1fP1zSqJ9KvkrxMXTR/ii5ZMEXnV5YpNys97JgAACBkFGog\nAe3dET27sUGPr9urJ9bv1f6uiHIy03R+ZZkuWTBFF82fookTssKOCQAAQpBooeZ2c0hp+dkZuuzk\nabrs5Gnq7evXq1ub9Me1e/THdXv1+Lq9SjOpavZEXbJgii44YbKOK5vAvGsAAPAOnKEGDsPdtXbX\n/kPl+q09bZKkaUU5Or+yVOdXlunceaWcvQYAIIkx5QMYQbVNHXpuU6Oe29SgF2oatb8rIjPppOlF\nOq+yVOdXlur0WSXKzmDuNQAAyYJCDQSkr9/1el2LntvUqOc3Neq1Hc2K9LtyM9N11tyJOr+yTO+p\nLNW8yflMDwEAYByjUAOjpL07opc379Nzmxr03KZGbWk8IEmaUpitc48r1RlzJuqM2ROZfw0AwDjD\nRYnAKMnPztDFC6bo4gVTJEl1zR16flOjntvUqGc2Nui/Vu2UJE2akKWq2SU6Y3a0YC+cXsja1wAA\nJAHOUAMBcndtaTygFVub9Oq2JlVva9aOpg5JUl5Wuk6bGS3Yp88q0SkVRSrMyQw5MQAAOIgpH8AY\ntae1Syu2NWnFtia9urVJG/a2yV0yk+aV5WtxRbEWzyzW4opinTClgLPYAACEhEINjBOtnb1aU9ui\n1bHHqh3Nau7olSTlZqbr5PIinVoRLdinzizR1KKckBMDAJAamEMNjBNFuZl6z/Fles/xZZKi00R2\nNHXEynW0ZP/shW3q6euXJJXmZ2vB9EItnF6oBdMKtWB6oWZPmqD0NC54BAAgDBRqYIwxM82aNEGz\nJk3QssUzJEndkT6t27Vfq2tbtHbXfq3btV93PrdFvX3Rv2HKzUzXidMKYiW7SAumF+qEKQXKzWJd\nbAAAghbolA8zWyrpB5LSJd3p7t8Z8P4XJX1aUkRSg6T/5e7bj/aZTPkAonoi/aqpb9faXa1atzta\nstft3q+2rogkKc2k48rytSB2Jnv+tEKdMLVAkwuyWb4PAIAEhD7lw8zSJd0m6f2S6iStMLPl7r4u\nbtgqSVXu3mFmfyHpnyR9LKhMQDLJykiLluXphYf2ubvqmjujZ7FjJbt6W7MeWb3r0JjCnAxVTinQ\n8VPyVTm5QMfHXpdRtAEAOCZBTvk4U1KNu2+RJDO7T9IySYcKtbs/FTf+ZUnXBpgHSHpmpoqJeaqY\nmKelJ009tL/5QI/W79mvTXvbtam+TRv3tuu/39yjeztqD40pys1U5eR8VU7J15zSCZpTGn2eOTFP\nWRmsNAIAwJEEWahnSKqN266TdNZRxn9K0h8O94aZ3SjpRkmaOXPmSOUDUkbJhCydc1ypzjmu9NA+\nd1dje4827W3Tpvp2bdzbpk172/XY2r1qOtBzaFyaSRUT82Ile4LmHizbZRM0rTBHaVwMCQBIcWPi\nokQzu1ZSlaT3Hu59d79D0h1SdA71KEYDkpaZqawgW2UF2TpnXuk73mvt6NXWfQe0tbFdWxsOaEvj\nAW1tPKBXtzapo6fv0LjsjDTNKZ2g2ZMmaNakPJVPzNPMiXmqKMnVjJJcZWdwUSQAIPkFWah3SqqI\n2y6P7XsHM7tY0t9Keq+7dweYB0CCivIytTgvuvZ1PHdXfVu3tjREC/bWxnZtbTygjfVtenJDvXoi\n/YfGmklTC3NUUZIXm4aSGy3bsdJdlp/N2W0AQFIIslCvkFRpZnMULdJXS/p4/AAzO1XSTyQtdff6\nALMAGAFmpimFOZpSmKOzj5v0jvf6+6Nlu7a5Qzv2dUSfmzpU19SpFzc3as+qLsUvKpSVkabyklyV\nl+RpelGOphXlalpRjqYV50Sfi3I1IXtM/CUaAABHFdh/rdw9YmY3SXpM0WXz7nL3tWZ2q6Rqd18u\n6Z8l5Uv6dWx1gR3ufkVQmQAEJy3NNLUoR1OLcnTG7Inver870qedzZ3a0dSh2uZO1TZ1qLapQ3XN\nnVq3a78a29/9F1QFORmHynX889SinFixz1ZRbiarkwAAQsWtxwGMCT2Rfu3d36XdrV3a3doZfW6J\nPccehyvdWRlpKsvP1pTCbE0uyNHkwmxNKcxRWUG2JhdEX08uyFZJXhZTTAAAQxL6OtQAMBRZGWmH\nlvw7koOle1dLp+rbumOPLtXvjz5vbmjXi5sbtT92c5t4memm0vxsTcrP0qQJ2SrNz1Zpftbb2wXZ\nmjQhS6X52Zo4IYulAgEACaNQAxg3EindktTV26eGtm7t3d8VLd37u7S3rVuNbd3ad6BHje3dqqlv\nV0N79zsupIxXlJupSflZKp2QrdKCaOmOlu8sFedlqSQvS8V5mSqZkKWSvEzlZqYz9QQAUhSFGkDS\nyclMT6h4u7sO9PTFina3Gtt7tK89Wrj3tXer8UCP9rV3a9Pedr3Uvk/NHb1H/KysjDSV5GW+XbTz\nDhbvd+4rmZCpotwsFeVmqjA3g6UFASAJUKgBpCwzU352hvKzMzS7dMKg4yN9/Wru6FVLR4+aO3rV\n3NHzztcHDu7rVU19+6Gxkf4jX6uSnZGmwtxMFeZkxEp2pgpzomW76NDr6PPBEn5wX0FOhjLTmZoC\nAGGjUANAgjLS0w7dDCdR7q727ohaOnrVdKBHzR09au3s1f6uiPZ39kYfXb3a3xlRa2d0zLbGA9rf\nFd3uO0oZl6QJWenvKOGFOf+/vfuPleys6zj++ZwzM/fevbsppcWmoeBq2MTUCJUQUpU/sEZSlQiJ\nREowEtKEhBhTEhGqf2g08of+IVglJqhoNfiDoFXCH4ambYBEBYss/SEaS1MiTcu2lrb74+7MnHO+\n/vE8M3Pm9u7e3Z3dO3fOfb+SyTnP9zlzznPzzc5+zzPPzPR1eD3dJBxe7+nwoDdtH1nv6fBaX5tr\n5XT/8HpPh/olH9gEgAVQUAPAFWRbR9b7OrLe33UJynYRoTOjelpwp+04FeStovyFVlH+9ItndeqZ\nSqeHlU6erTQ8xxrx+TFKm4NWEZ5n7Q8NSm22thv9UptrpQ4NUuzQoDfX3hz0dGit1KFByZpyAAcK\nBTUA7FO2tbnW0+ZaT9dfdWnnGFWNTg8rnWo/zs7vn8zbyXGpPdazp4Y6M6p1ZlTp9LDW1rje/YLT\nsUuH+qUOrbWK70GpjVbhvZkL8fV+emz0i7Sdi6XH+va+XqEey10A7BMU1ADQYYNeoUFvoKs3Bwuf\nq2lCW+Nap0eVzgzrWbE9qrWVi+5J+8yo1plhqy8fe/JspRMvDtM5RrVODy9sFn0n/dLTonu6HaRi\ne2Mwi6/ngnyj344VWuuVWpvb5v1eKt7XesW0f71faFAWzLoD2BEFNQDgghTFbMZcRy7feSNCw6rR\n1ijNgp8dz7Znx/PxWV+jrXGtrVGtYVW3jknx506PpsdujRoN8/75PiB6IVLRnQvu/qwAX+sV+eal\n1KCctfulU7wsc3/uK4tpuz/ZL9vnmR3Tb8dbz+sVpsAH9gkKagDAUtmeziRffYWvNa5TwT2qGp0d\n1xpWjYbjRsMq7U9jVSrCXxKr6nz8rH/y3GHV6MWtsUZVo1HdpO0O+5eLrVmBXb60EJ/bLwv1SqtX\nFuoXeVtavSLHi/m+Xmn1t/X1Cqs/OU+Rnl9OYuc45/bjJ+fpFem53BCgKyioAQAHRr8slvpVgxGh\ncR3zRXbVaFSnonzSTsekwn+4U2Hebp+neB9WjU6erTSqGtVNaNw0qupQVTcaN2lb1aGqCVVNuu5e\nahfg/bJIBfplKuonfS+9ibDKc9xYpOfsfGMx6WuPYdJXcpNw4FFQAwCwR2xr0EvLQHTh3764ZyJC\ndZMK7PG2Yruqcyz31U26OahasXR8arePr+pUrLf72kX9uJ5ct5me81wF/5lRla+X+ts3CtNrzG33\n9iahsFTm4rq0Z/vtWDnfV9ipMN/p+KJQ2T7nXKxQWWy/3iSWtzvFJs8vi9w/31c4Pa9o/Q2T/cJp\n+ddk3JPrujXGC4pPxzaLF9bK3pBQUAMAAEmpmElLQ9IvjnZBRMwK/O3Fdt0uxlOxX0+L+vkZ/Wqu\n4J8v/tPz0qPJ12vyjckkXscsNu2LUF3nbfv5OTYeN6qb+oLOPT2mfe4cWyXeoZjvldbx33jLsod2\nXhTUAACgs+y0XKNfShvqxk3CxYgINaG5grze/oh2kd5Mj588Z7KdnqdVsDcRahpNz3G+eDM9p6bn\nncXTsbFDfBVQUAMAAHSU7emSEVw5fCs+AAAAsAAKagAAAGABFNQAAADAAiioAQAAgAVQUAMAAAAL\noKAGAAAAFkBBDQAAACzAEav2Czp+RtK3lnT5ayU9u6RrY++Q5+4jxwcDeT4YyPPBsKw8f29EvGK3\ng1auoF4m2w9GxBuWPQ5cWeS5+8jxwUCeDwbyfDDs9zyz5AMAAABYAAU1AAAAsAAK6ovziWUPAHuC\nPHcfOT4YyPPBQJ4Phn2dZ9ZQAwAAAAtghhoAAABYAAU1AAAAsAAK6gtg+1bb/237Mdt3Lns8uHS2\nP2n7hO1HWrGX277X9v/k7dU5btt35bw/ZPv1yxs5LobtV9l+wPZ/2n7U9h05Tq47xPa67a/Y/nrO\n82/l+PfZ/nLO59/ZHuT4Wm4/lvuPLnP8uDi2S9tfs/253CbPHWP7CdsP2z5u+8EcW4nXbQrqXdgu\nJX1c0k9JulHSu2zfuNxRYQF/IenWbbE7Jd0XEcck3ZfbUsr5sfx4n6Q/3qMxYnGVpF+JiBsl3Szp\nl/K/W3LdLUNJt0TE6yTdJOlW2zdL+l1JH42I10j6rqTb8/G3S/pujn80H4fVcYekb7Ta5Lmbfjwi\nbmp95/RKvG5TUO/ujZIei4jHI2Ik6W8lvW3JY8IliogvSnpuW/htku7O+3dLensr/peR/Jukl9m+\nfm9GikVExFMR8R95/6TSf8KvFLnulJyvU7nZz4+QdIukz+T49jxP8v8ZST9h23s0XCzA9g2SfkbS\nn+a2RZ4PipV43aag3mcTN14AAAQBSURBVN0rJf1vq/3tHEN3XBcRT+X9pyVdl/fJfQfkt3t/WNKX\nRa47Jy8DOC7phKR7JX1T0vMRUeVD2rmc5jn3vyDpmr0dMS7RxyR9SFKT29eIPHdRSPq87a/afl+O\nrcTrdm9ZFwb2o4gI23yXZEfYPizp7yV9ICJebE9SketuiIha0k22XybpHkk/sOQh4TKz/VZJJyLi\nq7bfvOzx4Ip6U0Q8aft7JN1r+7/anfv5dZsZ6t09KelVrfYNOYbu+M7kbaK8PZHj5H6F2e4rFdOf\nioh/yGFy3VER8bykByT9iNJbv5MJo3Yup3nO/VdJ+r89Hiou3o9J+lnbTygtu7xF0h+IPHdORDyZ\ntyeUbpDfqBV53aag3t2/SzqWP008kHSbpM8ueUy4vD4r6T15/z2S/qkV/8X8SeKbJb3QetsJ+1he\nL/lnkr4REb/f6iLXHWL7FXlmWrY3JP2k0nr5ByS9Ix+2Pc+T/L9D0v3Br5vtexHxaxFxQ0QcVfo/\n+P6IeLfIc6fY3rR9ZLIv6S2SHtGKvG7zS4kXwPZPK63fKiV9MiI+suQh4RLZ/htJb5Z0raTvSPpN\nSf8o6dOSXi3pW5J+PiKey0XZHyl9K8gZSe+NiAeXMW5cHNtvkvQlSQ9rtuby15XWUZPrjrD9WqUP\nKZVKE0Sfjojftv39SjOZL5f0NUm/EBFD2+uS/kppTf1zkm6LiMeXM3pcirzk44MR8Vby3C05n/fk\nZk/SX0fER2xfoxV43aagBgAAABbAkg8AAABgARTUAAAAwAIoqAEAAIAFUFADAAAAC6CgBgAAABZA\nQQ0AK8R2bft463HnZTz3UduPXK7zAcBBwU+PA8Bq2YqIm5Y9CADADDPUANABtp+w/Xu2H7b9Fduv\nyfGjtu+3/ZDt+2y/Osevs32P7a/nx4/mU5W2/8T2o7Y/n3+BEABwHhTUALBaNrYt+Xhnq++FiPgh\npV8P+1iO/aGkuyPitZI+JemuHL9L0hci4nWSXi/p0Rw/JunjEfGDkp6X9HNX+O8BgJXHLyUCwAqx\nfSoiDu8Qf0LSLRHxuO2+pKcj4hrbz0q6PiLGOf5URFxr+xlJN0TEsHWOo5LujYhjuf1hSf2I+J0r\n/5cBwOpihhoAuiPOsX8xhq39WnzWBgB2RUENAN3xztb2X/P+v0i6Le+/W9KX8v59kt4vSbZL21ft\n1SABoGuYeQCA1bJh+3ir/c8RMfnqvKttP6Q0y/yuHPtlSX9u+1clPSPpvTl+h6RP2L5daSb6/ZKe\nuuKjB4AOYg01AHRAXkP9hoh4dtljAYCDhiUfAAAAwAKYoQYAAAAWwAw1AAAAsAAKagAAAGABFNQA\nAADAAiioAQAAgAVQUAMAAAAL+H+ce7yjmOMviAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 864x576 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "None\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "axA6WuGVO0nR"
      },
      "source": [
        "Note that the y-axes of the graphs are not zero-based."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "z-EvK7hGL0d8"
      },
      "source": [
        "### Setup the test dataset\n",
        "http://download.tensorflow.org/data/iris_training.csv:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "SRMWCu30bnxH",
        "scrolled": true,
        "colab": {}
      },
      "source": [
        "let testDataFilename = \"iris_test.csv\"\n",
        "download(from: \"http://download.tensorflow.org/data/iris_test.csv\", to: testDataFilename)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "jEPPL6FUO0nV"
      },
      "source": [
        " Now load it into a `Dataset`:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "w6SCt95HO0nW",
        "scrolled": true,
        "colab": {}
      },
      "source": [
        "let testDataset: Dataset<IrisBatch> = Dataset(\n",
        "    contentsOfCSVFile: testDataFilename, hasHeader: true,\n",
        "    featureColumns: [0, 1, 2, 3], labelColumns: [4]\n",
        ").batched(batchSize)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "HFuOKXJdMAdm"
      },
      "source": [
        "### Evaluate the model on the test dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Tj4Rs8gwO0nY",
        "outputId": "e19d14c4-451a-44a6-9045-f9e0b3e4c0e2",
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "// NOTE: With `batchSize = 32` and 30 examples in the test dataset, only one batch will run in the loop.\n",
        "for testBatch in testDataset {\n",
        "    let logits = model(testBatch.features)\n",
        "    let predictions = logits.argmax(squeezingAxis: 1)\n",
        "    print(\"Test batch accuracy: \\(accuracy(predictions: predictions, truths: testBatch.labels))\")\n",
        "}"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test batch accuracy: 0.96666664\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "HcKEZMtCOeK-"
      },
      "source": [
        "We can see on the first batch, for example, the model is usually correct:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "uNwt2eMeOane",
        "outputId": "8462f514-6e41-4bc5-adab-ffd52a4dfaba",
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "let firstTestBatch = testDataset.first!\n",
        "let firstTestBatchLogits = model(firstTestBatch.features)\n",
        "let firstTestBatchPredictions = firstTestBatchLogits.argmax(squeezingAxis: 1)\n",
        "\n",
        "print(firstTestBatchPredictions)\n",
        "print(firstTestBatch.labels)"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1, 2, 0, 1, 1, 1, 0, 1, 1, 2, 2, 0, 2, 1, 1, 0, 1, 0, 0, 2, 0, 1, 2, 1, 1, 1, 0, 1, 2, 1]\r\n",
            "[1, 2, 0, 1, 1, 1, 0, 2, 1, 2, 2, 0, 2, 1, 1, 0, 1, 0, 0, 2, 0, 1, 2, 1, 1, 1, 0, 1, 2, 1]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fsmD1eQunRiK",
        "colab_type": "text"
      },
      "source": [
        "### Could be better if use train, val, test set; current method confuse people"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "7Li2r1tYvW7S"
      },
      "source": [
        "## Use the trained model to make predictions\n",
        "\n",
        "* `0`: Iris setosa\n",
        "* `1`: Iris versicolor\n",
        "* `2`: Iris virginica"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "MTYOZr27O0ne",
        "outputId": "7411cb14-3d45-470e-a019-18108a86e377",
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "source": [
        "let unlabeledDataset: Tensor<Float> =\n",
        "    [[5.1, 3.3, 1.7, 0.5],\n",
        "     [5.9, 3.0, 4.2, 1.5],\n",
        "     [6.9, 3.1, 5.4, 2.1]]\n",
        "\n",
        "let unlabeledDatasetPredictions = model(unlabeledDataset)\n",
        "\n",
        "for i in 0..<unlabeledDatasetPredictions.shape[0] {\n",
        "    let logits = unlabeledDatasetPredictions[i]\n",
        "    let classIdx = logits.argmax().scalar!\n",
        "    print(\"Example \\(i) prediction: \\(classNames[Int(classIdx)]) (\\(softmax(logits)))\")\n",
        "}"
      ],
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Example 0 prediction: Iris setosa ([   0.9941551,  0.005844921, 9.754981e-11])\r\n",
            "Example 1 prediction: Iris versicolor ([0.0014089192,    0.9922127, 0.0063782893])\r\n",
            "Example 2 prediction: Iris virginica ([3.2954597e-06,      0.203958,     0.7960387])\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HwRFrSsul5dS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}