Basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Caffe2 Basic Concepts - Operators & Nets\n",
    "\n",
    "In this tutorial we will go through a set of Caffe2 basics: the basic concepts including how operators and nets are being written.\n",
    "\n",
    "First, let's import Caffe2. `core` and `workspace` are usually the two that you need most. If you want to manipulate protocol buffers generated by Caffe2, you probably also want to import `caffe2_pb2` from `caffe2.proto`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.\n",
      "WARNING:root:Debug message: No module named caffe2_pybind11_state_gpu\n"
     ]
    }
   ],
   "source": [
    "from __future__ import absolute_import\n",
    "from __future__ import division\n",
    "from __future__ import print_function\n",
    "from __future__ import unicode_literals\n",
    "\n",
    "# We'll also import a few standard python libraries\n",
    "from matplotlib import pyplot\n",
    "import numpy as np\n",
    "import time\n",
    "\n",
    "# These are the droids you are looking for.\n",
    "from caffe2.python import core, workspace\n",
    "from caffe2.proto import caffe2_pb2\n",
    "\n",
    "# Let's show all plots inline.\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You might see a warning saying that caffe2 does not have GPU support. That means you are running a CPU-only build. Don't be alarmed - anything CPU is still runnable without a problem."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Workspaces\n",
    "\n",
    "Let's cover workspaces first, where all the data resides.\n",
    "\n",
    "Similar to Matlab, the Caffe2 workspace consists of blobs you create and store in memory. For now, consider a blob to be a N-dimensional Tensor similar to numpy's ndarray, but contiguous. Down the road, we will show you that a blob is actually a typed pointer that can store any type of C++ objects, but Tensor is the most common type stored in a blob. Let's show what the interface looks like.\n",
    "\n",
    "`Blobs()` prints out all existing blobs in the workspace. \n",
    "`HasBlob()` queries if a blob exists in the workspace. As of now, we don't have any."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current blobs in the workspace: []\n",
      "Workspace has blob 'X'? False\n"
     ]
    }
   ],
   "source": [
    "print(\"Current blobs in the workspace: {}\".format(workspace.Blobs()))\n",
    "print(\"Workspace has blob 'X'? {}\".format(workspace.HasBlob(\"X\")))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can feed blobs into the workspace using `FeedBlob()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generated X from numpy:\n",
      "[[ 1.0922441  -0.65129787 -0.2511869 ]\n",
      " [ 1.3999398  -0.86516035 -2.0602188 ]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = np.random.randn(2, 3).astype(np.float32)\n",
    "print(\"Generated X from numpy:\\n{}\".format(X))\n",
    "workspace.FeedBlob(\"X\", X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's take a look at what blobs are in the workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current blobs in the workspace: [u'X']\n",
      "Workspace has blob 'X'? True\n",
      "Fetched X:\n",
      "[[ 1.0922441  -0.65129787 -0.2511869 ]\n",
      " [ 1.3999398  -0.86516035 -2.0602188 ]]\n"
     ]
    }
   ],
   "source": [
    "print(\"Current blobs in the workspace: {}\".format(workspace.Blobs()))\n",
    "print(\"Workspace has blob 'X'? {}\".format(workspace.HasBlob(\"X\")))\n",
    "print(\"Fetched X:\\n{}\".format(workspace.FetchBlob(\"X\")))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's verify that the arrays are equal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.testing.assert_array_equal(X, workspace.FetchBlob(\"X\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that if you try to access a blob that does not exist, an error will be thrown:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[enforce fail at pybind_state.cc:173] ws->HasBlob(name). Can't find blob: invincible_pink_unicorn \n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    workspace.FetchBlob(\"invincible_pink_unicorn\")\n",
    "except RuntimeError as err:\n",
    "    print(err)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One thing that you might not use immediately: you can have multiple workspaces in Python using different names, and switch between them. Blobs in different workspaces are separate from each other. You can query the current workspace using `CurrentWorkspace`. Let's try switching the workspace by name (gutentag) and creating a new one if it doesn't exist."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current workspace: default\n",
      "Current blobs in the workspace: [u'X']\n",
      "Current workspace: gutentag\n",
      "Current blobs in the workspace: []\n"
     ]
    }
   ],
   "source": [
    "print(\"Current workspace: {}\".format(workspace.CurrentWorkspace()))\n",
    "print(\"Current blobs in the workspace: {}\".format(workspace.Blobs()))\n",
    "\n",
    "# Switch the workspace. The second argument \"True\" means creating \n",
    "# the workspace if it is missing.\n",
    "workspace.SwitchWorkspace(\"gutentag\", True)\n",
    "\n",
    "# Let's print the current workspace. Note that there is nothing in the\n",
    "# workspace yet.\n",
    "print(\"Current workspace: {}\".format(workspace.CurrentWorkspace()))\n",
    "print(\"Current blobs in the workspace: {}\".format(workspace.Blobs()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's switch back to the default workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current workspace: default\n",
      "Current blobs in the workspace: [u'X']\n"
     ]
    }
   ],
   "source": [
    "workspace.SwitchWorkspace(\"default\")\n",
    "print(\"Current workspace: {}\".format(workspace.CurrentWorkspace()))\n",
    "print(\"Current blobs in the workspace: {}\".format(workspace.Blobs()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, `ResetWorkspace()` clears anything that is in the current workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current blobs in the workspace after reset: []\n"
     ]
    }
   ],
   "source": [
    "workspace.ResetWorkspace()\n",
    "print(\"Current blobs in the workspace after reset: {}\".format(workspace.Blobs()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Operators\n",
    "\n",
    "Operators in Caffe2 are kind of like functions. From the C++ side, they all derive from a common interface, and are registered by type, so that we can call different operators during runtime. The interface of operators is defined in `caffe2/proto/caffe2.proto`. Basically, it takes in a bunch of inputs, and produces a bunch of outputs.\n",
    "\n",
    "Remember, when we say \"create an operator\" in Caffe2 Python, nothing gets run yet. All it does is create the protocol buffer that specifies what the operator should be. At a later time it will be sent to the C++ backend for execution. If you are not familiar with protobuf, it is a json-like serialization tool for structured data. Find more about protocol buffers [here](https://developers.google.com/protocol-buffers/).\n",
    "\n",
    "Let's see an actual example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create an operator.\n",
    "op = core.CreateOperator(\n",
    "    \"Relu\", # The type of operator that we want to run\n",
    "    [\"X\"], # A list of input blobs by their names\n",
    "    [\"Y\"], # A list of output blobs by their names\n",
    ")\n",
    "# and we are done!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we mentioned, the created op is actually a protobuf object. Let's show the content."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Type of the created op is: <class 'caffe2.proto.caffe2_pb2.OperatorDef'>\n",
      "Content:\n",
      "\n",
      "input: \"X\"\n",
      "output: \"Y\"\n",
      "name: \"\"\n",
      "type: \"Relu\"\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(\"Type of the created op is: {}\".format(type(op)))\n",
    "print(\"Content:\\n\")\n",
    "print(str(op))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ok, let's run the operator. We first feed the input X to the workspace. \n",
    "Then the simplest way to run an operator is to do `workspace.RunOperatorOnce(operator)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "workspace.FeedBlob(\"X\", np.random.randn(2, 3).astype(np.float32))\n",
    "workspace.RunOperatorOnce(op)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After execution, let's see if the operator is doing the right thing.\n",
    "\n",
    "In this case, the operator is a common activation function used in neural networks, called [ReLU](https://en.wikipedia.org/wiki/Rectifier_(neural_networks), or Rectified Linear Unit activation. ReLU activation helps to add necessary non-linear characteristics to the neural network classifier, and is defined as:\n",
    "\n",
    "$$ReLU(x) = max(0, x)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current blobs in the workspace: [u'X', u'Y']\n",
      "\n",
      "X:\n",
      "[[-0.958609   -1.5830146  -1.1998546 ]\n",
      " [ 0.77405983  0.08448339  0.57201284]]\n",
      "\n",
      "Y:\n",
      "[[0.         0.         0.        ]\n",
      " [0.77405983 0.08448339 0.57201284]]\n",
      "\n",
      "Expected:\n",
      "[[0.         0.         0.        ]\n",
      " [0.77405983 0.08448339 0.57201284]]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(\"Current blobs in the workspace: {}\\n\".format(workspace.Blobs()))\n",
    "print(\"X:\\n{}\\n\".format(workspace.FetchBlob(\"X\")))\n",
    "print(\"Y:\\n{}\\n\".format(workspace.FetchBlob(\"Y\")))\n",
    "print(\"Expected:\\n{}\\n\".format(np.maximum(workspace.FetchBlob(\"X\"), 0)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is working if your Expected output matches your Y output in this example.\n",
    "\n",
    "Operators also take optional arguments if needed. They are specified as key-value pairs. Let's take a look at one simple example, which takes a tensor and fills it with Gaussian random variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Content of op:\n",
      "\n",
      "output: \"Z\"\n",
      "name: \"\"\n",
      "type: \"GaussianFill\"\n",
      "arg {\n",
      "  name: \"std\"\n",
      "  f: 1.0\n",
      "}\n",
      "arg {\n",
      "  name: \"shape\"\n",
      "  ints: 100\n",
      "  ints: 100\n",
      "}\n",
      "arg {\n",
      "  name: \"mean\"\n",
      "  f: 1.0\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "op = core.CreateOperator(\n",
    "    \"GaussianFill\",\n",
    "    [], # GaussianFill does not need any parameters.\n",
    "    [\"Z\"],\n",
    "    shape=[100, 100], # shape argument as a list of ints.\n",
    "    mean=1.0,  # mean as a single float\n",
    "    std=1.0, # std as a single float\n",
    ")\n",
    "print(\"Content of op:\\n\")\n",
    "print(str(op))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's run it and see if things are as intended."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,u'Distribution of Z')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAFPdJREFUeJzt3X+w5XV93/HnS35IIgoiC4O7q6thx+I4EZktIUMbI9iUX3FpG6KpygY3s52WWFPtKChJtNUW61Sj1ZLZEcqi+INiHDb+RpQxzgR1UUB0VbYU4XaRXURQJCZZfPeP89l4uXvv3nN3z73n7IfnY+bO+X4/38/3e9534b7u537O53xPqgpJUr+eMO4CJEmLy6CXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQa+JkOTPk/zxiK71jCQPJzmo7d+Y5A9Gce12vU8nWTeq6y3ged+a5P4kP1jq59aBLa6j12JLchdwLLALeBT4NnAVsLGqfr4P1/qDqvr8As65EfhgVb1/Ic/Vzn0zcHxVvWKh545SkpXA94BnVtWOWY5/GvinM5oPBp7Yzrl78avUpDp43AXoceO3q+rzSY4AXgi8G/g14IJRPkmSg6tq1yivOSGeCfxwtpAHqKozp++3v2ZuAL5vyMupGy2pqnqoqjYDLwXWJXkeQJIrk7y1bR+d5BNJHkzyQJK/SvKEJB8AngH8ZZuaeX2SVUkqyfokdwNfmNY2fSDzK0m+muShJNclOao9128mmZpeY5K7krw4yRnAG4GXtue7tR3/h6mgVtclSb6fZEeSq9ovM6bVsS7J3W3a5U1z/dskOaKdv7Nd75J2/RcD1wNPb3VcOcQ/9X8BjgL+7RB91TmDXmNRVV8FpthzugHgde3YMgZTPm8cnFKvBO5m8NfB4VX136ad80LgBOCfz/GU5wOvAp7OYArpPUPU+BkGgfnR9nzPn6Xb77evFwHPBg4H3jujzz8BngOcDvxJkhPmeMr/ARzRrvPCVvMFbZrqTGB7q+P391Z3krXAvwH+VVU9Ms+3qccBg17jtJ3BqHOmvweOYzC3/PdV9Vc1/4tJb66qn1bV38xx/ANVdXtV/RT4Y+B3d79Yu59eDryzqu6sqoeBi4GXzfhr4i1V9TdVdStwK7DHL4xWy0uBi6vqJ1V1F/DfgVcupJgkvwJcCayvqjv25RtSfwx6jdNy4IFZ2t8BbAM+l+TOJBcNca17FnD8+8AhwNFDVbl3T2/Xm37tgxn8JbLb9FUyjzAY9c90NHDoLNdaPmwhSQ4DrgWuqKqPDXue+mfQayyS/GMGIfblmcfaiPZ1VfVs4LeB1yY5fffhOS4534h/5bTtZzD4q+F+4KfAL0+r6yAGU0bDXnc7gxdKp197F3DfPOfNdH+raea1/t8CrvE+Bt/PGxb43OqcQa8lleQpSc4BPsJgyeM3Z+lzTpLjkwT4MYMlmY+2w/cxmMNeqFckeW6SXwb+E3BtVT3KYMniYUnOTnIIcAmDJYm73QesSjLXz8qHgf+Q5FlJDucXc/oLWvnTarkGeFuSJyd5JvBa4IPDnJ/kVcA5wO92uupI+8Gg11L5yyQ/YTCF8ibgncy9tHI18HngYeCvgf9ZVTe2Y/8VuKStyPmPC3j+DzCYu/4BcBjw72GwCgj4d8D7GYyef8rgheDd/nd7/GGSr89y3Svatb8E/F/gZ8CrF1DXdK9uz38ng790PtSuP4xLGLze8b22Mmf612wveOtxxDdMSVLnHNFLUucMeknqnEEvSZ0z6CWpcxNxU7Ojjz66Vq1aNe4yJOmAcvPNN99fVcvm6zcRQb9q1Sq2bNky7jIk6YCS5Pvz93LqRpK6Z9BLUueGCvokRya5Nsl3kmxN8utJjkpyfZI72uNTW98keU+SbUluS3LS4n4LkqS9GXZE/27gM1X1jxjcYnUrcBFwQ1WtZvBJNrvvMHgmg7ewrwY2AJeNtGJJ0oLMG/RJngL8BnA5QFX9XVU9CKwFNrVum4Bz2/Za4KoauAk4MslxI69ckjSUYUb0zwZ2Av8ryTeSvD/Jk4Bjq+pegPZ4TOu/nMfe+3uKWe6pnWRDki1JtuzcuXO/vglJ0tyGCfqDgZOAy6rqBQzurre3D4LILG173DmtqjZW1ZqqWrNs2bzLQCVJ+2iYoJ8CpqrqK23/WgbBf9/uKZn2uGNa/+kf8rCCwYczSJLGYN6gr6ofAPckeU5rOh34NrAZWNfa1gHXte3NwPlt9c0pwEO7p3gkSUtv2HfGvhq4OsmhDD4U4QIGvySuSbIeuBs4r/X9FHAWg8/8fIS5P1xCmnirLvrknMfuuvTsJaxE2ndDBX1V3QKsmeXQ6bP0LeDC/axLWhRzBbehrZ75zlhJ6pxBL0mdM+glqXMTcZtiadz29qKrdKAz6NUlg1v6BaduJKlzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnOvopX3kDdJ0oHBEL0mdc0QvLRH/AtC4OKKXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnXN5pTRifuiJJo0jeknqnEEvSZ1z6kYHNKdJpPk5opekzg0V9EnuSvLNJLck2dLajkpyfZI72uNTW3uSvCfJtiS3JTlpMb8BSdLeLWRE/6KqOrGq1rT9i4Abqmo1cEPbBzgTWN2+NgCXjapYSdLC7c/UzVpgU9veBJw7rf2qGrgJODLJcfvxPJKk/TBs0BfwuSQ3J9nQ2o6tqnsB2uMxrX05cM+0c6da22Mk2ZBkS5ItO3fu3LfqJUnzGnbVzalVtT3JMcD1Sb6zl76Zpa32aKjaCGwEWLNmzR7HJUmjMdSIvqq2t8cdwMeBk4H7dk/JtMcdrfsUsHLa6SuA7aMqWJK0MPMGfZInJXny7m3gt4Dbgc3AutZtHXBd294MnN9W35wCPLR7ikeStPSGmbo5Fvh4kt39P1RVn0nyNeCaJOuBu4HzWv9PAWcB24BHgAtGXrUkaWjzBn1V3Qk8f5b2HwKnz9JewIUjqU6StN+8BYIOCN7qQNp33gJBkjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc7lldKYzbV09K5Lz17iStQrR/SS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOdfSaKN6OWBo9R/SS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcy6vlCaUty/WqDiil6TOGfSS1DmDXpI6N3TQJzkoyTeSfKLtPyvJV5LckeSjSQ5t7U9s+9va8VWLU7okaRgLGdG/Btg6bf/twLuqajXwI2B9a18P/Kiqjgfe1fpJksZkqFU3SVYAZwNvA16bJMBpwL9uXTYBbwYuA9a2bYBrgfcmSVXV6MrWgc6bl0lLZ9gR/Z8Brwd+3vafBjxYVbva/hSwvG0vB+4BaMcfav0fI8mGJFuSbNm5c+c+li9Jms+8QZ/kHGBHVd08vXmWrjXEsV80VG2sqjVVtWbZsmVDFStJWrhhpm5OBV6S5CzgMOApDEb4RyY5uI3aVwDbW/8pYCUwleRg4AjggZFXLkkayrwj+qq6uKpWVNUq4GXAF6rq5cAXgd9p3dYB17XtzW2fdvwLzs9L0vjszzr6NzB4YXYbgzn4y1v75cDTWvtrgYv2r0RJ0v5Y0L1uqupG4Ma2fSdw8ix9fgacN4LaJEkj4DtjJalzBr0kdc6gl6TOGfSS1DmDXpI65ydMSQcYP3lKC+WIXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM55P3otqrnunS5p6Tiil6TOGfSS1DmDXpI6Z9BLUufmDfokhyX5apJbk3wryVta+7OSfCXJHUk+muTQ1v7Etr+tHV+1uN+CJGlvhhnR/y1wWlU9HzgROCPJKcDbgXdV1WrgR8D61n898KOqOh54V+snSRqTeYO+Bh5uu4e0rwJOA65t7ZuAc9v22rZPO356koysYknSggy1jj7JQcDNwPHA+4D/AzxYVbtalylgedteDtwDUFW7kjwEPA24f4R1S5phrvcs3HXp2UtciSbNUC/GVtWjVXUisAI4GThhtm7tcbbRe81sSLIhyZYkW3bu3DlsvZKkBVrQqpuqehC4ETgFODLJ7r8IVgDb2/YUsBKgHT8CeGCWa22sqjVVtWbZsmX7Vr0kaV7DrLpZluTItv1LwIuBrcAXgd9p3dYB17XtzW2fdvwLVbXHiF6StDSGmaM/DtjU5umfAFxTVZ9I8m3gI0neCnwDuLz1vxz4QJJtDEbyL1uEuiVJQ5o36KvqNuAFs7TfyWC+fmb7z4DzRlKdJGm/+c5YSeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1Dk/M1Yj4WfDSpPLEb0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnMsrpc75EYMy6LUgrpeXDjxO3UhS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ2bN+iTrEzyxSRbk3wryWta+1FJrk9yR3t8amtPkvck2ZbktiQnLfY3IUma2zAj+l3A66rqBOAU4MIkzwUuAm6oqtXADW0f4ExgdfvaAFw28qolSUObN+ir6t6q+nrb/gmwFVgOrAU2tW6bgHPb9lrgqhq4CTgyyXEjr1ySNJQF3aY4ySrgBcBXgGOr6l4Y/DJIckzrthy4Z9ppU63t3v0tVtLoeJ/6x4+hX4xNcjjwMeCPqurHe+s6S1vNcr0NSbYk2bJz585hy5AkLdBQQZ/kEAYhf3VV/UVrvm/3lEx73NHap4CV005fAWyfec2q2lhVa6pqzbJly/a1fknSPIZZdRPgcmBrVb1z2qHNwLq2vQ64blr7+W31zSnAQ7uneCRJS2+YOfpTgVcC30xyS2t7I3ApcE2S9cDdwHnt2KeAs4BtwCPABSOtWJK0IPMGfVV9mdnn3QFOn6V/ARfuZ12SpBHxnbGS1DmDXpI6t6B19Hr8mGuNtaQDjyN6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUud8w9TjmG+K0mz8QJL+OKKXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TO+c5YSUPZ2zupfdfsZHNEL0mdc0T/OOA9baTHN0f0ktQ5g16SOmfQS1Ln5g36JFck2ZHk9mltRyW5Pskd7fGprT1J3pNkW5Lbkpy0mMVLkuY3zIj+SuCMGW0XATdU1WrghrYPcCawun1tAC4bTZmSpH01b9BX1ZeAB2Y0rwU2te1NwLnT2q+qgZuAI5McN6piJUkLt69z9MdW1b0A7fGY1r4cuGdav6nWtockG5JsSbJl586d+1iGJGk+o15Hn1naaraOVbUR2AiwZs2aWftIOjD4ObOTbV9H9PftnpJpjzta+xSwclq/FcD2fS9PkrS/9jXoNwPr2vY64Lpp7ee31TenAA/tnuKRJI3HvFM3ST4M/CZwdJIp4E+BS4FrkqwH7gbOa90/BZwFbAMeAS5YhJolSQswb9BX1e/Ncej0WfoWcOH+FiVJGh1vatYRb16mSeOLtJPBWyBIUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzrm88gDkMkpJC+GIXpI6Z9BLUucMeknqnHP0kpact0ZYWo7oJalzBr0kdc6pG0kTwymdxeGIXpI654h+gvnGKEmjYNBPAANd0mIy6JeQgS5pHJyjl6TOOaKXdMBylc5wDPpF4BSNNFr+TO0fp24kqXMGvSR1zqkbSd1x7v6xHNFLUucWZUSf5Azg3cBBwPur6tLFeJ6l4gtBUh8eryP9kQd9koOA9wH/DJgCvpZkc1V9e9TPNWoGuvT41PsvgMUY0Z8MbKuqOwGSfARYCyxK0BvOkhbLQn8B7EseLcUvk8UI+uXAPdP2p4Bfm9kpyQZgQ9t9OMl3gaOB+xehplGyxtGwxtE5EOrsqsa8fXRPusBrzazxmcOctBhBn1naao+Gqo3AxsecmGypqjWLUNPIWONoWOPoHAh1WuNo7GuNi7HqZgpYOW1/BbB9EZ5HkjSExQj6rwGrkzwryaHAy4DNi/A8kqQhjHzqpqp2JflD4LMMlldeUVXfGvL0jfN3GTtrHA1rHJ0DoU5rHI19qjFVe0yfS5I64jtjJalzBr0kdW6igj7Jf05yW5JbknwuydPHXdNskrwjyXdarR9PcuS4a5opyXlJvpXk50kmaslYkjOSfDfJtiQXjbuemZJckWRHktvHXctckqxM8sUkW9t/59eMu6aZkhyW5KtJbm01vmXcNc0lyUFJvpHkE+OuZS5J7kryzZaPWxZy7kQFPfCOqvrVqjoR+ATwJ+MuaA7XA8+rql8FvgdcPOZ6ZnM78C+BL427kOmm3SLjTOC5wO8lee54q9rDlcAZ4y5iHruA11XVCcApwIUT+O/4t8BpVfV84ETgjCSnjLmmubwG2DruIobwoqo6caFr6Scq6Kvqx9N2n8Qsb7SaBFX1uara1XZvYvBegYlSVVur6rvjrmMW/3CLjKr6O2D3LTImRlV9CXhg3HXsTVXdW1Vfb9s/YRBSy8db1WPVwMNt95D2NXE/00lWAGcD7x93LYtlooIeIMnbktwDvJzJHdFP9yrg0+Mu4gAy2y0yJiqgDjRJVgEvAL4y3kr21KZEbgF2ANdX1cTVCPwZ8Hrg5+MuZB4FfC7Jze0WMkNb8qBP8vkkt8/ytRagqt5UVSuBq4E/XOr6hq2z9XkTgz+hr57UGifQULfI0HCSHA58DPijGX8RT4SqerRNxa4ATk7yvHHXNF2Sc4AdVXXzuGsZwqlVdRKDac8Lk/zGsCcu+SdMVdWLh+z6IeCTwJ8uYjlzmq/OJOuAc4DTa0xvRljAv+Uk8RYZI5LkEAYhf3VV/cW469mbqnowyY0MXvuYpBe5TwVekuQs4DDgKUk+WFWvGHNde6iq7e1xR5KPM5gGHeo1uImaukmyetruS4DvjKuWvWkfrPIG4CVV9ci46znAeIuMEUgS4HJga1W9c9z1zCbJst0r0pL8EvBiJuxnuqourqoVVbWKwf+LX5jEkE/ypCRP3r0N/BYL+IU5UUEPXNqmHm5j8I1M3JKx5r3Ak4Hr21KnPx93QTMl+RdJpoBfBz6Z5LPjrgkGt8hgMCX3WQYvIF6zgFtkLIkkHwb+GnhOkqkk68dd0yxOBV4JnNb+H7yljUonyXHAF9vP89cYzNFP7PLFCXcs8OUktwJfBT5ZVZ8Z9mRvgSBJnZu0Eb0kacQMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktS5/w/9te+8KK79PAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x111998a90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "workspace.RunOperatorOnce(op)\n",
    "temp = workspace.FetchBlob(\"Z\")\n",
    "pyplot.hist(temp.flatten(), bins=50)\n",
    "pyplot.title(\"Distribution of Z\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you see a bell shaped curve then it worked!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Nets\n",
    "\n",
    "Nets are essentially computation graphs. We keep the name `Net` for backward consistency (and also to pay tribute to neural nets). A Net is composed of multiple operators just like a program written as a sequence of commands. Let's take a look.\n",
    "\n",
    "When we talk about nets, we will also talk about BlobReference, which is an object that wraps around a string so we can do easy chaining of operators.\n",
    "\n",
    "Let's create a network that is essentially the equivalent of the following python math:\n",
    "```\n",
    "X = np.random.randn(2, 3)\n",
    "W = np.random.randn(5, 3)\n",
    "b = np.ones(5)\n",
    "Y = X * W^T + b\n",
    "```\n",
    "We'll show the progress step by step. Caffe2's `core.Net` is a wrapper class around a NetDef protocol buffer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When creating a network, its underlying protocol buffer is essentially empty other than the network name. Let's create the net and then show the proto content."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current network proto:\n",
      "\n",
      "name: \"my_first_net\"\n",
      "\n"
     ]
    }
   ],
   "source": [
    "net = core.Net(\"my_first_net\")\n",
    "print(\"Current network proto:\\n\\n{}\".format(net.Proto()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a blob called X, and use GaussianFill to fill it with some random data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New network proto:\n",
      "\n",
      "name: \"my_first_net\"\n",
      "op {\n",
      "  output: \"X\"\n",
      "  name: \"\"\n",
      "  type: \"GaussianFill\"\n",
      "  arg {\n",
      "    name: \"std\"\n",
      "    f: 1.0\n",
      "  }\n",
      "  arg {\n",
      "    name: \"run_once\"\n",
      "    i: 0\n",
      "  }\n",
      "  arg {\n",
      "    name: \"shape\"\n",
      "    ints: 2\n",
      "    ints: 3\n",
      "  }\n",
      "  arg {\n",
      "    name: \"mean\"\n",
      "    f: 0.0\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "X = net.GaussianFill([], [\"X\"], mean=0.0, std=1.0, shape=[2, 3], run_once=0)\n",
    "print(\"New network proto:\\n\\n{}\".format(net.Proto()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "You might have observed a few differences from the earlier `core.CreateOperator` call. Basically, when using a net, you can directly create an operator *and* add it to the net at the same time by calling `net.SomeOp` where SomeOp is a registered type string of an operator. This gets translated to\n",
    "```\n",
    "op = core.CreateOperator(\"SomeOp\", ...)\n",
    "net.Proto().op.append(op)\n",
    "```\n",
    "\n",
    "Also, you might be wondering what X is. X is a `BlobReference` which records two things:\n",
    "\n",
    "- The blob's name, which is accessed with `str(X)`\n",
    "\n",
    "- The net it got created from, which is recorded by the internal variable `_from_net`\n",
    "\n",
    "Let's verify it. Also, remember, we are not actually running anything yet, so X contains nothing but a symbol. Don't expect to get any numerical values out of it right now :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Type of X is: <class 'caffe2.python.core.BlobReference'>\n",
      "The blob name is: X\n"
     ]
    }
   ],
   "source": [
    "print(\"Type of X is: {}\".format(type(X)))\n",
    "print(\"The blob name is: {}\".format(str(X)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's continue to create W and b."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "W = net.GaussianFill([], [\"W\"], mean=0.0, std=1.0, shape=[5, 3], run_once=0)\n",
    "b = net.ConstantFill([], [\"b\"], shape=[5,], value=1.0, run_once=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, one simple code sugar: since the BlobReference objects know what net it is generated from, in addition to creating operators from net, you can also create operators from BlobReferences. Let's create the FC operator in this way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "Y = X.FC([W, b], [\"Y\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Under the hood, `X.FC(...)` simply delegates to `net.FC` by inserting `X` as the first input of the corresponding operator, so what we did above is equivalent to\n",
    "```\n",
    "Y = net.FC([X, W, b], [\"Y\"])\n",
    "```\n",
    "\n",
    "Let's take a look at the current network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current network proto:\n",
      "\n",
      "name: \"my_first_net\"\n",
      "op {\n",
      "  output: \"X\"\n",
      "  name: \"\"\n",
      "  type: \"GaussianFill\"\n",
      "  arg {\n",
      "    name: \"std\"\n",
      "    f: 1.0\n",
      "  }\n",
      "  arg {\n",
      "    name: \"run_once\"\n",
      "    i: 0\n",
      "  }\n",
      "  arg {\n",
      "    name: \"shape\"\n",
      "    ints: 2\n",
      "    ints: 3\n",
      "  }\n",
      "  arg {\n",
      "    name: \"mean\"\n",
      "    f: 0.0\n",
      "  }\n",
      "}\n",
      "op {\n",
      "  output: \"W\"\n",
      "  name: \"\"\n",
      "  type: \"GaussianFill\"\n",
      "  arg {\n",
      "    name: \"std\"\n",
      "    f: 1.0\n",
      "  }\n",
      "  arg {\n",
      "    name: \"run_once\"\n",
      "    i: 0\n",
      "  }\n",
      "  arg {\n",
      "    name: \"shape\"\n",
      "    ints: 5\n",
      "    ints: 3\n",
      "  }\n",
      "  arg {\n",
      "    name: \"mean\"\n",
      "    f: 0.0\n",
      "  }\n",
      "}\n",
      "op {\n",
      "  output: \"b\"\n",
      "  name: \"\"\n",
      "  type: \"ConstantFill\"\n",
      "  arg {\n",
      "    name: \"run_once\"\n",
      "    i: 0\n",
      "  }\n",
      "  arg {\n",
      "    name: \"shape\"\n",
      "    ints: 5\n",
      "  }\n",
      "  arg {\n",
      "    name: \"value\"\n",
      "    f: 1.0\n",
      "  }\n",
      "}\n",
      "op {\n",
      "  input: \"X\"\n",
      "  input: \"W\"\n",
      "  input: \"b\"\n",
      "  output: \"Y\"\n",
      "  name: \"\"\n",
      "  type: \"FC\"\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(\"Current network proto:\\n\\n{}\".format(net.Proto()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Too verbose huh? Let's try to visualize it as a graph. Caffe2 ships with a very minimal graph visualization tool for this purpose."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAADLCAYAAACS57LlAAAAAXNSR0IArs4c6QAAQABJREFUeAHtXQV4FFcXvSEJAQIEDxYIBEKQQBLcSnApRRvcrfy0SCkUKKVIsaJtkRZroUAFqADFiru7uwTXIEGT8L/zwiyTZXezm6zMJvfyhZ158+bJmd2ZO+fKc3n9+vUiYmEEGAFGgBFgBBgBRiCJIOAilJvXSWQuPA1GgBFgBBgBRoARYAQoBWPACDACjAAjwAgwAoxAUkKAlZukdDV5LowAI8AIMAKMACPAzA1/BxgBRoARYAQYAUYgaSHgpj+diJeR5Lugi34x7zMCmkMgd9rMdKzZVM2NyxEDguvctGnTaMCAAfT06VO7DyF9+vQ0efJk6tSpk9375g4ZAUaAEdBH4B3lRr8C7zMCWkUgKiZaq0Oz67guXLhAHTt2pB07dlC/fv0oJCTErv2js507d1LXrl1p0aJFNGvWLPLx8bH7GLhDRoARYAQUBFi5UZDgT0bAyRBQ2JqBAweSn58f7dmzh4KDgx0yi7CwMMIflKxixYrRpEmTqHPnzg4ZC3fKCDACjAA7FPN3gBFwQgTA1lStWpU+/fRT6tu3L+3bt89hio0CX/ny5enQoUOSwenWrRvVqVOHwsPDlcP8yQgwAoyA3RBg5cZuUHNHjEDiEQBbM3XqVCpevDg9ePBAsjUjRowgd3f3xDduhRZSpUpFEyZMoG3bttGlS5ckizNnzhwrtMxNMAKMACNgPgKs3JiPFddkBByKgBbZGmOAMItjDBkuZwQYAXsgwMqNPVDmPhiBRCAAtmbKlCmaZWuMTY1ZHGPIcDkjwAjYGgFWbmyNMLfPCCQCgfPnz0vfGvjVaMW3xtLpMItjKWJcnxFgBBKLACs3iUWQz2cEbICAwtaUKFFCk741lk6ZWRxLEeP6jAAjkBgEWLlJDHp8LiNgAwTA1oSGhkqmxlnZGmOwMItjDBkuZwQYAWsiwMqNNdHkthiBRCCgsDWIhIqIiNBcJFQiphbnVGZx4sDBO4wAI2ADBFi5sQGo3CQjYCkCarbms88+00TeGkvnYGl9ZnEsRYzrMwKMgLkIsHJjLlJcjxGwAQLJha0xBh2zOMaQ4XJGgBFIDAKs3CQGPT6XEUgEAsmRrTEGl8LiILMxZzc2hhKXMwKMgLkIsHJjLlJcjxGwIgJIyJfUfWsshQsszvjx4+NkN75165alzXB9RoARYASIlRv+EjACDkAADsNPnz6lP//80+FrQjlg+ia7BIuzcOFCevToET158sRkXT7ICDACjIAhBFi5MYQKlzECdkLAxcXFTj05VzeMi3NdLx4tI6A1BFi50doV4fEwAowAI8AIMAKMQKIQcEvU2QZOzpEmI5XInI8CM+clsSQOXXh0kw7evUARLyOpVNYCtPbqIQNnOabo46L16Hn0K5pzaq3VBlA1ZyBlSpUu3vb+Cz9Ij189I/0xuLm4UsXsAVTbJ4Q2Xj8q8TJUFm8HooKrSwoqmdWP9tw+a051i+tkT52RAjLmok3Xj1Emj3QUnCU/rb92WNdOyhRuYi6FqXhmX9p56xTtvX2OXot/itTPW5r+vbxX2eVPOyDw/Plz+vvvv9/pqXDhwhQUFCTLFy9eTFFRUbo6yJJcpEgR3T5vMAKMACOgdQSsxtzgATy8dCs60ux7qpKzGB27d4W23jhOmcWDfkW9r+hCq5myXEuAtPEPpZYFKlt1SEfuX6LSQombE9qTRpZpTR7iAQ8lA3/p3FNJBWB65e6U2zOz7Fd/DEUz+VDjfOWpR7F6BEURYqhMHjDxX3r31NQr8AM6cT/cRK3EHeoQUI3C/CrJRprmL0/tClXVNZglVXra03Qi+aTNQvPPbKL3hSLze81+5CL+KXL7WQR9V7GrxEYpc8bPcePGUa9evWjXrl2aHz6cdgsWLEi//fYbtWrVSv6lSZOGAgMDdWOvV68eYU4jR44kX19fWV93kDcYAUaAEXACBKzC3KRydZcKTEGvnFT732F04O553dR33z5Di89vp//qD6fUril15VrYqL58CMW8jrHqUO49f0y/n9tK3YvWFazVLfr13JZ32o8WfaZ0jYVefwyH712iWSf/ow4B1XXnGSrTHTSwAaVoUoXO9NHmafQk6rmBGtYpqparOP14fLVsrGquQFpz5aDchgIzv9qndOLBFfrlzEZZNnzfb3Qo7DsaWqoFDRPbEDBK6dzTSAXnk20zZJkz/oeIHqzajb9cuXJR+/btpdJQtGhRTU6nVKlStGTJEnrvvfdo9+7ddOrUKWrYsKFurNeuXZPMzYYNG8jb21tXzhuMACPACDgLAlZhbvqVaCzMHwVo0uF/4ig2CggwSQ3YNY9SuWlLuXka9UKapZRxWusT5iZTMvPEarr8+I6sYmgMUa+j5TG1CcdQmbE+RpdpK809j+IZh7HzzSn3SplGslAwSYGVqpyjqM4kBbNaefE37/QGXVMxwkb529kt1LVwLUrj5qErhxmrgFcOqp6rhK7M2Tbg/JoyZex3G4oBwpmLFStGAQEBNHbsWLp06ZLmpoTxgr1Jnz49DRs2jE6cOCHHiAildu3a0e+//86KjeauGg+IEWAEzEUg0cwN/C5g/nge9ZKmH19ltF/4j9x9/ijOcb/02al0toJUNGMeAsOj+F94p85AH/iWIfcUrrTx2lE6FXGVKmcvQsWEHw9k+aU9dDXynq6tSsKvI1D4dUTHxNCZh9eFD8hRs47BdFJH+LYsOLtJVx8sVKUcRaTfEBiWPwQLc+PpA93xXJ6Z6IO8ZWjGiTUUkCEX1ctbiq4+uUuLBDulVkZ0J+hthOWvSIsvbNeVGhqD7mACNkKy+FEtn2DquW3mO2endUtFNX2CqJAY9zWB34ZrR8TnfV09KCkwKULhOv/wpjAllSTfdN60XPjF7L9zTtbzF+wc6oCle/jiKcEcBaYImXbr5ikpscL1gOibxE4+CCdPYZqrlTuI/rm0W9bBfz+I782wUi3leMzBUHeiRjdevXolR3b69GkaMmQIDRo0iEqXLi0ZnbCwMMqWLZsmRp4vXz7JNoFpatu2Le3YsYPatGkjx6tV1kkTwPEgGAFGQPMIJJq5gRMpTCzhkXfpZcxbJ0RDMz96/7Ku+H/CbPOt8LeACWfWyTU0SrANnQJqyOO3hC/G3WcPaUzZdkL5KSDLtt48IUwYqWUZHqyKDCnZnPILJQkPSJg5hpRsphwS24aPpRBv2q0KvEcHw76lr0o119X3FIzCgQ+/lYra5CNLCX5Ea4Q5DQoPBIrQ5gZjaGy59sLsVIc+Lva+8K8pSDOqfEx9ijfQtWNsA4xF/6DG8rCxMRg719zy3sU/EI67Z98xRxXLlEfOJSomWpq9vFJ60u4mE6nFG5+jnGky0c9Ve9NftQcJZbU+Ta3cjYplyiuPr3l/GDUQCh0kUpi5oKTkTZeVVocfoLMPb5CfYF7WXz0sy89hP30OWffms7dKIQruvFFuUV8tu26dlg7owDepieKYu2/fPurduzdlz56datSoQcuXL9fEVMHSNG7cmA4cOCDz7UAJU5uoNDFIHgQjwAgwAhYikGjlJiBjbtnl5ce3DXZdRZgrJpTvKP0q4Dw6KPhDghIB88Qp8ZCEXBHMx1HhiKt+uJ2KuPZOe0eEP4q+dChUTUZkofzQvQu08sp+XRVjx2AigS8MWCG11MtTirKnyUCnI64LX5zX4uG9n/KkzUqFM/rIaniYz3/jQ3Li/hWCn0iLdePp0N2L1FAwTfoCRmppncHyb1W9oXSqxXTRfqyTsLEx6Ldh6X4x0aeaacL5YMB+Cu0lmTGwMPALmnpsBa0SWH1fsZtkcq4/vU9f7V0ou3sZHUXN146nfjt/psr/DKKIF5E0plw7aX4C07Pt5kmpUP56drNkyeAcveTCDlkOli1rai/Jor0SipRawAhBsgtmTi1QZiNePKGgLPnUxfFu35q3jWAScvTfL7/8Eu9YwWxFR0dLhmv9+vXSFIST+vfvT/fuvWUh423IBhVmzpwp2aSTJ09SlSpVbNADN8kIMAKMgH0RSLRZ6qUIpYakECYNQ7JZRExdfHxLRFFNoVeC2Snw60fi7f8Fvb9yhDR/4ByYSfCABDNjqYA5AOPQe/ssqdhMOfqvrglTx1BJGbtyAh7Qh+9dFAzDQ/IQbE1FYQqDwHyGcHbIs+iX8hPmL0VOiwe6IZ+R48KhtuHqUUo1yiDYkg0NRur2saE/hjgHLdyBEqOYkdSn1sgVRP4C47134oaEw98lzK8itfWvSl/uWaC7HmolEljMO7OBPivRSLA12aQiiWsFpgftwfcmMJOvjIxT+gS7Y0hg9oLcEqycvjx8+Ux+D/TLTe17VQmgmZ2+MlXFLsf++OMPs5gYtRIGhgTRVcOHD6fMmTPbZZzGOrly5QplyJCBbt++TR06dKDDhw9TunTpjFXnckaAEWAENI9AopUbKAMQmIaMCZgZRCXBj+Phy6eyGtgF5ISpkyeEtt84SRdFZFGQMHFZKv0FuzCvWh/6tUY/mW+l66apUjlBO6aOGeoH/h63xYP3i+Aw4Wj8UjhHxyo0xhQ3pQ345ggCIV6BY/Wkw0vjrZfQChk90pJrihT0TPg/qaWQyEUDiXwVV+nYefOULC/kFXtc7hj4D6YmCK5XT2GKA5uFPsaV60DegukCVkgDcFwwcbNFpBf8eTAO5LlRmyrTvlFeoQzqCxSinMKfyRJJ5ZuF4MPiaEHEkSlxc3OT0UflypWTzroY8+XLl6lkyZKEMGxHChSaTp060cqVK2n06NH0008/SfMZPlkYAUaAEXBWBAzTLRbMBm/5eGjmEflM8oqHnjGBGUbtLDo4JEz4nzShoXt/pWWX9xAUhIQI/HjeE6YTPFThdLyl0WjJkKAtU8cM9YXxb200lvbfPUeThM9N+JPYiCZDdRNapnZeTmgbxs6DYgYTUlrhtKsWmHwgZbL5q4ulORBsGpQuU4JcNZB1IgHj6ANL6EXMK5pxcrXchlMxTHUo/11EQ0FOvzEp5nqTy0cWiv+Q8whiyOQIVuvaE8eaZ+TgrPSfu3usnxYcc8eMGUPh4eHSYbd79+4OZ2qUKcLxuVmzZlKp8fPzo8mTJ1OePHno559/NpjoTzmPPxkBRoAR0DoCiVZuEG7cd8ccaZYaVbatWfOFEgHFZtH5rbpQbH12BI6vEA8TuXHADDQXSeSQywX+IWFrvxH+HBmpgfB/MXXM2CAHCn8gmHbWiOzBEP0xGTtPS+WKz4t6TPveRDpVEOHZaikifIncBYZ7RKSaKXlP+E0dEizWZaHswUxVJqs/rby8X26X8y4kfXdQruTUgbLzQpgrcUwtQSJzNZRhhQlSjiEvTjbhpwPzpbMKHIcVhSZv3rwy4gj5Y44dO0b9+vWj3LljfdO0NL+ePXtS1apVCUn7IAgLnzVrltzu1q0b3bx5U27zf4wAI8AIOBsCiVZuMOE/zm+j6cdWEtLpf1uhyzvJ+qS5RPhbKOwMwoEhTfJXkH425cVDEA/eDB6e0tkYIcvnHt2QuWCaimy9Pp5ZZOhxo3zl5HklRNg3HojwYegUUFOW4T+ENsNZFn+mjiknpBR+NelFEjnFFySNu4d0+K0pQpWxnECXwrFtI9QZviUQxS8IypMiYCTQliKIRILAfBOf6I8B9TEmiKfAQRFDZcox9SdMTVBa1HJMOD/D+beCCNFWMiPjOJSP8w9v0NzT69XVRUbkPLp9zD1ELOEAhg1SXPjXxIh/8CfyFT44UEoQxq8WMEgzRag8oq4UgQ8TQsV7CidsNYOH4zk9M5KbUCrVzuDKec7wCWdh+M188skntHfvXpnXBr40hQrFVe60NBewNHv27JGh6upx1apVi+rWrUt3796l1q1b08uXcU2c6rq8zQgwAoyAVhFwFQm8hqkHh7WWvj2yTF1k1vZ6oVggoRvS8X8mkvqVEflrsMZUy4Lv0UdFass1kuC0eu/FY/HG/4hgsqgnHnZNRJ6U82L9qWUid82HIgdMWW9/uY1xREY9o2Z+lambOB8RNngIh4pMuFeF+eKSeMsHazQwuKlMJgdlB/lT8ND96fQ6+bA0dgyh3VCKWomxpRNKCx68UADOiiip0FzFqJ1wsEU+l1EHFsm1keoKv6Bw4TeUXtT9VIR8ZxC+LQjrRiZmrAGFyC8cE884Qt6afsL5No8IlYZCBCYJkWT6OX6MjSEgQ276IuRD6cOEqKOrIsQ+Q8q075Qh+7EhgQ9U3xINRW6aPXHMTeuE83C2VF7UT4SiI2opWEQmQdnosPE7US/WDwpKZ0+hkNwXymE57wAZho/Q9S92z6f/3qwJhtBxmCGRk6ipUE6B3e+CgdMX5DWCOat7kbpSUawjcu/gGuN7oi/tRcQb8Bx36C/9Qyb3YX77pNhbBcpkZRsexNpLQ4cOpTp16lDOnDnN6unGjRuEKCUs25Apk2W+RmZ1YKQSnIcRko5Eg1Bc0qZNS8hYjJcByNy5cwnRXGBtLl68SFu2bJFZl2G2sqc4Ch97zpH7YgQYAdsh4CLeOsUj+a3A/8J3QZe3BQnYQn4YP6/sYl0ldzr98Jo0URhqBgyNYsrAcX0HVJTh4eku2kM9tAv2R/3mD9YFOWOQ+E+d2A/nmjqG44YESlJqkUlZCVtGHZiq9MOaDZ2rlbIOhaqLxIg+1H/X3HeGhDWnEL4PBRHh32oBC3Om5Y80Yt/vMm8Q9mGKSozg2mT2SK9z8jbU1kYRQfb5znnvRHMZqqsuQ9j+qRY/qIucZht5ZeBQfO7cObK34uAMIDE+znCVeIyMgHYRsIpZSn96WCoATqVYRBK+F8ZErdigjjqyRjkH5yv10K5asUEdKDtQPPQVm/iOKe3rf6J9tWKD486k2GC888SyB1iZHCYkfQHbhWSH+oqNfj2EvCdWsUGbcCSHP44xwVIRiCDTD1M3Vp/LGQFGgBFgBBiB+BCwiXITX6d83LYIQEH735YfqFPhGtJkZ25vyppPXsL3yR7SRyzbcUiY0ZBYkIURYAQYAUaAEbAWAqzcWAtJjbUDFqzP9tl0x0DCPENDRSg/skdDGoqlFloXrCLNcYbqWqsMjuiLxF9yFMXHBX43ylpUyREHQ3N+/vw5zZkzx9AhLmMEGAFGwCwEbOJzY1bPXElTCMCvSGFulIEpCReVfa19OrPPDVzdpkyZIkPGCxQoIB15g4ODtQax3cezc+dO6tixI8GheNKkSdS5c2e7j4E7ZAQYAedHgJkb57+GVpkB/IqgzKj/rNIwN2IQATA3iJQ6cuSIXPqgTJkyMuIqubI4YGuwzlalSpXI19dX5gdixcbgV4cLGQFGwAwEWLkxAySuwgjYCgFESm3atIkmTpxIEyZMkGHZBw/GJpG0VZ9aaxdsTVBQkAyNnzFjBq1evZp8fOLmatLamHk8jAAjoG0EWLnR9vXh0SUDBJIri2OIrenSJXFpKJLB14WnyAgwAmYgwMqNGSBxFUbAHggkJxYHbE2xYsVo6tSpNH36dGZr7PEF4z4YgWSEACs3yehi81S1j0BSZ3HUbE26dOkoOjpamuSWLFmi/YvDI2QEGAGnQYCVG6e5VDzQ5IRAUmRx9H1r4Ft0+vRpgjM1VicvW7as9D9KTteZ58oIMAK2QYCVG9vgyq0yAolGIKmwOGq2RomEUnxr8uXLRwsWLCAst5AhQwa5Svn7778vo8gSDSA3wAgwAskWAVZuku2l54k7CwLOzOLoszXGIqEQLbVmzRpat24d3bp1i5Dzp3379oSFPlkYAUaAEbAUAVZuLEWM6zMCDkDA2VgcU2yNKfiqV69Oe/fupYULF9L27dvJ39+f+vXrR/fvx13k1VQbfIwRYAQYAVZu+DvACDgRAgqLg+y9Ws2LYy5bYwx2KHItWrSgkydPyjnOnz9frpw+duxYevbsmbHTuJwRYAQYAR0CrNzooOANRsA5EMDDv2fPnprLbpxQtsYY6u7u7vTJJ5/QuXPnZDbnkSNHUsGCBeW6U4iyYmEEGAFGwBgCrNwYQ4bLGQGNI6AlFiexbI0pqBEyPnz4cDp//jw1bNiQunfvToGBgbR06VJTp/ExRoARSMYIsHKTjC++s089hQt/ffVZnJCQEEKZvf8qVKhAvm/WhFIioaz9/fL29qZp06bRiRMnZALARo0aybWo4JvDwggwAoyAGoF3VgXHAoorruxT1+FtRkCTCHi6eVDN3EGaHJsjBoWVxletWkWRkZF2797Ly4tq1apl137heDxgwADauHGjZHTGjBlDhQsXtusYuDNGgBHQJgLvKDfaHCaPihFgBBgBwwhAoRs4cCAdP36cOnbsSMOGDaNcuXIZrsyljAAjkCwQYF4/WVxmniQjkHQRqFu3LiHb8c8//0xr166VTseDBg2ihw8fJt1J88wYAUbAJALM3JiEhw8yAoyAMyHw4sUL6ZczatQoOezBgwfTxx9/TB4eHs40DR4rI8AIJBIBVm4SCSCfzggwAtpDAKwN8uJ89913lC1bNhoxYgS1adOGUqRgslp7V4tHxAhYHwH+pVsfU26REWAEHIwAHJzhYIwcOTVr1qROnTrJJR3gn8PCCDACSR8BVm6S/jXmGTICyRaBnDlz0qxZs+jo0aOERTrr1asnF+dEpBULI8AIJF0EWLlJuteWZ8YIMAJvEECI+D///CPXq3r16hWVKVOGmjVrRmfPnmWMGAFGIAkiwMpNEryoPCVGgBEwjACSDW7btk1mN0boeJEiRahHjx5yJXLDZ3ApI8AIOCMCrNw441XjMTMCjECiEGjQoIFcm+vHH3+kZcuWyYU5hw4dSo8fP05Uu3wyI8AIaAMBjpbSxnXgUTACjICDEMBK44iq+uabbwiLdQ4ZMkSuX4VtFkaAEXBOBFi5cc7rxqNmBBgBKyNw//59Gj16NE2dOlVmOEaunObNm8t1uqzcFTfHCDACNkaAzVI2BpibZwQYAedAIFOmTDRhwgQ6c+aMXJCzdevWVLp0aVq3bl28E/jiiy/owoUL8dbjCowAI2AfBFi5sQ/O3AsjwAg4CQJ58uShefPm0aFDhwgrkSNPTu3ateUSD4amAMdkJAysVq0a3b5921AVLmMEGAE7I8DKjZ0B5+4YAUbAORAIDAykFStWyFXHIyIiqGTJkgQ25+LFi3Em0L9/f3J1daVr165RjRo12Ck5Djq8wwg4BgFWbhyDO/fKCDACToJAaGgo7d69mxYtWkRI/hcQEEB9+vShu3fv0tatWwlZj6OiouTfyZMnqX79+oQ1rlgYAUbAcQiwQ7HjsOeeGQFGwMkQgBKDjMdYq+rp06dy3SowOdHR0bqZgMVBqPmSJUt4LSsdKrzBCNgXAVZu7Is398YIMAJJAIHIyEjq0qUL/f777wZngwU6u3btSsijw8IIMAL2R4DNUvbHnHtkBBgBJ0cgVapUtGfPHqPMTExMDM2cOZOGDRvm5DPl4TMCzokAKzfOed141IwAI+BABObMmSMdi6HEGJPXr1/T8OHDafr06caqcDkjwAjYCAE2S9kIWG6WEWAEkiYC8LXJmzevdCg2Z4YuLi70xx9/UFhYmDnVuQ4jwAhYAQFmbqwAIjfBCDACyQcBMDGIlIK4ubnJJRtMzR4MTsuWLWn9+vWmqvExRoARsCICzNxYEUxuihFgBJIHAshpg7Bv5e/IkSN04sQJevDggQQADsVQfF6+fKkDJHXq1HJF8pCQEF0ZbzACjIBtEHhHuYl4GUm+C7rYpjdulRGwIgK502amY82mWrFF520K7MC0adNowIABMkTZ3jNJnz49TZ48mTp16mTvrjXVH5QbKDybDuyk4f/OpphrERR99SG9jngmx+mSzoM8B9UgV+90mhp3Uh9M7TzB9EeNz5P6NM2a340bN+TCsMuWLTOrvrUrlS9fnn7++WcqVKiQtZuO055bnD3eYQScCIGomLe5RZxo2FYfKtY06tixI+3YsYP69etHjmAGdu7cKUOfkegOeWB8fHysPk9naDBjxoxUoUIFylLUl8al2acb8uuXURRz6zFFX39E0WfvsHKjQ8Y+GzFC+WchWrBgAfXq1YvwPYWC4enpaVdYkCdq4sSJFBQURF9//TX17dvXaMRhYgfGyk1iEeTzGQEHIaCwNQMHDiQ/Pz8ZmhwcHOyQ0cBZFn9QsooVK0aTJk2izp07O2QsWuzUJaUbufpklH9aHB+PKWkjoLA1y5cvp48//liuhWZvxUZBGPeJb775hgYPHkx//fWXzVgcdihWEOdPRsCJEABbU7VqVfr000/l28++ffvIUYqNAhvoZiw2ieR13bp1ozp16lB4eLhymD8ZAUbAAQiArSlatCgdO3ZMrpM2ZcoUuzM26mnDFw2Kzf79+6VPGlgcsDmm0iqozzd3m5Ubc5HieoyABhAAWzN16lQqXry4dF5FIjksBeDu7q6B0REhud2ECROk4+ylS5cki4OcMCyMACNgXwTA1jRs2JDatWsnF3yF03uVKlXsOwgTvYHh3bVrF3355Zf0xRdfUKVKlej06dMmzrDsECs3luHFtRkBhyGgRbbGGBjM4hhDhssZAdsjoDW2xtiMbcnisHJjDHUuZwQ0ggDYGlDJWmVrjMHELI4xZLicEbANAlpna4zN2hYsDis3xtDmckZAAwicP39e+tYgqgB/WvCtsRQWZnEsRYzrMwKWI+AsbI2xmRlicWDiTqgvDis3xpDmckbAgQgobE2JEiU06VtjKTTM4liKGNdnBMxDwFnZGmOzU7M4cDxOqC8OKzfGEOZyRsBBCICtCQ0NlUyNs7I1xqBjFscYMlzOCFiOgLOzNcZmbA0Wh5UbY+hyOSNgZwQUtga+NRERETJvjZYioawFB7M41kKS20muCKjZmjZt2pDWIqGsdV0Sw+KwcmOtq8DtMAKJQEDN1nz22WdO6Vtj6fSZxbEUMa7PCMRmGVbnrfn+++8dmrfG1tckoSwOKze2vjLcPiNgAoHkwtYYg4BZHGPIcDkjEBcBsDUNGjSQeWuSMlsTd9Zv9yxlcVi5eYsdbzECdkUgObI1xgBWWBxkNubsxsZQ4vLkioDiW4OV5zdu3EhJna0xdp0tYXFYuTGGIpczAjZEAAn5krpvjaXwgcUZP358nOzGt27dsrQZrs8IJCkE4HeHLMNgaw4fPqypLMOOAlqfxenQocM7Q+GFM9+BhAsYAdsjAIfhp0+f0p9//kkFChSwfYdO1ANYnIULF1KpUqXoyZMn5O3t7USj56EyAtZF4Nq1azLXFdgalrcIKCzO5cuXCS+L+sLMjT4ivM8I2BEBFxcXO/bmPF0xLs5zrXiktkeAfw/GMTaGDSs3xjHjI4wAI8AIMAKMACPghAhY3SyVI01GKpE5HwVmzktiSRy68OgmHbx7gSJeRlKprAVo7dVDmoHp46L16Hn0K5pzaq3VxlQ1ZyBlSpUu3vb+Cz9Ij189I/0xuLm4UsXsAVTbJ4Q2Xj8q8TJUFm8HooKrSwoqmdWP9tw+a051i+tkT52RAjLmok3Xj1Emj3QUnCU/rb92+J12sqX2In+vnLTt5sk4x+rnLU3/Xt4bp4x3bIPA2bNnZXi5uvX3339f5scIDw9XF1PlypUpd+7csmz16tUyQzJ20qRJI1cZjlOZd6yCgKX3DaXTlCncxL3WlwIz5aG8abNReORdOhNxjfbdOUcf5C1Diy9sV6qa/AzKnJ9OR1ylZ9EvTdZL6MH47hXv5ShKNXMH0a1nEfTnhR104+kDXVclxPzuPX9MVyPv6cp4w/YIYKkX3DfUki9fPipXrpyu6I8//oizPEKjRo0oderUuuOO3LAac4MH8PDSrehIs++pSs5idOzeFdp64zhlFg/6FfW+ogutZspyR05Wv+82/qHUskBl/eJE7R+5f4lKCyVuTmhPGlmmNXmImw+UDPylc08lFYDplbtTbs/Msh/9MRTN5EON85WnHsXqERRFiKEyecDEf+ndU1OvwA/oxP24Dy4Tp1h8qENANQrzqyTPa5q/PLUrVDVOG7j2X5duTYfDvicoMvpyW9zIvqvYVWKjf8yZ9seNG0e9evWiXbt2aXbYOXPmlIpMq1atqG3bthQUFETp06eXfi2wV6McTotwcs6VK5duHhUrVqRZs2bR0KFD5Tm6A8loo1mzZjR27FiCbd9WYul9A+MIyeJHOxqPo3Hl2os9F1p5ZT89filemIq9TzfazaNJFTubNdw64kUKv1VbKTYYhKl7RR9xnxor5pBO3LN6FqtPx5tPpVq5g3VjP3b/CvUt0ZAqeAfoypxxY/PmzYTv0j///EMvXrzQ/BSgyBw6dEjeG3B/uHjxorxfqAder149Gj16NI0ZM4b8/f01o9hgjFZRblK5utOa+sOoY6HqVPvfYTRo9y+0Knw/7b59hmad/I8q/P05nX14nVK7plTj4vDt6suHUP1VX1t1HHjD+P3cVtnmhUe36NdzW2jR+W3yb+7pDfTlngX044nVlNI1ljTTH8Phe5ckZupBGSpTH9ffhlI0o8rHNEdg/yTquf5hq+1Xy1Wc1l+NZWqq5gqkDVePxGk7T9qsEovUboavOxilZZf2SAUnzolOtoOIHqzaDUdYMB5YD+X48eOamoWnp6e8AUGpiY6OFqyqoFWFIEIJ4w0ICKCoqChKmTIlqW3Y6dKlo7x589J3330nPzU1KTsNBm+wgwYNIl9fXypbtixNmzaN7ty5Y9XeLb1vfJi/Av1Xf7hkxeuuGE5zT6+nvXfOSqam5boJNOnIUkrj6hHvGMEc5xIvWoYY13hPtqCCsXuFb7psdPnJHfmM6LNjNoUs6SMY7efUo2hdXevRr2Oo386f6dPiDalIRh9dubNtPH/+nBYvXkyNGzemzJkzU6dOnWjdunXy96jFuWCM33zzDX300UdyeNu2bSM48arl4cOHMjDiv//+o5IlS6oPOXzbKspNvxKNhfmjAE06/A8duHv+nUnBJDVg1zxKZeQh984Jdip4GvVCmqWs3R3MTaZkplBuLj+OvTkaGkPU62h5+muKfQBhx1CZsT5Gl2krzT2P4hmHsfPNKfdKmUayUDBJgZWqLGhl/RskzJFnHl4z2RzOKeCVg6rnKmGynpYPQhmAUgBBZAPCmRGqCIUBb/yXLl2Sx7TwX48ePeQwfv755zjD6dKli9z/6aef4pRD4dm/fz/VqlUrTnly2lEUQcx579691Lt3b8qePTvVrFmTfvnlF3r8+LFV4DD3vpElVXqaUL6jNGv33TGHXsZEvdP/mANL6NrTewSzlTEpnCE3dS1cy6pmeUN9mbpXgPH/++JbxjNS3JNhqta/d8UIZXza8RVO/yKk4BMZGUnIXYPvEKIB+/TpQ7t371YOa+pz8uTJ8l62atUqmjdvXpyxffzxxzLnDn4PWpNEKzewpcL88TzqJU0/vsro/OA/8qPecb/02amFMAvBdKE2W3inzkBdxI/uf0J7DxA/QEjl7EXkPsoUk47SWaXsheWxboVrU6jweVGLqWO4SbQpGKquTmChauQuQZ+VaER9ijfQmYaUSrk8M1H3InUECexCuDmgXnNhmsG+ORKWvyJdi7wvfZBQ39AYzGnHWB1Q1bV8gmnpxXd/KGndUgmTVzkaGNyU2vqHije2THGagZKCN6xy3oUoayov6lCoGg0r1VIqrkpF+M7ghjg4pBk9fPGUYI4aHBImmYC6eUrGuY7KOfF9/iC+F+jHXAzja8/Rx1+9eiWHcPr0aRoyZAiB3i1Tpox84799+7ZDh9eiRQvpOzN//nzJ1CiDgSIGwcMazI4i8LmpWrUqubq6KkXJ+hOKDvCJiYmhDRs2UMeOHeVbeNOmTenvv/+2mblBfd/oH9SYMnikpdmCmTWmEOFlaNCuXyiFiWg8uBEY88mBnwvuc2BLcE/Ql5xpMlHngJqyGPfYoSVbyPsC7p+KmHOvOPfohlJdfuIekC+dN007tiJOOXbwIpVWmPY/MGDifqeyExQo94l79+7R9OnTpS+Lj48Pffnll4RkfVoR+NBAEQNrAyXs+vXrcmhgoWDahu+eFiXRyg2cSGFigSOboTcI9aSP3n9rs4aS8q3wt4AJZ9bJNTRKsA2dAmrI6nAqu/vsIY0p245KZysgy7bePCFtsigrKB6wigwp2ZzyCyUJD0iYOYaUbKYcEtuGj+EH36rAe3Qw7Fv6qlRzXX1PNw868OG3UlGbLGhdvFWsEdSv8oOFbXpzgzHSPty9aB1p2y6dtaA0AUERik/SiPZxY4IYG0N8bcR3vHfxD2ivwEHfHFVMOBxiLlEx0dLs5ZXSk3Y3mSiVS7SJm9XPVXvTX7UHCWW1Pk2t3I2KZcorj695fxg1EM6JkEhh5jr5IJzypstKq8MPCHPjDfITzAvMUyg/J/YtlV23TksHdOCb1ATMBwSmDeWNv0aNGrR8+XKHTBVmJjyIYUr7999/dWPAzSswMFDeuPCGpgiUHSQPY3kXASg4+MNDatmyZdSkSROp6EDh2bFpC70Wx6wh6vsG2kNgBuTovbf3U1mg99+KK/uMMtN4McNL0Lo3ZmX1qaPKtKE+gQ3k7xu/6xFCCVpedwhlFAoVBIrWjsbfCJ/CNjSpQidqLl5Qi4r7y3jBJsG/EvdNiKX3CpjTZwpz+h7hzgCXBkOy+9YZ8UIZew81dNxZyxRF5+rVqwQfPqwdVbhwYWkWshYzmBhsYHKC+Rr5uWCmwid8bWCu1qokWrkJyBjLrFx+bPiNtIowV4BCheMo/gYFf0hQIvD2f0o8DCFXntylo8IRV/1wOyU8/vXliPBH0RewC4jIghy6d0E61Sl1jB0DxQlfmI3XjipV5We9PKUoe5oMImrgOqHOauE3BL+Rwm/svHiYzz+zUdY9IZzcPtk2g1qsG0+H7l6khr6xD391g0Uz5qGldQbLv1X1htKpFtNF+xllFWNjUJ+fkO1iok91pAHacE/hSj+F9pJ073JB+cK+P1W8Ga0SDojfV+xGhTLkoutP79NXexfKLl9GR1HzteOlnbvyP4Mo4kUkjSnXTpqfwDoh6gkK5a9nN4u3qaOSSVsiIhxQfkpEXFgqUGYjXjyhoCz5LDr11rxt0j8EZiFH/kEBiE+UN358rl+/noYNGyZP6d+/P+HNzZ4Cx2GIYoLCjQos04QJEwyWly79rjO4rGiH/5Dg0JHXFn2bc30UJRbmhrlz51LHRi3pcd+l9GLVSaHkvDUvmwOZqfsGzi/0hs2+/MTwPdesPoQyArmpikrCPpj0tv5Vqff2WXRJ3NPh6Nx+w7fC7FyExooXSwjYnjUi2hMvfTNP/Ec9t82kZmuFY/PBvyTLiyAJiCX3ilARhLKs7pciQKGiZMyh5BiSk+L+ghc13NMskf3f/eXw71GdOnXMGrKi6Jw6dYoGDhxIv/32mzQNHzx40KzzbVUJjFJISIh8KXrvvfckw5QpU1z231Z9J6Rdt4ScpD7npQilhqQQJg1DsllETF18fEtEUU2hV8I2XODXj4RG/4LeXzmC4G8CwcMVpiZ4y1sqYA7AOODHiGiBKUffvo2aOoZ+lLErfeIBffjeRbrz/CF5iB9uRWEKg8B8Bv8RiBJRcEY4SCuCEEpDPiPHH1yhhqtHKdUog2BLNjQYqdvHhv4Y4hy0cAc/eF9B6UKBUUuNXEHkLzCGw6Fa4O+CmwluZnB0Vq6HWokEFvPObJDmt7zC+Q+KJK4VmB60B3t6YCZfGRmnbtvS7YciygPfA0vEq0oAzez0lSWn2KQuwiHNYWLUD2koDIiuGj58uHzbt8nAjDRarVo1GREFhgaL8S1dupTCwsIIjBKch8HowHwGMwtYHkcKsrI62p4PKt6ct2fQ9lBy0qZNS3WbNKCV3jfI1T8buaQwz2St4BzffQMOthBj91ylHVOfym8NLxZqAaOO4A+1z8t58ZuHogOGBo69MIXhXgHTl/plBmw3opoqCjMVHJwh5t4rYHIq/edn4mUyCy2o/hk1E6b+Jed30H9X4z7QH718Sm7iPoeXq9MGXoDVc1Fv569Xlr5p309dZPdtRB6B7TBH3N3dJSMI3z18n2AWDg4ONudUm9XB9xvO9AicwHgcfW+Ib6KJVm6gDEDwZTMmYGZixA/y/MOb9FB8OSFgF5DboU6eENp+4yRdFJFFQcLEZan0Fz+2edX60K81+kmbbNdNU6VygnZMHTPUDxx4bwtz2BfBYYLOfSmco2MVmvhuIrjZmDBt67qCY/Wkw0t1+9beAG3smiIFPRP+T2opJHLRQCJFFIJadt48JXcLeZlWKhRTE65XTxFmCjYLfYwr14G8BdMFrGC/Py6YOPgBJERAYefU8wGKr51UvlnkQzm+erY+Hp8joPLQQ34IsCZQJBBWDKoXuWPsLSnEdwSmJkRCgHVCaCoUNJTDpAJWSSk3h5Wy5fgRaurn52fLLuJt+/PPPzdaBzd5mKbgUN6wYUOJa+3atenSs7v0n3hYW0P07xt4mSqTzT/OS5el/cDXDywi8nypBfcCQyYh3CsQ2QSXAENBI2gDL35ga7KIsHIoOIjosvRegWdF181Thcl8gnRJ0FdulHsYXq4sUW4yFsxFYTXD1FO1+zb8U0wpN4pC4+vrS+3bt5ch2AivhhkIi+xqQZSlULSSy8YUJobpFlNn6B3DWz6+cNC484qHnjGBGUYd/QMn1P5BTWjo3l9p2eU9pLyNGDvfWDn8eN4TphM8VOF0vKXRaMmQoL6pY4baw/i3NhpL+++ek6GU4SJE0dqy4Owmazepaw+KGUxIcLpTC0w+ENwQ1YIbCdg03DxNiY+4tpB1IgHjaBGF8SLmFc04uVpuw6kYpjqU/352i6lmTB4Dq3XtiX3NMyYHlMiDuFFBYDtHDggkytuxYwd1797d7kyNoakg1w1k0qRJMhw8T548ch/KDZQcREhA8ue3/IVDnpiE/wM+UGrwh2gX+CvBdAUF8YMPPtBFzlkTAvV9Y9uNE7LpqgYcfc3tE8wzmES4CKgF94IQkfhT3xEZ7A3E1L0CkVneImEnWB4w3Qm9V0BpuSHM5LfE/UxfMnh4yqJrSSShn3KfyJo1K/Xs2VP65iGfDF4woNiwJByBRCs3oC8Rjgh2Y1TZ2BtmfMOBEgHFZtH5rbo3B312BI6vEA8TuXHwY0KkEpxnQZeGrf2GEL3VQPi/mDpmbHwDhT8QTDuwJ0P0x2TsPC2VgybOKm4wakG2UkgFkflYLcgZ4S4whAOfKUH20EPiZoV8FDBTlcnqTysv75fbiKyC7w7K9Z2YTbWpPoYICWQxhvnSWQXmCOVGBdMO8qLAZn7s2DHq16+fLuOvVuYHpQsOxDA/KYoOxgYlB+YpREQg4RjLWwTAwEHAwIGeh1M2THtIcIY8QvYSsL94+CMBKfxPjAleOPHSYEjg/A8xdK+Ae0DxTHH93xA9dUcoG5dM/EbLZCso033ANxFmq4TeK5BQEAEPG64deWfoYIrBOOFe5KyifI/g3N+hQwfauHEj3bx5kyZOnKi5XDHOijHGnWjlBo38IZLUTT+2UoYBf1uhyzvJ+qS5RCg/Cjvj+YZZaCJoS/yQyosHJB680MrxJoGQZYQIIhdMU5Gt18czi6RDG4kwZgh+aHgg4s2j05twRJTjxwBnWfyZOoa6kJTCrya9exrpKIv9NO4e0uEXacCxnECXwjVRLMPB4VsCUfyC1Pkj8GNEW4rghwkBJRuf6I8B9TEmiKfAQRFDZcox9SfoY/1EV8jwCeffCoIqVofRQzE5L3yWFPu40g4iHxRBBAPe5MCwQYoL/xoRIyJMUFckTQ2lxBCNrZyfIWVshEUqE0pqTs+M0o4OnylnFNxskfDqk08+kXlQkNcGvjSFChXS9HQQFg56+cMPP4wzTlDi+P1o3aYeZ9A23IHSCt8H5CwCA7d9+3ZpKsA1t6aYe9/AS8RHm6fTA8HILq414J3Mvbg3NfItK7P9wtxrSBAEAQVE/14xbO9v9EKYqloUqKQ7DfdasL7D9v0mAy2UA4iKQri3Ig1En2CVlJdDc+4V8FWEE7M6wSt8AHG/UQJFlPbxiXsq7vMYozMKfm/4XcFHD2zfzJkzKTQ0VLKlzjCfBw8eyGE+evRI88NNtM+NMsMv9syX5qXhpVrR3qaTxNomZ6UfTQ7hRwGH3HkiO+/04ytl9RPirQGmjJYiHHtzw9H0vXAC/nznXJotliyA7wy88yETDv8lcuC0oZ1NxtNq8eD76dQ66bWfTeTByZ/eW641gpBkLHWATLd4U8E6UQiBhEOwsWPw8m/nX42QnwGJBREyjuihqUdXyMR0C6r3Jaz9NHD3PCorftSfijBvvLXgbUHJsdBX5LcZdWCRaKOIUM4CpNIzIKipdLBDJBgE5hwoezNE0j54+avF2Bjw40UeGkjLgu9JJe/+8yfvlBkK4cQ53x1dTohWgH0c9LAinwp2DeZD3AyBt5ug1mv5BFGD1SOFaSqWJVPqIs8QoqjuCjYGOS4+2jyN4BgOQSZiJcoMN6YdQpnSP19pB/mCWhWoInffz1tK2urxVgfzmVqw3ATCwRWGSX3MGbYR8YTEfTBXOJNAuTly5IjMVaEeN3xHwN6ol2FQH09u23iztjUWuK98JPLKQEzdNxTst4jfY3mR+X1i+U60SPymz4pkmchk7ivWl8I9F5nh+++aq1R/51Px48H6U+qXCrxUIghixnsfS0Vmq1BWwISPO/QXLRQvSGqBqwHykcHXBi9NCFlvIaIsFTHnXoHzwPjDf++vCztl1CYUpB23Yv0BlbbwCVb9fRHR2mnT9+pip9lGdmtktrYny2dNcKCIIXILggzsYKTr1q1L1atXt2Y3VmvLRbx1xolTxJfed0GXRHUAjd7PK7tYV8mdTosfnTEtGwyN2pSBNw79XDlQUtxFe6iHdsH+qH13kHgO9mE8kPUXVjN1zNgE8ZaC5QKUyCHUw4/K2APcWDuOLO8glsEoKkxOhm5uWHMK4ftXhX8Lwr/VAhbmTMsfacS+32XeIOzbg/7dKCLIPt85751oLvXYDG0jbP9Uix8MHdJ82YEDByQFfe7cOYc6zN69e5eyZIn1qVKDhrdKazMT6vbj29YKPvGN09hx+LSUsZJDsbE+lHLc//Kny06502amcOFHB/MuFI/4BPfW7Y2+oforv6abz2LfyNXnFEifQ/jvpaYTgqXVvy9PrtBZvkRlndtWJgN9JKIdjSUUVLdpaBv3XDg4w4xlSsBGYS271usnmqpm8FhN8SK3uOYAg8e0Xqg4FGOpBpZ3ETCGj01eNREiCKcw5EgwpthgiGrFBvv6PyCU4XylHtpVKzY4DmUHioe+YhPfMRw3JGhfrdigjjMpNhjvPMGSYWVy0ML6Ah8pJDvUV2z06+FtzB6KDZaKgA+Bfpi6/nh43zYIGFJs0JMjFRvbzDTptgpFBowLwqnh+GuOYgM0cG/tJXLUDAr5UKgX74aro03kDjN0X1ajiQiphCo2aAf33PgUG0RpQbHp7KSsjRov3rYPAjZRbuwzdO7FGAK4Wfxvyw/UqXANaWYzVk+/HLQyxOtNRIL+cWvvYzXgQyKVgH5eHmv3w+0xAoyAYQRg/sHaTiPLtDao4Bg+iwS77SGZdP1oK2P1E1MOn8u+YhmIj7f+qAtASUx7fG7yQICVmyR6nfG21Wf7bOkrZM4U4a+E7NGQhsIO37pgFWmOM+fchNaBIzpWTE+OAoddCOzYSkbS5IiDoTlj9eQ5c+YYOsRlNkAAWcbhqwc/PHMEyy9UE753+A4jv1WgWKbFloJ72f+2/mAyDN2W/Tu6beCMqMvNm+P6PDl6XFro3xQubloYII/BdggYMtcZ6g1JFT8XDoj4U8TW5jj9ZSKUfpPDZ1BQkFyXBSHjWJxy7ty5Ds9AqgXcd+7cKRMJInPy7NmzHeqPpAU87DUGfSd/U/2uEUEB6uR6plwPTLVj7jH9LMrmnpdU6g0dOlSmZsACtojIRNSeI5J/aglPpN7AGlyISsWSEEjNoC/mqer6Z/F+kkMAigyyR6v/ktwkNTQhvI316tVLRitlyJBBrhqOm1hyZXHA1iDqrFKlSuTr6yvfVDt37qyhK8ZDURCA3576PqGf5Vipx5/WQSBHjhxyYdZ58+bJhJHFixenLVu2WKdxJ2wFbA1yTX399dc0cuRImZrBUNoNVm6c8OLykJMOAlhaYNOmTTKBFxauLFWqFDl6gTx7owu2BkwWTHQzZsyQTJaPj4+9h8H9MQKaRgDJNhGCXaRIEQoNDaXevXvT06dPNT1maw4ObA2Wr8CyNcg9hbW68EJkLAUHKzfWRJ/bYgQSgEByZXEMsTVduiQuDUUC4OdTGAGnQUDN4syfP5+SC4tjLlujvpCs3KjR4G1GwIEIJCcWh9kaB37RuGunR0CfxYGJOymyOJayNeoLy8qNGg3eZgQcjEBSZ3GYrXHwF4y7TzIIqFkcLN6a1FichLA16ovLyo0aDd5mBDSCQFJkcZit0ciXi4eRpBBIaixOYtga9YVl5UaNBm8zAhpCIKmwOMzWaOhLxUNJkggkFRYnsWyN+uKycqNGg7cZAQ0i4MwsDrM1GvxC8ZCSLALOyuJYi61RX1hWbtRo8DYjoFEEnI3FYbZGo18kHlaSR8DZWBw1WzNq1CijeWssvXCs3FiKGNdnBByIgMLiTJo0ibSaF4fZGgd+QbhrRuANAlpncQyxNf369TOat8bSC8vKjaWIcX1GwMEIgMXp2bOn5rIbM1vj4C8Gd88I6CGgVRbHVmyNevqs3KjR4G1GwIkQ0BKLw2yNeV+c11HRFH3jkXmVuRYjYCUEtMLigK2B6UmdZdiabI0aLl44U40GbzsVAilcWDdXWJx69epRp06d5CJyjrqItWvXprVr1xIvnUD05NFjirpwj2KEIoO/6GsPKfpqBL2OeEapO5Yh1xzpHXWZkmW/LuSSLOetnrTC4iAnDpL+TZkyRX3YbtupUqWSCk7fvn2tZoIyNHiX10LUB7CA4oor+9RFvM0IaBIBTzcPqpk7SJNjc8Sg8FNetWoVRUZG2r17Ly8vqlWrlt37dXSHt27dopMnT+r+jh49Ktf/uXPnjhyaSwoXcnV1o6hXr+R+hy96UsOurR097GTXf440GalsNv9kN29jE75x4wZt27bN2GGblgcHB1OBAgVs2gcaf0e5sXmP3AEjwAgwAk6MwMKFC+nbb7+lU6dO0ZMnT+RM3NzcCCyasVXdcQz0+7hx45x45jx0RsB5EGDlxnmuFY+UEWAENIDA+fPnqVChQhQdHW3WaLBqcZs2bWjevHlm1edKjAAjkHgE2Gkh8RhyC4wAI5CMEIAj90cffURga+IT1IEv0pw5c+KryscZAUbAiggwc2NFMLkpRoARSB4I3L59m/LmzUsIfzcmUGxCQkJo06ZNlDp1amPVuJwRYARsgAAzNzYAlZtkBBiBpI1AtmzZZHSaq6urwYlCsQHDs3r1alZsDCLEhYyAbRFg5ca2+HLrjAAjkMQQuHr1KnXu3Jl++OEHQlirvkCxgfKzfv16ypgxo/5h3mcEGAE7IMDKjR1A5i4YAUbA+RGIiIigAQMGUMGCBWnDhg00f/58Gjt2bJxcHWByPD095fFcuXI5/6R5BoyAkyLAPjdOeuF42IwAI2AfBOBXM3XqVBo9erTIWeNKgwcPph49elDKlCll6DdydoSHh8vBeHh40JYtW6h06dL2GRz3wggwAgYRYObGICxcyAgwAskdgZiYGJo7dy75+/vT0KFDpUKDMPA+ffpIxQb4uLu70/jx4wkJFBHyvXTpUlZskvsXh+evCQSYudHEZeBBMAKMgJYQWLFiBQ0cOFAm6sOyFsOGDSOkrzckUGzKli1LSCffokULQ1W4jBFgBOyMACs3dgacu2MEGAHtIrB79276/PPPpWmpSZMm0hSFhH3xyaNHjyh9el4vKj6c+DgjYC8E2CxlL6S5H0aAEdAsAuE41iQAABgmSURBVGfOnKGmTZtSuXLlpIkJq5z/+eefMhOxOYNmxcYclLgOI2A/BFi5sR/W3BMjwAhoDAEsINi9e3cqWrQonT59mpYvXy5ZGyg5LIwAI+C8CMSfP9x558YjZwQYAUbAIAIwI8EReNKkSZQ5c2aaOXMmtW/fPk5Yt8ETuZARYAScAgH2uXGKy8SDZAQYAWsg8PLlS5l8b+TIkXLhy0GDBlHPnj0NJuOzRn/cBiPACDgGAWZuHIM798oIMAJ2RAARTb/99ht9+eWXBFMUFBooNpxB2I4XgbtiBOyIAPvc2BFs7ooRYATsj8B///1HJUuWpLZt21JoaCjBeXjcuHGs2Nj/UnCPjIDdEGDlxm5Qc0eMACNgTwT2799PNWvWpNq1axOWQjh8+DD99NNP5OPjY89hcF+MACPgAARYuXEA6NwlI8AI2A6BCxcuUMuWLWWm4CdPnsjoJ0RBFStWzHadcsuMACOgKQRYudHU5eDBMAKMQEIRuHPnDvXq1YsCAgLo4MGDtGTJEkK+msqVKye0ST6PEWAEnBQBdih20gvHw2YEGIFYBCIjI2nixIk0YcIESps2rVzkEksmuLnx7Y2/I4xAckWAf/3J9crzvBkBJ0cgKiqKZs2aRcOHD6dnz57RgAED6NNPP6U0adI4+cx4+IwAI5BYBAwqN39f3EUdN36X2Lb5fEbApggc+HAy5U+f3aZ9OEvjCHWeNm2afMA/ffrU7sPG8gOTJ08mMCb2kMWLF9PgwYPp8uXL9L///U+GeGfJksUeXXMfjAAj4AQIGFRunGDcPERGgBF4gwAcaDt27Eg7duygfv36UUhIiN2xgW9L165dadGiRZJNsVVE0qZNm6QCt3fvXmrVqhUhzNvX19fu8+UOGQFGQNsImFBuXMTIX2t79Dw6RiAZI6CwNQMHDiQ/Pz/as2cPBQcHOwSRsLAwwh+ULEQlYVmDzp07W20sR44cIcxz1apVVKtWLTpw4AAFBQVZrX1uiBFgBJIWAhwtlbSuJ88mmSAAtqZq1arSx6Rv3760b98+hyk2CuTly5enQ4cOSQanW7duVKdOHQoPD1cOJ+jzypUrcs0nKG23b9+mdevW0Zo1a1ixSRCafBIjkHwQYOUm+VxrnmkSQABszdSpU6l48eL04MEDydaMGDGC3N3dNTG7VKlSyailbdu20aVLlySLM2fOHIvHdv/+ffrss8/I39+ftm/fTr/++ivBFFW9enWL2+ITGAFGIPkhwMpN8rvmPGMnRUCLbI0xKBPK4iDqaezYsZQ/f35asGCBVJROnjxJzZs3JxcXmMpZGAFGgBGIHwFWbuLHiGswAg5FAGzNlClTNMvWGAPHEhYnOjqaZs+eTQULFqRRo0ZR79696fz58/TJJ59ohpUyNk8uZwQYAe0hwMqN9q4Jj4gR0CGABzx8a+BXoxXfGt3gzNyIj8VZunQpBQYGUo8ePahhw4Z07tw5mbsGCflYGAFGgBFICAKs3CQENT6HEbAxAgpbU6JECU361lg6fUMsztChQ6lSpUrUuHFj6Ztz4sQJmavH29vb0ua5PiPACDACcRAwEQoepx7vMAKMgJ0QAFuDZHjIWzNo0CAaMmRIkjHNKCzOl19+SV9//TVlzJiRli1bRvXr17cTutwNI8AIJAcEmLlJDleZ5+gUCChsDSKhIiIiNBcJZS0QFRYHUVBZs2al1q1bU0Iiqqw1Hm6HEWAEkh4CrNwkvWvKM3JCBMDWhIaGSr8ahEBrIW+NrWFUWBxkNrZWXhxbj5nbZwQYAedAgJUb57hOPMokikByYWuMXT6FxUlsXhxj7XM5I8AIJE8EWLlJntedZ60BBJIjW2MMdoXFAYPDLI4xlLicEWAEzEWAlRtzkeJ6jIAVEUBCvqTuW2MpXGBxxo8fT2oW59atW5Y2w/UZAUaAESBWbvhLwAg4AAE4DD99+pT+/PNPh68J5YDpm+wSLM7ChQvp0aNH9OTJE5N1+SAjwAgwAoYQYOXGECpcxgjYCQFeUsAw0IyLYVy4lBFgBMxDgJUb83DiWowAI8AIMAKMACPgJAjYJIlfLs9MVCJzPiqaKQ/FiHVxzj+6SQfvnKfX4l9Oz8y069ZpJ4HHsmFWzRlImVKli/ek/8IP0uNXz+jjovXoefQrmnNqrTwnt8Cmlk8wBWXOT722z5Rlbi6uVDF7ANX2CaGN14/S2quH4m0fFVxdUlDJrH605/ZZs+pbWil76owUkDEXbbp+jDJ5pKPgLPlp/bXDumbSuqWiD/0qkm+6bHRBXP/F57fTs+iXuuP185amfy/v1e3zhv0ROH78OK1cuZIqVKhAFStWtP8AuEdGgBFgBGyEgFWZG/cUrjSidGva33QylfUuRIfvXqI9t87IB9zmhqPpcNj3VDKLn42m4vhmj9y/RKWzFqA5oT1pZJnW5JHCTSoZUDTSuaeSCsD0yt0JSgykjX8otSxQWW57unlIzPoHNaYauUvIMvxXNJMPNc5XnnoUq0c50mTUlZvaSO+emnoFfkAn7oebqpaoYx0CqlGYXyXZRtP85aldoaq69gqkz0H7P5xMPYu9Tz2EAvd9pW60o/E4ypbaS1fn9rMI+q5iV4mPrtBJNwYPHkz9+/engwcPOs0Mzpw5Q2PGjKHPP/+cwsNt9z1xGkB4oIwAI5CkELCacuPh6k5r639N7QtVo0ZrRtPQvb/Sf1cP0tabJ+jbI8sodNlguhZ5n1KLh7i9pcUbBcIW/arbvvf8Mf1+bqvs5sKjW/TruS206Pw2+Tf39Ab6cs8C+vHEakrpGkuYVV8+hOqv+lrWj4x6QX9e2EH7bp+LM8zD9y7RrJP/xSkztQMFaEaVj2mOOOdJ1HNTVRN1rFqu4rT+aixTUzVXIG24ekTX3piy7ajJmjFU8s++VPiPHjRPzD1fem8aUrK5rg4YpWWX9kgFR1fopBtXr16lCRMmUEhICPn5+dGIESPo7FnbMGbWgsjf35969uxprea4HUaAEWAENIWA1ZSb/iUaU1CWfPT90eUGzU6XHt+m8Yf+IjAU9pTK2YvQVyVb2KRLQ23D3GRKZgrl5vLjO7LKU6HQwCyllujXMdJ8py6Leh0td2HWi09Gl2krzT2P4hlHfO2YOu6VMo1koWCSAitVOUdRnUkqSJgjodAdf3BFNgGFb/SBxcI8GUNls/nHaRZmrAJeOah6rrdMVZwKTrSTIkXsTwkh3lgzCcoDFr2cPHkyXb9+XZMzcXV1leNi511NXh4eFCPACCQCAav43MDc0Lv4B4SH9Yzjq40O59dzm6lenlK64/DLqOkTRIUy5BKszj3acO2IZHeUCsqD87V4MOJNv06eECrolVMyHPDjUUul7IUpMLMvRcfE0JmH14UvyFGC8vFrzX6ELLAdClWnm08f0OrwA/K0VIJpqpSjiPQNgkLxh2BcbojjEHP6NdW2bMTAf2H5K9LiC9t1R7KkSk91hC/NgrObdGWJ2QgRJj/47PTcFuuvo27LHKyr5Cwmr+H5hzfp/bwlhTnRm5YLv5j9d2LZJH+BPergGjx88ZRgjgJTBHzr5ikp8dt+8yQdundR3TXdEiaoQ3cvkqKkqQ/+cHwVDSvVUl57c5Q39bla3Y6KipJDO3LkiDRXYTkF+LS0b9+emjZtKheL1NrY79+/T4sXL5bh12FhYeTr66u1IfJ4GAFGgBEwGwGrMDfFhVLhLvxLwM6YMoW8iommpZd2y8EVE87Ga+oPpyhRBrOLV0pP2t1kIilmngxif6Ywr/xT5wtqLXxT4LdRRrz5dylck1bU+4pwXBGYO/Knz054UEIJGlKymTwU8TKSjt+/Qi+jo+icUHigQEHAHh348Ft6HvWSJh9ZSnDaxVig8Jjbr7G2ZQcG/ksj+oQ/DSSFiwu1KvAeHQz7lr4q9dZUY+A0i4qgYO4V89e/BvFhnTNNJvq5am/6q/Yg4atTn6ZW7kbFMuWV12LN+8OoQd4ychyRwsx18kE45U2XVSqJZx/eID/BvMA8hfJzYv/BC8N5SXIJPyNDztBwLg/MnFcqeRZN1kkqR0dHS+UPi0R+9NFHcqFIrIC9Zs0azcxgxYoVVL16dbk69/Dhw6V5be9edvbWzAXigTACjIDFCFhFuSmSwUd2fFkoN+YIHI9/Cu0lzSdgBmC6mHpsBa26sp++r9hNMjlQHnps/VE2l12wA923TKdBu3+hXttmEfbLer81cXQQfj6IyIEcuneBVop2IEfvX6a7zx9J0882wShgHwL2KHuaDHQ64rqM5lodvp/ypM1KhTP6kLn9GmtbdiD+K5oxDy2tM1j+rao3lE61mC7HjeOIIIM/zsZrR5XqVvksJvpU2CelQXOwvv70Pn21d6E8BYpg87Xjqd/On6nyP4Mo4kUkjSnXTrJZ8JkCjlAkfz27WbJjcI5eInyFUH4q4qrSbZzPCt4BkrWZfmxlnHLsgNWJEAoRTJqWyLPfD0jFCiYVR//9888/8Q4d7FaMYBWh7ECZ+OKLL+Q5X331FT1+/Dje821ZASY1OENjXOvXr5eJ8z755BNbdsltMwKMACNgUwSsYpaKEmYdCMw55kiNXEHkL0xRe+/EdbqED0aYCB9u619VOt++EP4oeChcFM65MB1BlAdobs8suq7AIIB56L19llRsphz9V3cMG/rmDjyMDwvTyZ3nDwmO0BWF+QriJx7aB+9eIHP7xTn6baMMAp+ThqtHxe6I/8EIbWgwUrePjZd6/jZxDlq4AyVGMSOpTzUXa5gUIUeEA7MiwGfemQ30WYlGgq2JDemGMgOmB9cOvjeBmXxp643jyinvfIKl+iIkjFqunUBwmjYkD18+kwqtoWPGylJWzEcTW/cjb6GkOlpmzJhBGzdujHcYil8OPmGm2rx5MyHSKl26+NMHxNt4Iio0adJEd3bZsmWpZMmStGvXLrp79y5lyfL2d6arxBuMACPACGgcAasoNzBJQKAcmCOFRH4USOSr53Gq77x5Su4X8oo9Hufgmx04pkLUTpD9Bcswr1of+rVGP5l3peumqVJxeXPKOwoIFJLbzx7SF8FhgtV5SQeEQgNJYUI5M9QvzjGm3OCYWsAITTq8VF1k1e2MHmnJVTw0nwlTm1oSgzXagakJghw+CO0Gw4U+xpXrIBUL4De8dCuhzIXTbANRXSNLt6FpgrFBmLwxgbkrp8iNZIm4+mSkeo0/kCySJefZou6//8ZVptV94HsKZQasTZUqVaht27YEZQKLZkKJ8PCwr4O9emzGtpH3BsoNHKFZuTGGEpczAoyAlhEwj2qJZwZwIH0iFBUwB0jaFp/ADAGBD41arjy5S69ioqRpSF0e3zZMRO8JEwoernD03dJodByfHEH+xJG84gG9tdFY2n/3HE0SPjfhT2Kjl+JUMnNHv21Tp1nLcdhQH1DWYEJKK/LpqCWxWPukjX1zXyeSB44+sIRexLyiGSdXy204Fc8/s1Fu/352i7pbuQ1zIZSaVcLsZ0rAal17EusPZaqeMx1zc4t9b0B4OCKmbty4QRs2bKCOHTuSl5eXpqeSM2dO+fKQL59lpkJNT4oHxwgwAskKAasoN3AiHSPCfcEcIImfKQkUjqr73kTfVBCZd9VSRPi8wDF5z+0z6mKT2ylF/eYimRycaOEnErb2G0L23Aa+sU6wMGvpm8sGBn8o+hFOxCJTMMQUY2Oqc0Ntm6pv62Mw2WVVJcpDf4nF+j0R5n1IMFuXhQIIM1WZrP608vJ+uV3Ou5D0k0K5vhMzMhALfk2X90eZe0UR1aYW4TEjk/tdfHxLXex022Bm3N3d5bgRBg7HXISF79u3T+aT8fb2dpo5wVwGs5mjzWVOAxgPlBFgBDSHgFXMUpgVktOVEtl5G4vwYGSeHbBrbpwcLj7CR+azoEYy5HqniJCBQ+oHQgGBD8fVN1FMeFieF2aQuafXS6AQ1QRaHwqMIpk80svN1MJXBoLjnQJq0h8itwoE4eRwUMYfBA6r3mm8dIwSGI407h7Subdm7iAR5nxeRmChLsKa4UeCCK74+jXWNqK+IDDfxCcpxRzSu6eRypfiU5Re9K+fCwh1IJ4idN6UwKxXXZXdGHWPiWgxc7BW2sWSGYoAjxCxhENL4WAMKS78a2LEP/gTgaFDCoDdBhTRUBEu3kdkSMY16Vq4ljwXCmZAhtx0QpivEC6uSE7PjOQmFE3FCVwpd7ZPsB3t2rWjVq1aUWBgoFMN/+HDh7rx3rlzR5qk1q5dqyvjDUaAEWAEnA2Bt1pDIkeOh3PHTd8LE8QBmYn2SLPvJWsAJaO8iJY5KhxVRwl259yjWB+OT3fMkT43i2sNEIn//hUPuBQiR0sQNVg9Upimogmh00pGW2TDrS3yt2A5h89KNJQjbeZXWTiynqCTgq1AaDKWPEDG2zzCjIK1mlZc2Sfr/XNxl8xxs6nBaJlMbubJNTT16AqZhG5B9b6EdZ4G7p4nE8x9WryByN8SKdZMyi3PNdUvTHH6bd8QUUcfFakjz4U559sKXWiGUPowRrUg5LydfzVCbp5UbinlPGeeWEMN85WVWKUWZYMEu4QQeShJA4ObytNbFnxP4rfuTWZgdZvY/k4kUMSSDlA8EJavSHxYK/Xw6Z06g4xYuyvYGMz/o83TaPMbh2FkIlYivJB4b4dQpnCt1FJCpAVYWP0z8hTmsVLZCqoPydD7gN97xCnD0hIIB1cYpjgHnWQHyxjMnTtXKsROMmQ5TChhnTp1kizTgQMHKE2aNNIXCIoNEhCyMAKMACPgrAi4CNOKnkcK0d9CIei48Xsxp3cOmT1P+FEUFkoCHn5wSoVDrSHBOkhQJq4KnwuEJCdEwAogKgcPZoUFUreDPhB+rTadwBwCJUKJEkJ9mKr0H9bqdgxtG2rbUD17lSFZYVFh3usvmDN9MYU1WJgzLX+kEft+l/mCsA9TlK1lo4gg+3znvHci58zp94BYvwph6c4oUCbgUHzu3Dm5ZIOj54AlJDJlyiQVHEePBf1rDR8tYMJjYAQYAfMRsBpzo98llBmYn+ITLBOQ2JWrwRpFCz3MkGKD/g0tRYAoJ7Vig3qWKjbG2ka5o2SeWMdpdugn0oSkH6FkLtZYvdseig2WikAEmX5KAEdhl5z7zZ07lq1Mzhjw3BkBRiDpIGAVh+KkA4fzzwRK2/+2/ECdCteQpjdzZwQzIMTLI9ZnyNzzEloPPjkw7SGJIwsjwAgwAowAI2BNBFi5sSaaGmnrpQin77N9Nt0RztPmCPyU4OMDaSiWWmhdsIo00ZlzbkLrwNkYC2wmV4HDOmTmzJn06tWr5AqDwXk/f/6c5syZY/AYFzICjAAjYA4CNjNLmdM517EtAsbMdPq9YsmGz4WPDv4USYiJTjnXnE/9ZSLMOScp1QkKCqLvvvuOBg0aRKtXr5YOycHBwUlpigmay86dO2UuIOQFmj17tib8kRI0ET6JEWAEHIoAMzcOhV8bnUORefjyaZw/bYws6Y4CzE2vXr0IK4dnyJCBypQpQ0OHDk22LA7Ymv79+1OlSpXI19eXjh07Rp07d066XwCeGSPACNgUAVZubAovN84ImEbAz8+PNm3aRBMnTqQJEyZQqVKl5CKWps9KWkfB1oDJgokO63SByfLxiV2MN2nNlGfDCDAC9kKAlRt7Ic39MAJGEEiuLI4htqZLly5GUOJiRoARYATMR4CVG/Ox4pqMgE0RSE4sDrM1Nv0qceOMQLJHgJWbZP8VYAC0hEBSZ3GYrdHSt43HwggkXQRYuUm615Zn5sQIJEUWh9kaJ/5C8tAZASdDgJUbJ7tgPNzkg0BSYXGYrUk+31meKSOgFQRYudHKleBxMAJGEHBmFofZGiMXlYsZAUbApgiwcmNTeLlxRsA6CDgbi8NsjXWuO7fCCDACCUOAlZuE4cZnMQIOQUBhcSZNmqTZvDjM1jjkq8GdMgKMgAoBVm5UYPAmI+AMCIDF6dmzp+ayGzNb4wzfHh4jI5A8EGDlJnlcZ55lEkRASywOszVJ8AvGU2IEnBgBE8rNayeeFg+dEUgeCOizOCEhIYQye/9VqFBBtyYUZxlOHt89niUjoGUEXF4L0R/gtcj7tPfOWf1i3mcENIVAzdxB5OnmoakxOXIw+CmvWrWKIiMj7T4MLy8vqlWrlt375Q4ZAUaAETCEgEHlxlBFLmMEGAFGgBFgBBgBRsAZEDBhlnKG4fMYGQFGgBFgBBgBRoARiIsAKzdx8eA9RoARYAQYAUaAEXByBFi5cfILyMNnBBgBRoARYAQYgbgI/B8GumLmg9JBCAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 22,
     "metadata": {
      "image/png": {
       "width": 800
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from caffe2.python import net_drawer\n",
    "from IPython import display\n",
    "graph = net_drawer.GetPydotGraph(net, rankdir=\"LR\")\n",
    "display.Image(graph.create_png(), width=800)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we have defined a Net, but nothing has been executed yet. Remember that the net above is essentially a protobuf that holds the definition of the network. When we actually run the network, what happens under the hood is:\n",
    "- A C++ net object is instantiated from the protobuf\n",
    "- The instantiated net's Run() function is called\n",
    "\n",
    "Before we do anything, we should clear any earlier workspace variables with `ResetWorkspace()`.\n",
    "\n",
    "Then there are two ways to run a net from Python. We will do the first option in the example below.\n",
    "\n",
    "1. Call `workspace.RunNetOnce()`, which instantiates, runs and immediately destructs the network \n",
    "2. Call `workspace.CreateNet()` to create the C++ net object owned by the workspace, then call `workspace.RunNet()`, passing the name of the network to it\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current blobs in the workspace: []\n",
      "Blobs in the workspace after execution: [u'W', u'X', u'Y', u'b']\n",
      "W:\n",
      "[[ 4.6133149e-01  2.6810661e-01 -1.0076044e+00]\n",
      " [ 1.2857534e+00 -7.2475618e-01  6.1970286e-04]\n",
      " [-2.4734004e-01  1.1377212e+00  1.2011141e+00]\n",
      " [ 6.0975075e-01 -6.4694040e-02  5.8932066e-01]\n",
      " [ 6.5264893e-01 -6.8908274e-01  1.4284889e-01]]\n",
      "X:\n",
      "[[-0.09317103 -1.3330355  -0.60921687]\n",
      " [ 0.7734029  -0.1529701  -0.2614098 ]]\n",
      "Y:\n",
      "[[ 1.2134712   1.8459532  -1.2253168   0.67040426  1.7707379 ]\n",
      " [ 1.5791805   2.1051095   0.32068616  1.327425    1.5728276 ]]\n",
      "b:\n",
      "[1. 1. 1. 1. 1.]\n"
     ]
    }
   ],
   "source": [
    "workspace.ResetWorkspace()\n",
    "print(\"Current blobs in the workspace: {}\".format(workspace.Blobs()))\n",
    "workspace.RunNetOnce(net)\n",
    "print(\"Blobs in the workspace after execution: {}\".format(workspace.Blobs()))\n",
    "# Let's dump the contents of the blobs\n",
    "for name in workspace.Blobs():\n",
    "    print(\"{}:\\n{}\".format(name, workspace.FetchBlob(name)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's try the second way to create the net, and run it. First, clear the variables with `ResetWorkspace()`. Then create the net with the workspace's `net` object that we created earlier using `CreateNet(net_object)`. Finally, run the net with `RunNet(net_name)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current blobs in the workspace: []\n",
      "Blobs in the workspace after execution: [u'W', u'X', u'Y', u'b']\n",
      "W:\n",
      "[[-0.49780178  0.0896128  -1.1650279 ]\n",
      " [-1.9277409   0.91667885  1.2152035 ]\n",
      " [-0.80195075 -0.17179072  0.8195034 ]\n",
      " [ 0.21857451 -0.9942886   0.9358572 ]\n",
      " [-0.7295994  -1.085294    1.1550978 ]]\n",
      "X:\n",
      "[[-0.94531673  0.2626604  -1.3781935 ]\n",
      " [-0.15628844  1.2679638   0.40703335]]\n",
      "Y:\n",
      "[[ 3.099752    1.3883154   0.58354056 -0.7575747  -0.18730962]\n",
      " [ 0.71722126  2.9582276   1.2410765   0.0860424   0.20807779]]\n",
      "b:\n",
      "[1. 1. 1. 1. 1.]\n"
     ]
    }
   ],
   "source": [
    "workspace.ResetWorkspace()\n",
    "print(\"Current blobs in the workspace: {}\".format(workspace.Blobs()))\n",
    "workspace.CreateNet(net)\n",
    "workspace.RunNet(net.Proto().name)\n",
    "print(\"Blobs in the workspace after execution: {}\".format(workspace.Blobs()))\n",
    "for name in workspace.Blobs():\n",
    "    print(\"{}:\\n{}\".format(name, workspace.FetchBlob(name)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a few differences between `RunNetOnce` and `RunNet`, but the main difference is the computational overhead. Since `RunNetOnce` involves serializing the protobuf to pass between Python and C and instantiating the network, it may take longer to run. Let's run a test and see what the time overhead is."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Run time per RunNetOnce: 9.96360778809e-05\n",
      "Run time per RunNet: 8.03422927856e-06\n"
     ]
    }
   ],
   "source": [
    "# It seems that %timeit magic does not work well with\n",
    "# C++ extensions so we'll basically do for loops\n",
    "start = time.time()\n",
    "for i in range(1000):\n",
    "    workspace.RunNetOnce(net)\n",
    "end = time.time()\n",
    "print('Run time per RunNetOnce: {}'.format((end - start) / 1000))\n",
    "\n",
    "start = time.time()\n",
    "for i in range(1000):\n",
    "    workspace.RunNet(net.Proto().name)\n",
    "end = time.time()\n",
    "print('Run time per RunNet: {}'.format((end - start) / 1000))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Congratulations, you now know the many of the key components of the Caffe2 Python API! Ready for more Caffe2? Check out the rest of the tutorials for a variety of interesting use-cases!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}