ActivationFunctions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Activation Functions\n",
    "\n",
    "This notebook explores the activation functions that are available in ConX.\n",
    "\n",
    "First, we import a special function `plot_f` that is designed to plot functions. We also import all of the activation functions:\n",
    "\n",
    "* softmax\n",
    "* elu\n",
    "* selu\n",
    "* softplus\n",
    "* softsign\n",
    "* relu\n",
    "* tanh\n",
    "* sigmoid\n",
    "* hard_sigmoid\n",
    "* linear\n",
    "\n",
    "There are additional advanced activation functions not defined and examined here, including:\n",
    "\n",
    "* ThresholdedReLU - Thresholded Rectified Linear Unit\n",
    "* LeakyReLU - Leaky version of a Rectified Linear Unit\n",
    "* PReLU - Parametric Rectified Linear Unit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n",
      "ConX, version 3.6.10\n"
     ]
    }
   ],
   "source": [
    "from conx.activations import *\n",
    "import conx as cx"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## softmax\n",
    "\n",
    "The `softmax` activation function takes a vector of input values and returns a vector of output values. This function is unique in that the output values are not computed independently of one another. Mathematically, the outputs of softmax always sum to exactly one, but in practice the sum of the output values will be close to but not exactly equal to one. The outputs of softmax can be interpreted as probabilities, and can be used, for example, with ConX's `choice` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function softmax in module conx.activations:\n",
      "\n",
      "softmax(tensor, axis=-1)\n",
      "    Softmax activation function.\n",
      "    \n",
      "    >>> len(softmax([0.1, 0.1, 0.7, 0.0]))\n",
      "    4\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(softmax)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.21155263483524323,\n",
       " 0.21155263483524323,\n",
       " 0.38547399640083313,\n",
       " 0.19142071902751923]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "softmax([0.1, 0.1, 0.7, 0.0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9999999850988388"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(softmax([0.1, 0.1, 0.7, 0.0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.032058604061603546,\n",
       " 0.08714432269334793,\n",
       " 0.23688283562660217,\n",
       " 0.6439142823219299]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "softmax([1, 2, 3, 4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0000000447034836"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(softmax([1, 2, 3, 4]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'c'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cx.choice(['a', 'b', 'c', 'd'], p=softmax([0.1, 0.1, 0.7, 0.0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how softmax can be used to make probabilistic choices, and see if they match our expectations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softmax: [0.21155263483524323, 0.21155263483524323, 0.38547399640083313, 0.19142071902751923]\n",
      "picks  : [0.22, 0.24, 0.31, 0.23]\n"
     ]
    }
   ],
   "source": [
    "bin = [0] * 4\n",
    "for x in range(100):\n",
    "    pick = cx.choice([0, 1, 2, 3], p=softmax([0.1, 0.1, 0.7, 0.0]))\n",
    "    bin[pick] += 1\n",
    "print(\"softmax:\", softmax([0.1, 0.1, 0.7, 0.0]))\n",
    "print(\"picks  :\", [b/100 for b in bin])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "f = lambda x: softmax([x, 0.5, 0.1, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmYXHWd7/H3t5eQliTdjAkmnWUCjxFFwQEyLKKIIIsg4jhettErjk7u1WEMXIUBHZlcXNgeBbxydXABVxgeZDJhyBhHEVFATSASTCDARZB0BxOWbiALdHe+949zqnPSqeVU1Tl1TlV9Xs+Th6pTp6t/HaA+/f19z+93zN0REREB6Mh6ACIikh8KBRERGadQEBGRcQoFEREZp1AQEZFxCgURERmnUBCZwMw+ZmZ/MrOXzOzVORjPpWZ2bvj4QDO7J+sxSesyrVMQ2cnMuoEXgMPd/YHwmAML3P2xDMYzA/gd8Fp33xYeWw58zd1va/R4pPWpUhDZ1WuAycDarAcSOhtYXgiE0A+A/5HNcKTVKRSkpZnZP5rZgJm9aGbrzexYM9vDzK42s8Hwz9XhsdcB68MvHTKzO8zsrvD5A+F00ulmdrSZbTCzC8xsk5ltNLP3mtlJZvaImT1nZp+OjOFQM7vXzIbCc79qZpPC195iZs+Y2dzw+ZvN7Hkze3345e8CfjHhx7oTONbM9kjr703al0JBWpaZ7QecA/ylu08FTgCeAD4DHA78BfBm4FDgn9z9EeCN4Zf3ufsx7n5U+PzN7j7F3f81fD6ToKKYDVwMfAP4AHAI8Dbgs2a2T3juGHAeMB04AjgW+DiAu98D/AvwHTPrAb4PfNbdHw6/9gB2BhXh1wwAI8B+df0FiRShUJBWNgbsAexvZt3u/oS7/z/gb4BL3H2Tu28G/jfwwSrfewT4gruPADcRfOBf4+4vuvtaYB1B4ODu97n7r9191N2fIAiBt0feawnQC/wWGACujbzWB7xY5Pu/GL4mkiiFgrSssDF8LsGH7iYzu8nM+oF+4MnIqU+Gx6rxrLuPhY8L8/1/iry+DZgCYGavM7P/MLOnzewF4IsEIVIY5whwA/Am4Eu+69UfzwNTi3z/qcBQlWMWqUihIC3N3X/o7m8F/hxw4HJgMHxeMC88lpavAQ8TXME0Dfg0YIUXzWw28M/A9cCXJvQK1gCvi75ZeP4kJkwriSRBoSAty8z2M7Njwg/Z7QS/ve8AbgT+ycxmmNl0gp7A98u81Z+AfesYylSCy1xfChvIH4uM0QiqhG8BHwE2Ap+LfO1ydp1qInx+h7u/XMeYRIpSKEgr2wO4DHgGeBrYG7gI+DywiuC38AeB+8NjpSwhaAQPmdlpNYzjU8BZBH2AbwD/GnntE+G4PhtOG30Y+LCZvS18/bvASWETuuBvgK/XMA6RirR4TSTnzOyLwCZ3v9rMDgT+xd2PyHpc0poUCiIiMk7TRyIiMk6hICIi41ILBTP7drgFwO9LvG5m9hUze8zM1pjZwWmNRURE4ulK8b1vAL5KcPVEMe8CFoR/DiO4lvuwSm86ffp0nz9/fjIjFBFpE/fdd98z7j6j0nmphYK732Vm88uccirw3fAyvF+bWZ+ZzXL3jeXed/78+axatSrBkYqItD4ze7LyWdn2FGYDT0WebwiP7cbMFpnZKjNbtXnz5oYMTkSkHTVFo9ndr3P3he6+cMaMitWPiIjUKMtQGADmRp7PCY+JiEhGsgyFZcB/D69COhwYrtRPEBGRdKXWaDazG4GjgelmtoFgF8huAHf/OsFGXycBjwFbCfZ8ERGRDKV59dGZFV534O/T+v4iIq1i6eoBrlyxnsGhbfT39XD+Cfvx3oOKXpdTtzTXKYiISI0KQTAwtA0juBkIwMDQNi669UGAVIJBoSAikhOlgmDitqXbRsa4csV6hYKISCuJTgv19nSz5ZVRRsaCCKi0f/Xg0LYKZ9RGoSAi0kClqoGhbSNVvU9/X0/lk2qgUBARSVncaaG4ero7Of+E/RIa3a4UCiIiKUg6CArvMVtXH4mINJelqwe46NYH2TYyBuQ/CKIUCiIiCYlWB7Xo7jCmTO5iaOtI6usRSlEoiIjUodQ0UVxZVAPlKBRERGpU6zRR3oIgSqEgIlKlWqaJ8hwEUQoFEZEqTKwO4sh7EEQpFEREYqilOujp7uTS9x3QFGFQoFAQEamgmuqgWaaJSlEoiIiUUG110KxBEKVQEBEpoprqoBmniUpRKIiIRLRjdRClUBARCbVrdRClUBCRttfu1UGUQkFE2pqqg10pFESkLak6KE6hICJtR9VBaQoFEWk7V65YHysQ2qU6iFIoiEjbiDtl1G7VQZRCQUTaQtwpo3asDqIUCiLSFipNGbVzdRClUBCRllWYLhoc2lb2BjjtXh1EKRREpCVVM11094XHNGhU+deR9QBERNIQ5wqjnu5Ozj9hvwaNqDmoUhCRlhLnCiMD+jVlVJRCQURaRpwpI00XlafpIxFpGXGuMNJ0UXmphoKZnWhm683sMTO7sMjr88zs52a22szWmNlJaY5HRFrT0tUDHHnZHWWnjGb39eiS0xhSmz4ys07gWuA4YAOw0syWufu6yGn/BNzs7l8zs/2B5cD8tMYkIq1HU0bJSrNSOBR4zN0fd/dXgJuAUyec48C08HEvMJjieESkBWnKKFlpNppnA09Fnm8ADptwzhLgJ2b2D8CewDuLvZGZLQIWAcybNy/xgYpI84lzlZEWpVUv60bzmcAN7j4HOAn4npntNiZ3v87dF7r7whkzZjR8kCKSL4Upo0qBcPeFxygQqpRmKAwAcyPP54THoj4C3Azg7vcCk4HpKY5JRFqApozSk2YorAQWmNk+ZjYJOANYNuGcPwLHApjZGwhCYXOKYxKRFjCoq4xSk1pPwd1HzewcYAXQCXzb3dea2SXAKndfBnwS+IaZnUfQdD7b3cvtWyUibazQRyj1IaGrjOqX6opmd19OcJlp9NjFkcfrgCPTHIOItIZKl55qyigZ2uZCRJpCuT6CrjJKjkJBRJpCqT6CgaaMEqRQEJFcq9RH6O/raeh4Wp1CQURyS32ExlMoiEhuqY/QeAoFEckt9REaT6EgIrmjPkJ2FAoikivqI2RLoSAiuaI+QrYUCiKSK+ojZEuhICK5oD5CPigURCRz6iPkh0JBRDKnPkJ+KBREJHPqI+SHQkFEMqM+Qv4oFEQkE+oj5JNCQUQyoT5CPikURCQT6iPkU0fWAxCR9rT3tD2KHlcfIVuqFESkoZauHuCKFQ/zpxde3u019RGyp1AQkYYp1lw2wFEfIS8UCiLSMMWay4VAUB8hH9RTEJGGKdVcLnVcGk+VgoikTovUmodCQURSpUVqzUWhICKp0iK15qJQEJFUaZFac1GjWURS9Zppk4seVx8hn1QpiEgqCs3lp1/Yvttr6iPkl0JBRBKnRWrNS6EgIonTIrXmpZ6CiCROi9SaV6qhYGYnmtl6M3vMzC4scc5pZrbOzNaa2Q/THI+INMarp0wqelzN5fxLbfrIzDqBa4HjgA3ASjNb5u7rIucsAC4CjnT3581s77TGIyLpW7p6gCt+/DDPvPTKbq+pudwc0uwpHAo85u6PA5jZTcCpwLrIOX8HXOvuzwO4+6YUxyMiKVJzuTWkGQqzgacizzcAh00453UAZnY30AkscfcfpzgmEUmJmsutIeurj7qABcDRwBzgLjM7wN2HoieZ2SJgEcC8efMaPUYRiUHN5daQZqN5AJgbeT4nPBa1AVjm7iPu/gfgEYKQ2IW7X+fuC9194YwZM1IbsIjUru9V3UWPq7ncXNKsFFYCC8xsH4IwOAM4a8I5S4EzgevNbDrBdNLjKY5JRBJUWLU8OLQNZ2cPoUDN5eaTWii4+6iZnQOsIOgXfNvd15rZJcAqd18Wvna8ma0DxoDz3f3ZtMYkIskp1ljuMJjW083Q1hH61VxuSuZe6rYX+bRw4UJftWpV1sMQaXtHXnYHA0X6BWos55OZ3efuCyudpxXNIlITNZZbk0JBRGpSqoGsxnJzy/qSVBFpMoXmcrGpIzWWm59CQURiK3e/Za1abg0KBRGJrdT9ltVcbh2xegpmtjjOMRFpbWout764jeYPFTl2doLjEJEmoOZy6ys7fWRmZxKsQt7HzJZFXpoKPJfmwEQkP9Rcbh+Vegr3ABuB6cCXIsdfBNakNSgRyQ81l9tL2VBw9yeBJ4EjGjMcEckbNZfbS6yrj8zsRXbuczUJ6Aa2uPu0tAYmIvmg5nJ7iRUK7j618NjMjOAOaoenNSgRyY/+vp6ivQQ1l1tT1dtceGApcEIK4xGRnFi6eqDkpndqLreuuNNH74s87QAWAttTGVEKhm+7jU1XXc3oxo1Yby8dwNjw8C6Pu2bNYsrbj+KlX9xV9rw8PO6aNYu9zzuX3lNOyeBvU9qBmsvtK9bW2WZ2feTpKPAE8A1335TSuEqqduvs4dtuY+NnL8a3N02GxdPVReeUKQohSYW2xW49cbfObvn7KTx6zLGMDg6mOKL2Yn19sUIkWnUpVJrPPhfeTrFPBgP+cNnJjR6OJCBuKMSdPtoXuIaguezAvcB57p77W2eObtyY9RBaig8NMVbh8ejgIEM33jT+NaODgwyefwGD518QO1QUItlSc7l9xd0Q74fAtcBfhc/PAG4EDktjUEnqmjVLlUKOxA2VYiGisEifVi5L3J7CGnc/cMKxB9z9zamNrAT1FARQWKRAzeXWlmhPwcwuB54HbiKYPjod2Au4EsDdG7YPUi33aG61q4+stxe2bMFHRpL6a20ZCovaqbnc2pIOhT+Uedndfd9qBlePWkKhFcUJukQeDw2BGTTZBQkTdfX3KyAqUHO5tSUdCpPdfXulY42gUGi8agNol6prcDB3oaJqojhVCq0t6VC4390PrnSsERQKzaeqUMmiMoms+WjHkKjUXL70fQeol9ACEgkFM5sJzAa+T3BfBQtfmgZ83d1fn8BYq6JQaH1FQySDsGiHKSc1l9tHUqHwIYI7rC0Eop/ELwI3uPutdY6zagqF9pVlWLRqQGjKqH0kPX301+7+o0RGVieFgkzU6LBopYBQc7l9JB0K/wy7/7fj7pfUNrzaKRQkrvGwSLHZ3ewBoUqhfcQNhbhbZ78EbAn/jAHvAubXPDqRBug95RQW3PEz3vDwQ/RfcTld/f1ghvX10dnXN/7Yurtr/h6F1dcPvf4NPHrMsQzfdluCP0F6tC22lFLThnhmtgewwt2PTnxEFahSkKQlWVHY5MnM+twlua4c1FxuT6nukmpmewEr3f21tQyuHgoFSVNSAZHnaSVNGbWnRKePzOxBM1sT/vk9sJ5g11SRllJ0ygmCgKhCnqeVdM9lKSfuLqnvJtjr6G1AH7Dc3e9LbVQiOdB7yinjv+nXU0GMDg6y8bMXj79n1rQttpQTt9F8KvA9YDrQDVxvZv+Q2qhEcqbeCsK3b2fw/AsyrRrUXJY4Ym+dDRzh7lvC53sC907cTrvI151IMM3UCXzT3S8rcd5fA7cAf+nuZRsG6ilInuxSQVSpkX0HNZcl6UtSDYj+1zTGzi0vSg2gk+DGPO8C9gfONLP9i5w3FVgM/CbmWERyo1BB9F95BTZ5clVfW5hWakTlcOWK9SUD4e4Lj1EgyLi4oXA98BszW2JmS4BfA9+q8DWHAo+5++Pu/grBvRhOLXLe54DLAd0FR5pW7ymnMOtzl+R2WknNZYkrVii4+5eBDwPPhX8+7O5XV/iy2cBTkecbwmPjzOxgYK67317ujcxskZmtMrNVmzdvjjNkkYYr2XeIIe2qoVQTWc1lmShupYC73+/uXwn/rK73G5tZB/Bl4JMxvvd17r7Q3RfOmDGj3m8tkrpappXSqBrUXJZqxQ6FGgwAcyPP54THCqYCbwLuNLMngMOBZWZWsREi0ixqmVZKao1DoblcaqGa7pMgxcRdp1CLlcACM9uHIAzOILgnAwDuPkxwiSsAZnYn8KlKVx+JNJuS6x1iqGeNQ6XmskgxqVUK7j4KnAOsAB4Cbnb3tWZ2iZm9J63vK5JnNU8rXXgRD71h/6oqBzWXpRZpVgq4+3Jg+YRjF5c49+g0xyKSJ4Xf+mNXDWPBb/zVVA79fZMZGNr9oj41l6WcNHsKIlJGrWscKjWkdzaXdw8ENZelklQrBRGpbLeqIebeSsWqhmIrl43gDllauSxx1LR1dpa0zYW0uujtRenoGJ86KqWrv58Fd/wM0LbYUlrS21yISIOML4J7aB39l11acWppdHBwfCpJzWWpl0JBJMd2W+dQQmEq6dRnHyz6uprLEpdCQSTn4jakfft2Fv3yO9yw4vMc/dTO252ouSzVUKNZpEnEuYzVgNdsG2Lx724B4NED3qrmslRFlYJIEylUDZWmkyaPjXDBfTdyw0++wDs23N+g0UkrUCiINKG9zzu3YgPaaOw9G6Q1KBREmlDvKacw8JHzeGbPvah0UXkebgUqzUOhINKElq4e4JxNe/PB4z7DFYecyfbO7opfo6pB4miPRvOam+Fnl8DwBujZKzi27fldH/fOgQXHw6M/KX9eHh73zoFjL4YDT0vm70eaTnQH1DvnHgLA2ev+k723DZW9T65v386mq65uyH2hpTm1/ormNTfDbZ+AkRZbvNPRDXtMzU9QKbQaap8Lby86bfSOp+7jH9f+G769/N1tu/r72fu8cxUObSTuiubWD4Wr3gTDT1U+T5JVT2gpVCo69As/ZdOLL+92fHZfD8vfuCXW7qs2eTKzPneJgqFNKBQKlvRBxVac5E+4jVvPnwVPFSJA0Eu44scPMzhcfAfU6N3Uhm+7jY2fvVhVgwDxQ6H1ewq9c1QpNKUwyLc9t/NQqcfDT8Gti+DWvysdIi0QHNXugBr3ng313N1NWk/rVwqt2lOQGhSpPpooLOrZAfXRY46tOJ0U3W1VWo92SS048DQ45SvQOxew4AOh5892f9w7FxZ+pPJ5eXncOanhf5XNL1J9bHsueD78FCz9OFy+TzDVePk+Ox9f9abgl4qcqGcH1DiL3aK7rUr7av1KoVXFucw2y8evvARjryT/czdcWF30zs2soli6eoArVjzMYJE7qUH8eyWM36dBDei2pEazZKvm0HqOnTPledP46adifYSoic3lONSAbk8KBWleVQVKHkIkvWqiVB8B6ru9pqqG9qNQkPZRMUQaGRzJBkSpRWoG/OGyk+t6b1ADup0oFESiygVHav2P2qablq4e4MoV6xkc2lYyxpK657KmktqHQkEkrqKBkWZ1UbqaqNRDgNr6COVoKqk9KBRE6jUeFk/RqIA4cvn0oj2ETjN2uNNfRx+hkjhVg6aSmpdCQSRJDagmdgDmMODTuWL0NJbteOv4a0n1ECqJUzVoKqk5KRREGiGlamKrT+LCkY+OB0NSPYS4KjWgNZXUfLSiWaQRDjwNzvs9LBmG910XrogHyt7VoLJX2Stc0/1/+dWkT/D+Sfdw/gn71T/WKlRaAa27ubUuVQoiaUhwumnMuuicPK3hezWpAd1aNH0kkkeR6San1nqisVtvaC1Da9D0kUgehdNNS09dx/k7zmHDjunscNhR1e9m4cmFLcOX9Ka6eZ8202svrX8/BZEcunLFegZeeQu38BYA3tPxKy7oupnZ9gxWVfkQCYjbPhE8Trhy0H0Z2kuq00dmdiJwDdAJfNPdL5vw+v8CPgqMApuBv3X3J8u9Zy3TR7c/fjvX3H8NT295mmmTpmFmDL88vMvjmXvO5Kg5R3HXhrvKnpeHx3kf68w9Z7L44MWcvG/6l1A2m8Jq5VL7GZ3a8Suu2fP6+u7/keK0UqwV0J2dsGMHXbNm6dLVHMm8p2BmncAjwHHABmAlcKa7r4uc8w7gN+6+1cw+Bhzt7qeXe99qQ+H2x29nyT1L2D5Wfhm/JKvLupgyaYrCJiLOauXZfT3cfdIzuzapa9qGI72+Q9wGNKgJnSd5CIUjgCXufkL4/CIAd7+0xPkHAV919yPLvW+1oXD8LcezccvG2OdLc+id1Nt0gVJux1Mos31FvWshunuCG00lXDnEaUCDmtB5kYdQeD9wort/NHz+QeAwdz+nxPlfBZ52988XeW0RsAhg3rx5hzz5ZNkZpl0c+J0D8VzuzS9ZiQbKxKm4NMOj1I6nUMU22PUERMJVQ9zN9ECroPOgqULBzD4AnAO83d1fLve+qhSkUSaGR61hUamPUPNq5V0CIq5kp5XGp5I2boSODhgrPTWmqaRs5SEUYk0fmdk7gf9DEAibKr2vegqStVJTV8WCI407p+1mzc3BlUfVNqcTnlbSNtz5lodQ6CJoNB8LDBA0ms9y97WRcw4CbiGoKB6N8766+qg5xrp1dCsjO0aq+vfUSmbtOYvFBy/mizf3pHLntN3kZFpJq6DzK/NQCAdxEnA1wSWp33b3L5jZJcAqd19mZj8FDgAK8zt/dPf3lHtPrWhuDnGCuJrHQy8PZf0jVc+Dj2Yf6wEM69yKj/Tx8uYTGHvhoPR2Pc3BtFI1TWhVDY2Ri1BIg0KhfdUaNHkLFPfgI7h3j2R6FiVlOK1UTRNaVUNjKBREIkoFSnQqLi8XJBSmnhIJiAynlapZzwCqGtKmUBCpQbHwyLLSSC8g4qp/WklVQz4oFEQSVGnqamj7ULDraX23USgqujo8kWmmDKaVqqkatNgtHQoFkQaIrkHomraaPWaswLqDyiKNgIiqax1FRtNKumw1OwoFkZSVW4MQDQjb8SpeNamLrWMvNGRcVU85NXhaSZetZkOhIJKSSiuUC4otTGt0z6KqgGjwtJKqhsZSKIikIM5Op1D9wrRCWKR5BVSsgGjwtJKqhsZRKIikoNJOp1DHXkahidVEGqvDqw+ImDq6YY+pVd9PWovd0qdQEElQPVNG9Uq7iqgYELVOK0HsqaVqLlstUEBUR6EgkpC0poxqkXZPomRA1HtPhxhTS9UudgNNK1VDoSBSpyyrg2qlUU3EC4hqxLtiSVVDOhQKInXIU3VQrYYFRIrTSqoakqdQEKlB3OoA6m8oN0LqATFeNdR4P2lVDQ2jUBCJKRoEcWfL8zBlVK2GVBApLITbpWowC7aZrfSOqhp2o1AQiSHuNFFUHqeMqtWQCqKmqaUqAiIGVQ07KRRESihUBoND2+gwYyzm/wPNWB3EkWRA7LJ5X/c0Fj8/xMmba7xiqUxAaFqpegoFkSJqqQygNaqDONJaEzGruzcSEDUo0piupRlNVxedU6YwNjxM16xZbRUSCgWRiGoayFGtWh3EkVpAjI6x+LnnOXnL1uq/OKGqoaCdeg8KBWl7tTSQYedER7tUB3EkHhDh586s0bGggqgqIHafVqqpaohoh6klhYK0pVqDoNOMHe70KwgqynNADD/ZU3PVUNCqAaFQkLZRaxAUtPMUUb0S37wvgYAY3jSbTWumMfrMcOxLWEtppYBQKEhLqzcICjRFlKxEq4gkA+LZF7DeXtiyBR+pLbSaPSAUCtJykgoCUHXQCHkLCHrnMjzpVDb96Nc19x4KmjEgFArSEpIMAjWQs5NGQPSO7cAMhjs6mBk3LML7PQw/tI2NK/vwsfpvpN0sAaFQkKalIGhtqd0fospqYviJHjatmcro1s7wSP0BYX19dEAu10EoFKSpKAjaU9oBEbeaSCMgovJQTSgUJJeiW0z09nRjBs9vHVEQSEPuU921YwdT3Bnu6GBaicBIOyCyqiYUCpKptD78oxQErasRAbGbItXFyWvGOPMXY3Rv6QhPSjYgorr6+5ny9qN46Rd3MbpxY+KBoVCQVBT7sB/aOrLb4y2vjDIylvx/WwqC9pNJQESFn5HHPTjG+34Je70Q/HeYXjzsLonqQqEgVav0gZ/0b/pxKQikIO17VMd15NoxzrrTeXUGAQG17dmkUGhTcX+Tz8sHfikKAqlGltVENCBemgwYTNmWflh09fez4I6fxT5foZATtX5Il3vc39fDO14/g58/vDnVOftGUxBIEopVE8MvDyezDUcVUq8mDN7w0EPxT89DKJjZicA1QCfwTXe/bMLrewDfBQ4BngVOd/cnyr1nLaEQ54O51AdtPR/gzf4hnZbC30nfhL9/BYGkLavpp7jVhBM/PEamwoErmygUzKwTeAQ4DtgArATOdPd1kXM+Dhzo7v/TzM4A/srdTy/3vtWGQq03VZH6dHcYUyZ3FQ1fffhL3uQhLJ6dBqteCwsfo2J1sb0Lbj4eLv1yc4XCEcASdz8hfH4RgLtfGjlnRXjOvWbWBTwNzPAyg6o2FI687I6qb6wi5RX7TV8f/NKKyk1FpR0cpaqLZ6fBD4827tm/gzVn/z72+8UNha56Bl3BbCB6770NwGGlznH3UTMbBl4NPBM9ycwWAYsA5s2bV9UgBhUIVdEHvshOJ+97Mifve3LZc9IKjrvf2Mndbyz9+qzu3pret5I0QyEx7n4dcB0ElUI1X9vf19N2lUKlD/ZyfRV94ItUJ4ngGH55mJl7zuSoOUdx14a7Kk5jTbZuFh9+USo/T5qhMADMjTyfEx4rds6GcPqol6DhnJjzT9gv855CrR/StTTF9cEukj9xgqOcaKjM3HMmiw9eXNf7lZNmKKwEFpjZPgQf/mcAZ004ZxnwIeBe4P3AHeX6CbUofDhmcfWRPqRFJAn1hko1UguFsEdwDrCC4JLUb7v7WjO7BFjl7suAbwHfM7PHgOcIgiNx7z1otj6URURiSLWn4O7LgeUTjl0cebwd+G9pjkFEROLrqHyKiIi0C4WCiIiMUyiIiMg4hYKIiIxrul1SzWwz8GTW46jBdCas1G4T7fhz62duH830c/+5u8+odFLThUKzMrNVcfYdaTXt+HPrZ24frfhza/pIRETGKRRERGScQqFxrst6ABlpx59bP3P7aLmfWz0FEREZp0pBRETGKRRERGScQiEDZvZJM3Mzm571WNJmZlea2cNmtsbM/s3M+rIeU5rM7EQzW29mj5nZhVmPJ21mNtfMfm5m68xsrZktznpMjWJmnWa22sz+I+uxJEmh0GBmNhc4Hvhj1mNpkP8C3uTuBwKPAOncLioHzKwTuBZ4F7A/cKaZ7Z/tqFI3CnzS3fcHDgf+vg1+5oLFwENZDyK8PLoTAAAC6ElEQVRpCoXGuwq4gOBmbC3P3X/i7qPh018T3IGvVR0KPObuj7v7K8BNwKkZjylV7r7R3e8PH79I8CHZ8jcvMbM5wMnAN7MeS9IUCg1kZqcCA+7+QNZjycjfAv+Z9SBSNBt4KvJ8A23wAVlgZvOBg4DfZDuShria4Je7HVkPJGmp3mSnHZnZT4GZRV76DPBpgqmjllLuZ3b3fw/P+QzBVMMPGjk2aQwzmwL8CDjX3V/IejxpMrN3A5vc/T4zOzrr8SRNoZAwd39nseNmdgCwD/CAmUEwjXK/mR3q7k83cIiJK/UzF5jZ2cC7gWOTvgd3zgwAcyPP54THWpqZdRMEwg/c/dasx9MARwLvMbOTgMnANDP7vrt/IONxJUKL1zJiZk8AC929WXZYrImZnQh8GXi7u2/OejxpMrMugmb6sQRhsBI4y93XZjqwFFnwG853gOfc/dysx9NoYaXwKXd/d9ZjSYp6CpK2rwJTgf8ys9+Z2dezHlBawob6OcAKgobrza0cCKEjgQ8Cx4T/fn8X/gYtTUqVgoiIjFOlICIi4xQKIiIyTqEgIiLjFAoiIjJOoSAiIuMUCiJlmNk9KbznfDM7K+n3FUmCQkGkDHd/SwpvOx9QKEguKRREyjCzl8J/Hm1md5rZLeH9IX4QrubFzJ4wsyvM7EEz+62ZvTY8foOZvX/iewGXAW8LF3qd1+ifSaQchYJIfAcB5xLcK2FfgtW8BcPufgDBCu6rK7zPhcAv3f0v3P2qVEYqUiOFgkh8v3X3De6+A/gdwTRQwY2Rfx7R6IGJJEWhIBLfy5HHY+y6y7AXeTxK+P+YmXUAk1IdnUgCFAoiyTg98s97w8dPAIeEj98DdIePXyTYJFAkd3Q/BZFk7GVmawiqiTPDY98A/t3MHgB+DGwJj68BxsLjN6ivIHmiXVJF6tQu98aQ9qDpIxERGadKQURExqlSEBGRcQoFEREZp1AQEZFxCgURERmnUBARkXH/H9eqGJ6BiNEFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cx.plot_f(f, frange=(-5, 5, 0.1), xlabel=\"input\", ylabel=\"output\", title=\"softmax()\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## elu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function elu in module conx.activations:\n",
      "\n",
      "elu(x, alpha=1.0)\n",
      "    Exponential Linear Unit activation function.\n",
      "    \n",
      "    See: https://arxiv.org/abs/1511.07289v1\n",
      "    \n",
      "    def elu(x):\n",
      "        if x >= 0:\n",
      "            return x\n",
      "        else:\n",
      "            return alpha * (math.exp(x) - 1.0)\n",
      "    \n",
      "    >>> elu(0.0)\n",
      "    0.0\n",
      "    >>> elu(1.0)\n",
      "    1.0\n",
      "    >>> elu(0.5, alpha=0.3)\n",
      "    0.5\n",
      "    >>> round(elu(-1), 1)\n",
      "    -0.6\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(elu)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "elu(0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VOXZ//HPlQRCWANNQHZQEVFQlrhrXZ/i1uL2s+BSH2sLUm3tpg/a9tHaPtWqtbW22lK1WkCsK2prxa3W1j0hsokoIiJBIIhhzZ7r98dMdBgmMFlmzizf9+uVV2buuXPOdQgzV+5zn/s65u6IiEj2ygk6ABERCZYSgYhIllMiEBHJckoEIiJZTolARCTLKRGIiGQ5JQKROJjZdWY2uxX9DzCzUjOz8PM3zOzAxEUo0nZKBCKJ8TPgFv98oc4twPUBxiPSIiUCkQ5mZv2B44F5Ec1PAMeb2V7BRCXSMiUCkQhmNsDMHjGzSjP7wMy+E6PPcWa2JqptlZmdFH76X8ACd69pfj38uAyYmMj4RdoiL+gARFKFmeUATwKPA1OAQcBzZra8lZsaA8T6mWXAwe0KUiQBNCIQ+dwhQLG7X+/ude6+EvgTMLmV2ykEtsZo3xp+TSSlaEQg8rmhwAAzq4poywX+DXzYiu18CvSI0d4DqIrRLhIojQhEPvcR8IG7F0Z89XD3U6P6bQe6Nj8xs1ygOOL1RcB+MbY/CljY0UGLtJcSgcjn3gC2mtn/mFmBmeWa2WgzOySq37tAFzM7zcw6AT8G8iNefxYYb2ZdmhvCjyeEXxNJKUoEImHu3gicDowFPgA2AncBvaL6bQa+FX6tgtAIYU3E6+uBF4BJET/2ZeBFd1+bwEMQaRPTjWlEOp6ZHQDcBxzq7m5mrwOXuPuSgEMT2YUSgYhIltOpIRGRLKdEICKS5ZQIRESyXFosKCsqKvJhw4YFHYaISFopKyvb6O7Fe+qXFolg2LBhlJaWBh2GiEhaMbO4VsTr1JCISJZTIhARyXJKBCIiWU6JQEQkyykRiIhkuYQlAjO7x8w2mNkutVXM7Adm5mZWlKj9i4iks3nlFRx14wsMn/F3jrrxBeaVVyRsX4kcEdwLnBzdaGaDgS8BqxO4bxGRtDWvvIKrH11MRVU1DlRUVXP1o4sTlgwSlgjc/SVgU4yXfg1cBajanYhIDDfPX051feNObdX1jdw8v7W3z45PUucIzGwSUOHue7xLk5lNNbNSMyutrKxMQnQiIqlhbVV1q9rbK2mJwMy6AtcA/xtPf3ef6e4l7l5SXLzHFdIiIhmjZ0Hsog8DCgsSsr9kjgj2AYYDC81sFTAIWGBmeyUxBhGRlPbayk/YUt1Aju3cXtAplysnjkzIPpOWCNx9sbv3dfdh7j6M0K39xrv7umTFICKSytZWVXPZnAUML+7GL84czcDCAgwYWFjADWeN4YxxAxOy34QVnTOzucBxQJGZrQGudfe7E7U/EZF0VlPfyPTZZdQ2NDHzwhL27dudyYcOTcq+E5YI3H3KHl4flqh9i4ikE3fnJ/OWsHDNZv544QT27ds9qfvXymIRkYDNfn01D5Wt4Tsn7MvEA5M/bapEICISoDdXbeKnTyzl+JHFfPek/QKJQYlARCQg6zbXMH32Agb1LuA3k8eRE32pUJKkxR3KREQyTW1DI9PnlLGjroH7v3kYvQo6BRaLEoGISJLMK6/g5vnLWVtVTUHnXHbUNXLH+ePZr1+PQONSIhARSYLmQnLNNYR21DWSl2PUNTQFHJnmCEREkiJWIbmGJk9YIbnWUCIQEUmCZBeSaw0lAhGRJOjfq0vM9kQVkmsNJQIRkSQYXtRtl7ZEFpJrDSUCEZEEe7D0I15+/xOOH1mctEJyraGrhkREEmjhR1X8eN4Sjtr3C/zpayXk5abe39+pF5GISIbYuK2WS2eXUdw9n9unjE/JJAAaEYiIJER9YxOXzVnApu11PDL9SPp06xx0SC1SIhARSYBfPLWM1z/YxK3nHszogb2CDme3UnOcIiKSxh5dsIY/v7yKi48axlnjBwUdzh4pEYiIdKAlFZu5+tHFHDa8D9ecOirocOKSsERgZveY2QYzWxLRdrOZvWNmi8zsMTMrTNT+RUSSbdP2OqbNKqNPt878/vzxdErRyeFoiYzyXuDkqLZngdHufhDwLnB1AvcvIpIU88orOPLG5xn/s2epqKpmyqGDKeqeH3RYcUtYInD3l4BNUW3PuHtD+OlrQOqfPBMR2Y3mqqJrq2o+a7vzxZXMK68IMKrWCXLc8nXgHwHuX0Sk3WJVFa2ub0yJqqLxCiQRmNmPgAZgzm76TDWzUjMrraysTF5wIiKtkMpVReOV9ERgZv8NnA6c7+7eUj93n+nuJe5eUlxcnLT4RETiVbWjrsX7DKdCVdF4JXVBmZmdDFwFHOvuO5K5bxGRjtTY5Hx7bjm4k5+XQ23EncZSpapovBKWCMxsLnAcUGRma4BrCV0llA88a2YAr7n7pYmKQUQkUX71zHL+/d5GbjhrDAWdcj+7F/GAwgKunDgyJaqKxithicDdp8RovjtR+xMRSZanFn/MHS++z5RDhzDl0CEAafXBHy09VjuIiKSId9dv5YcPLWTckEKu+8oBQYfTIZQIRETitLm6nql/KaVbfh5/uGAC+Xm5QYfUIZQIRETi0NTkfPeBctZ8Ws2d54+nX8/Y9yBOR0oEIiJx+M1z7/LP5ZVc+5UDKRnWJ+hwOpQSgYjIHjyzdB2/fWEF55YM4oLDhgQdTofTjWlERGKYV17x2SWhAIN7F3D9pNGEL33PKBoRiIhEaS4kV1FVjQMObNhay9NL1gUdWkIoEYiIRIlVSK62oSmtCsm1hhKBiEiUTCgk1xpKBCIiUfp06xyzPZ0KybWGEoGISISVldvYVlNP9JxwuhWSaw0lAhGRsG21DUybVUbX/Dx+ctooBhYWYMDAwgJuOGtMWtcT2h1dPioiArg7Vz60kPcrtzH7ksM4ct8ivn703kGHlRQaEYiIAHf+633+sWQd15w6iiP3LQo6nKRSIhCRrPfi8g3cPH85Xzl4AJccPTzocJJOiUBEstqHn2znO3PL2X+vnvzy7IMycuXwnigRiEjW2lEXmhzOyTFmXjiBgs6ZUVa6tZQIRCQruTtXPbyId9dv5fYp4xjcp2vQIQUmkfcsvgc4Hdjg7qPDbX2AvwLDgFXAue7+aaJiEBGJ1lxMriK8SvjLB/XnmBHFAUcVrESOCO4FTo5qmwE87+4jgOfDz0VEkiKymFyz55atZ155RYBRBS9hicDdXwI2RTVPAu4LP74POCNR+xcRiRarmFx1feYWk4tXsucI+rn7x+HH64B+LXU0s6lmVmpmpZWVlcmJTkQyWrYVk4tXYJPF7t5c5rul12e6e4m7lxQXZ/f5OxFpP3enS6fYVwVlajG5eCU7Eaw3s/4A4e8bkrx/EclS97y8iur6RvJydl4nkMnF5OKV7ETwBHBR+PFFwONJ3r+IZKFX3/+EXzy1jIkH9uPmsw/KmmJy8Urk5aNzgeOAIjNbA1wL3Ag8aGaXAB8C5yZq/yIiABVV1Vx+/wKGF3XjV+eOpXt+HmdOGBR0WCklYYnA3ae08NKJidqniEikmvpGLp1VRl1DE3+8cALd81VwORb9q4hIRnJ3fvTYEhZXbOaur5WwT3H3oENKWSoxISIZadZrH/LIgjVcceIITjqgxSvVBSUCEclAb3ywieuffJsT9+/LFSeOCDqclKdEICIZ5ePN1XxrThlD+nTl15PHkpOTfWWlW0tzBCKS9poLya2tqiYv1zBg7jcPp2eXTkGHlhY0IhCRtBZZSM6B+sZQwYKla7cEG1gaUSIQkbQWq5BcXaNnfSG51lAiEJG0pkJy7adEICJprV/PLjHbs72QXGsoEYhI2qpraKJLp10/xlRIrnWUCEQkbV3/t6Ws+mQHFx0xVIXk2kGXj4pIWvrrm6uZ/dpqph27N1efMoqfTgo6ovSlEYGIpJ23PqriJ/OWcsyIIq6auH/Q4aQ9JQIRSSuVW2u5dFYZfXvm89vJ48jVyuF206khEUkb9Y1NXDZnAVXVdTwy/Uh6d+scdEgZQYlARNLG//19GW+s2sRtk8dy4IBeQYeTMXRqSETSwsNla7j3lVVccvRwJo3VFUEdKZARgZl9D/gG4MBi4GJ3rwkiFhFJXZHF5BzYp7gbV5+iyeGOlvQRgZkNBL4DlLj7aCAXmJzsOEQktUUXk4PQ/Yf/tujjQOPKREGdGsoDCswsD+gKrA0oDhFJUbGKydXUN6mYXAIkPRG4ewVwC7Aa+BjY7O7PRPczs6lmVmpmpZWVlckOU0QCpmJyyRPEqaHewCRgODAA6GZmF0T3c/eZ7l7i7iXFxcXJDlNEAlbYNfZNZVRMruMFcWroJOADd69093rgUeDIAOIQkRS1dO1mttbUE71WTMXkEiOIRLAaONzMupqZAScCywKIQ0RS0Kfb65g2q4yi7l247isHqphcEiT98lF3f93MHgYWAA1AOTAz2XGISOppaGzi23PL2bCllr9OO5xxQ3rztSOGBR1WxgtkHYG7XwtcG8S+RSR13fzMcv6zYiO/PHsM44b0DjqcrBHXqSEzuyKeNhGRtvrborX88V8rOf+wIXz1kCFBh5NV4p0juChG2393YBwiksXeWbeFKx9axIShvbn2ywcGHU7W2e2pITObApwHDDezJyJe6gFsSmRgIpIdNu+oZ9qsMnp0yePO88fTOU8l0JJtT3MErxBa9FUE/CqifSuwKFFBiUh2aGxyrvhrOWurqnlg6uH0beFG9JJYu00E7v4h8CFwRHLCEZFMF1lIrnt+HltrG/j5GaOZMLRP0KFlrbiuGjKzrfBZ3afOQCdgu7v3TFRgIpJ5mgvJNdcQ2lrbQG6O0a1zbsCRZbe4Tsa5ew937xn+4C8AzgbuSGhkIpJxYhWSa2xybnnm3YAiEmjDymIPmQdMTEA8IpLBVEguNcV7auisiKc5QAmgG8mISKv0L+zC2qpdPzpUSC5Y8a4s/nLE4wZgFaEKoiIicTtoYK9dEoEKyQUvrkTg7hcnOhARyWzPvr2ep5eu55ChvVm7uZq1VTUMKCzgyokjVUguYPGeGtobuA04nNDVQ68C33P3lQmMTUQyxPuV2/j+X99izMBezPrGYXTppKuEUkm8k8X3Aw8C/QndTOYhYG6ighKRzLG1pp6pfymlU14Of7hwgpJACoo3EXR191nu3hD+mg1oCaCI7FZTk/PDhxay6pMd/O68cQzUpHBKiney+B9mNgN4gNCpoa8CT5lZHwB3V90hEdnFnf96n/lL1/Pj00Zx5D5FQYcjLYg3EZwb/j4tqn0yocSwd4dFJCIZ4Z/LN3DLM8uZNHYAlxw9POhwZDfiTQSj3H2na77MrEt0m4gIwKqN27libjn779WTG886iNBdaSVVxTtH8EqcbSKS5bbXNjBtVhk5OcbMCydQoDpCKW9P9yPYCxgIFJjZOKA5rfcEurZ1p2ZWCNwFjCZ0aunr7v5qW7cnIsELVRV9h4rwgrFLj92bwX3a/DEhSbSnU0MTCd2JbBBwa0T7VuCaduz3NuBpdz/HzDrTjqQiIsGLrioKcN8rH7L/Xj21WCwN7Ol+BPcB95nZ2e7+SEfs0Mx6AV8kfKtLd68D6jpi2yISjFhVRavrG7l5/nIlgjQQ72TxaDPb5Uai7n59G/Y5HKgE/mxmBwNlwBXuvj2yk5lNBaYCDBmiG1mLpDJVFU1v8U4WbwO2h78agVOAYW3cZx4wHrjT3ceFtzkjupO7z3T3EncvKS4ubuOuRCTRqusayc2JfVWQqoqmh3iLzkXerxgzuwWY38Z9rgHWuPvr4ecPEyMRiEjqc3dmPLqIhianc24OdY1Nn72mqqLpo9U3pgnrSmgCudXcfR3wkZk1/w85EXi7jXGISIDu/s8HPP7WWq6cOJKbzjmIgYUFGDCwsIAbzhqj+YE0EW/10cV8fs/iHKAv8LN27PfbwJzwFUMrAZW5Fkkzr6zYyA3/eIeTD9yLbx23D2amD/40Fe9k8elAb+AYoBB4yt3L2rpTd3+L0F3ORCQNrfl0B5fPLWd4UTduOfdgrRxOc/GeGpoEzAKKgE6Ervj5dsKiEpGUVVPfyKWzy6hvaGLmhRPonh/v35OSquL9DX4DOLz5Ek8z+yWhm9PcnqjARCT1uDvXPLaYJRVbuPuiEvYu7h50SNIB4h0RGKHLRps18nm5CRHJEn959UMeXVDBd08awYmj+gUdjnSQeEcEfwZeN7PHws/PAO5OTEgikopeX/kJP/vb25w0qi/fOWFE0OFIB4p3HcGtZvYicHS46WJ3L09YVCKSEkKF5JaztqoaM/hCt87c+tWx5LSwgEzSU9yzPO6+AFiQwFhEJIVEF5Jzhy01DbywbIMuE80wbV1QJiIZLlYhudqGJm6evzygiCRRlAhEJCYVksseSgQiEtMXuneO2a5CcplHiUBEdrF+Sw219Y27XCOuQnKZSYlARHZS29DI9NllNDpcdfJIFZLLAlobLiI7+emTb7NgdRV3nD+eU8f0Z/px+wYdkiSYRgQi8pkH3ljN/a+vZvpx+3DqmP5BhyNJokQgIgCUr/6U/318KceMKOKHX9I8QDZRIhARNmytYfrsBfTrlc/tU8a1eOtJyUyaIxDJcnUNTVw2ZwFV1XU8Ov0oCrvGvmxUMpcSgUiW+7+/v82bqz7ltsljOWBAz6DDkQAElgjMLBcoBSrc/fSg4hDJRs3F5CrCq4SPH1nEpLG6LDRbBTlHcAWwLMD9i2Sl5mJyFRGlIl5duYl55RUBRiVBCiQRmNkg4DTgriD2L5LNYhWTq6lXMblsFtSI4DfAVUBTSx3MbKqZlZpZaWVlZfIiE8lwKiYn0ZKeCMzsdGCDu5ftrp+7z3T3EncvKS4uTlJ0Ipmva35uzHYVk8teQYwIjgK+YmargAeAE8xsdgBxiGSdeeUVbK9t3GWdgIrJZbekJwJ3v9rdB7n7MGAy8IK7X5DsOESyzZKKzcx4dBGHDe/DTWePUTE5+YzWEYhkgU3b65g2q4zeXTvzu/PGU9wjn7MnDA46LEkRgSYCd38ReDHIGEQyXUNjE9+eu4DKbbU8NO0IinvkBx2SpBjVGhLJcDfPX87LKz7h52eM5uDBhUGHIylIiUAkgz25cC1/fGklFx4+lHNLdCpIYlMiEMlQ76zbwlUPL6JkaG9+cvoBQYcjKUyJQCQDVe2oY+pfyujRJY87zh9P5zy91aVlumpIJEM0F5JbW1VN57wc6hubeOjSI+nbs0vQoUmK058JIhkgspCcA7UNTeTmGB9t2hF0aJIGlAhEMkCsQnL1ja5CchIXJQKRDKBCctIeSgQiGWCvXrHnAVRITuKhRCCS5pqanN5dO+3SrkJyEi8lApE095vn3+Ptj7dy9viBKiQnbaLLR0XS2DNL1/Hb59/jnAmDuPmcgzCzPf+QSBSNCETS1IoN2/j+gws5aFAvfn7GaCUBaTMlApE0tLWmnmmzSsnPy+EPF0ygS6fYdx0TiYdODYmkmaYm5wcPLmTVJzuYfclhujJI2k0jApE08/t/ruCZt9fzo1NHccQ+Xwg6HMkASgQiaeSFd9Zz63Pvcua4gVx81LCgw5EMkfRTQ2Y2GPgL0A9wYKa735bsOETSRWQxOYABhV34xZljNDksHSaIEUED8AN3PwA4HLjMzFQsXSSG6GJyDmzcVsf8peuCDk0ySNITgbt/7O4Lwo+3AssArXoRiSFWMbnahiYVk5MOFegcgZkNA8YBr8d4baqZlZpZaWVlZbJDE0kJKiYnyRBYIjCz7sAjwHfdfUv06+4+091L3L2kuLg4+QGKpIA+3TrHbNclo9KRAkkEZtaJUBKY4+6PBhGDSKpb/ckOdtQ1ED0lrGJy0tGSnggsdKnD3cAyd7812fsXSQc76hqYOquUznm5/Oi0USomJwkVxMrio4ALgcVm9la47Rp3fyqAWERSjrvzP48sZvn6rdx78aEcu18x3zhm76DDkgyW9ETg7v+BXUa7IhJ2178/4MmFa7nq5JEcu5/mxyTxtLJYJIX8572N3PCPZZw6Zi+mH7tP0OFIllAiEEkRH23awbfnLmDfvt25+ZyDtXJYkkaJQCQF1NQ3cunsMhqanD9eWEK3fBUGluTR/zaRgLk7Vz+6mLc/3sLdF5UwvKhb0CFJltGIQCRg976yisfKK/jeSftxwv79gg5HspBGBCIBiKwo6sDoAT25/Ph9gw5LspRGBCJJFl1RFGBF5TaeWLg20LgkeykRiCRZrIqiNfWqKCrBUSIQSTJVFJVUo0QgkmQ9CzrFbFdFUQmKEoFIEr25ahNbquvJiVorpoqiEiQlApEkWbe5humzFzCsqBs/P3O0KopKytDloyJJUNvQyPQ5Zeyoa+D+bx7Gfv16cN6hQ4MOSwRQIhBJiuueeJvy1VXcef549uvXI+hwRHaiU0MiCXb/66uZ+8ZqvnXcPpwypn/Q4YjsQolAJIHKPvyUa59Ywhf3K+YHX9JksKQmJQKRBNmwpYbps8vo36uA304eS270pUIiKUKJQCQB6hqa+NacBWytaeCPF06gsGvnoEMSaVEgk8VmdjJwG5AL3OXuNwYRh0hHay4mVxFeJfy1I4Yyqn/PgKMS2b2kjwjMLBf4PXAKcAAwxcwOSHYcIh0tsphcs4dK1zCvvCLAqET2LIhTQ4cCK9x9pbvXAQ8AkwKIQ6RDxSomV13fqGJykvKCSAQDgY8inq8Jt+3EzKaaWamZlVZWViYtOJG2qG9s2mkkEEnF5CTVpexksbvPdPcSdy8pLi4OOhyRFm3cVssFd73e4usqJiepLohEUAEMjng+KNwmknbe+qiKL9/+H976qIrzDx9MQafcnV5XMTlJB0FcNfQmMMLMhhNKAJOB8wKIQ6Rd/vrman4ybynFPfJ5ZPqRjB7Yi0OGfuGzW1AOKCzgyokjVUxOUl7SE4G7N5jZ5cB8QpeP3uPuS5Mdh0hb1TY08tMn3+b+11dz9L5F3D5lHL27hdYJnDFuoD74Je0Eso7A3Z8Cngpi3yKtFXmj+X49u5CfZ3y4qZpLj92HKyeO1IphSXuqPiqyG81rA5ovC123pQaAi44YyoxT9g8yNJEOk7JXDYmkgpvmv7PL2gCA55ZtCCAakcTQiECyVuQpn8iJ3cqttfzr3Ur++c4G1lbVxPxZrQ2QTKJEIBmlpQ/3WP0iT/lUVFXzw4cW8qtnlvPRp6EP+b498unaOZcddbuOCLQ2QDJJxiaCeD8QWtNX20ztbcb6cL/60cUATBo7gKod9Xy8uYb1W2q47omlu5zyaWhy1m2p4Qf/tR/H79+XAwf05PG31u60TdDaAMk85u5Bx7BHJSUlXlpaGnf/6A8ECL15Y90gPN6+2mZyt+nuNDY5DU3O4+UVXPvkUmrqmz7rm5+Xw7Rj9+aQYX2ormukur6Rax9fSlV1PdFyc4y8HKO2oWmX16IZ8MGNp+1yTFobIOnIzMrcvWSP/TIxERx14wsx677k5RjDi7rt1PbBxu00NO36bxDdN95+HbXNYVHbXBVn3931G/qFrju1ffjJjhb7Donou7qFfrk5xuDeBTS/4h76K7wxRt8cg+Ie+TR5qN+m7bXE6PZZ35Zea6upX9ybvXp2Ya9eoa/ps8tYv6V2l34DCwt4ecYJHbtzkYDEmwgy8tRQSxN5DU3OiH7dd2p7b8O2uPrG26+jtjky6gbnK+Lsu7t++0fVxX+/cnuLfQ+I6LuyhX6NTc7BgwuB0F/SAKs37YjZt8nh+JF9MQv1nvvG6pj9AC47fl9yLPRXfG6ucdPTLVfvfPjSI+jSKZeCzrmc96fXWvxwv+bUUTu1XX3KKJ3yEQnLyEQwoLAg5ohgYGEBd5w/Yae2lkYP0X3j7ddR2/z9+eN3ansrzr677Xde1DZXt9z3dxF9y3fT77bJ43Zqe3PVpy32vfHsgz57/tK7lS32i76375zXVrfYt2RYn8+et+bDvfnUjk75iGToOoIrJ46Mu/hXvH21zdTf5hnjBnLDWWMYWFiAEUoUseYxIvu/POMEPrjxNF6ecYKSgGStjBwRtOavvXj7apupv83mvvpAF2mdjJwsFhGR+CeLM/LUkIiIxE+JQEQkyykRiIhkOSUCEZEsp0QgIpLl0uKqITOrBD5s448XARs7MJxUkGnHlGnHA5l3TJl2PJB5xxTreIa6e/GefjAtEkF7mFlpPJdPpZNMO6ZMOx7IvGPKtOOBzDum9hyPTg2JiGQ5JQIRkSyXDYlgZtABJECmHVOmHQ9k3jFl2vFA5h1Tm48n4+cIRERk97JhRCAiIruhRCAikuUyNhGY2f8zs6Vm1mRmJRHtw8ys2szeCn/9Icg449XS8YRfu9rMVpjZcjObGFSM7WFm15lZRcTv5dSgY2oLMzs5/HtYYWYzgo6nI5jZKjNbHP69pF0ZYDO7x8w2mNmSiLY+Zvasmb0X/t47yBhbq4VjavN7KGMTAbAEOAt4KcZr77v72PDXpUmOq61iHo+ZHQBMBg4ETgbuMLPcXX88Lfw64vfyVNDBtFb43/33wCnAAcCU8O8nExwf/r2k43X39xJ6b0SaATzv7iOA58PP08m97HpM0Mb3UMYmAndf5u4t3+w2zezmeCYBD7h7rbt/AKwADk1udBJ2KLDC3Ve6ex3wAKHfjwTI3V8CNkU1TwLuCz++DzgjqUG1UwvH1GYZmwj2YLiZlZvZv8zsmKCDaaeBwEcRz9eE29LR5Wa2KDzsTauhelgm/S4iOfCMmZWZ2dSgg+kg/dz94/DjdUC/IIPpQG16D6V1IjCz58xsSYyv3f0V9jEwxN3HAd8H7jeznsmJePfaeDxpYw/HdyewDzCW0O/oV4EGK5GOdvfxhE55XWZmXww6oI7koWvoM+E6+ja/h9L6nsXuflIbfqYWqA0/LjOz94H9gMAnwdpyPEAFMDji+aBwW8qJ9/jM7E/A3xIcTiKkze+iNdy9Ivx9g5k9RugUWKy5t3Sy3sz6u/vHZtYf2BB0QO3l7uubH7f2PZTWI4K2MLPi5slUM9sbGAGsDDaqdnkCmGxm+WY2nNA9BytPAAACLElEQVTxvBFwTK0WfjM2O5PQ5Hi6eRMYYWbDzawzoUn8JwKOqV3MrJuZ9Wh+DHyJ9PzdRHsCuCj8+CLg8QBj6RDteQ+l9Yhgd8zsTOB2oBj4u5m95e4TgS8C15tZPdAEXOruHTbpkigtHY+7LzWzB4G3gQbgMndvDDLWNrrJzMYSGqKvAqYFG07ruXuDmV0OzAdygXvcfWnAYbVXP+AxM4PQ58X97v50sCG1jpnNBY4DisxsDXAtcCPwoJldQqjE/bnBRdh6LRzTcW19D6nEhIhIlsu6U0MiIrIzJQIRkSynRCAikuWUCEREspwSgYhIllMiEIlgZq8kYJvDzOy8jt6uSEdRIhCJ4O5HJmCzwwAlAklZSgQiEcxsW/j7cWb2opk9bGbvmNkcC6+qCtfnvylco/8NM9s33H6vmZ0TvS1Ci5eOCdeI/16yj0lkT5QIRFo2DvguoXsL7A0cFfHaZncfA/wO+M0etjMD+He4RvyvExKpSDsoEYi07A13X+PuTcBbhE7xNJsb8f2IZAcm0pGUCERaVhvxuJGda3N5jMcNhN9TZpYDdE5odCIdRIlApG2+GvH91fDjVcCE8OOvAJ3Cj7cCPZIWmUgrZWz1UZEE621miwiNGqaE2/4EPG5mC4Gnge3h9kVAY7j9Xs0TSKpR9VGRVjKzVUCJu28MOhaRjqBTQyIiWU4jAhGRLKcRgYhIllMiEBHJckoEIiJZTolARCTLKRGIiGS5/w/r71Krb8f2BgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cx.plot_f(elu, frange=(-15, 15, 1), xlabel=\"input\", ylabel=\"output\", title=\"elu()\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "elu(0.5, .3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8XHW9//HXJ93SPZSG0n0BoS3Q0hKhlEUU2YtFuAoUtIr+ylb1XhQteF1Qf1JZFK+oXDZBaZGl0KIFihYBERDTdKUL0JWmpUkpKV2SNsl87h8zqdPJJJlJMnNmeT8fjzwy8z3fOedzms585nu+53yOuTsiIpK/CoIOQEREgqVEICKS55QIRETynBKBiEieUyIQEclzSgQiInlOiUCyhpn90MweSaL/aDMrNTNLoO+XzOzVBNebcN9MYGYXmtljQcchmUuJQHLZj4E7PEsvljGzM81stZntNbO/mdnQZvr+zcwqzewjM1tqZpMblrn7n4BjzGxMWgKXrKNEIDnJzPoDnwTmBh1La5hZX+Ap4HtAH6AUaO5b/TeA/u7eC5gGPBL5N2jwaKRdpBElAsk4ZjbAzOZEvuGuN7Ovx+lzhpltjmnbYGafjjw9Cyhz95qo5TPMbK2Z7TKzlWb22WZicDP7upmtM7PtZna7mRXE9LnDzD6MxHheVPuXzWxVZDvrzOzqVvwzXAy85e5PRPbhh8BYMxsZr7O7L3P3uoanQCdgcFSXl4ALWhGH5AElAskokQ/bPwFLgYHAmcB/mtk5Sa7qOGBNTNta4DSgN3ALjb81x/osUAKMByYDV0UtOymy/r7AbcADUXMRFcAkoBfwZeAXZjY+sn9DzKyqmZ8pkXUcE/k3AMDd90TiP6apYM3sz2ZWA/yT8Ad/adTiVcAwM+vVzP5KnlIikEzzcaDY3X/k7vvdfR1wH3BZkuspAnZFN0S+XW9x95C7Pwa8A5zYzDp+5u473H0TcBdwedSyje5+n7vXAw8D/YF+ke3Md/e1HvYy8ALhBIS7b3L3omZ+ZkfW3wPYGRPPTqBnU8G6+6TI8vOBF9w9FLW44d+iqJn9lTylRCCZZigwIPpbMnAzkQ/ZJHxIzIemmX3RzJZErfdYwt/om/Je1OONwICo5+83PHD3vZGHPSLbOc/M3jCzHZHtnN/CduLZTXhEEa0XMcktlrvXuvtzwNlm9pmoRQ3/FlVJxiF5QIlAMs17wPqYb8k93f38mH57gG4NT8ysA1ActXwZcFTU8qGERxbTgUPdvQhYATR3amn0MfYhwJaWgjezLsAc4A6gX2Q7zzZsJ3JoaHczP1dEVvUWMDZqvd2BIyLtiegY6d9gFLDB3T9K8PWSR5QIJNO8Cewys++YWVcz62Bmx5rZx2P6vQ0UmtkFZtYJ+G+gS9TyvwDjzaww8rw74UnUSghP6BIeETTnRjM7xMwGEz4rJ5Fz8TtH4qgE6iKTyGc3LIwcGurRzM+sSNengWPN7JLIPnwfWObuq2M3aGYjI6OQrmbWycyuBE4HXo7q9gnguQTilzykRCAZJXLMfRJwPLAe2A7cT3iCN7rfTuC6yLJywiOEzVHLtwEvEp7kxd1XAncCrwPbCE8m/6OFcOYBi4AlwHzggQTi3wV8HXic8OGpKcAzLb0uznoqgUuA/x9Zz0lEzZOY2T1mdk/DU8JnFVUQTkDfAC5197KoVV4O/G+ycUh+sCy91kakRWY2mvBE7onJXlRmZg58zN3fTUlwaWRmFwJfcPfPBx2LZCYlApE4cikRiLREh4ZERPKcRgQiInlOIwIRkTzXMegAEtG3b18fNmxY0GGIiGSVRYsWbXf34pb6ZUUiGDZsGKWlpS13FBGRA8xsYyL9dGhIRCTPKRGIiOQ5JQIRkTynRCAikueUCERE8lzKEoGZPWhmFWa2Is6yb0ZuBZhsjXYRkbwwd3E5p8x8keEz5nPKzBeZu7g8ZdtK5YjgIeDc2MZISd+zgU0p3LaISNaau7icm55aTnlVNQ6UV1Vz01PLU5YMUpYI3P0VYEecRb8Avk24NryIiMS4fcEaqmvrD2qrrq3n9gWxt+FuH2mdIzCzyUC5uy9NoO80Mys1s9LKyso0RCcikhm2VFUn1d5WaUsEZtaN8L1nv59If3e/191L3L2kuLjFK6RFRHJG726d4rYPKOqaku2lc0RwBDAcWGpmG4BBQJmZHZ7GGEREMtqa93exu6aWgpi7aXft1IEbzzk6JdtMWyJw9+Xufpi7D3P3YYRvKzje3d9PVwwiIplsz746rp21iKJuXfjhhaMZWNQVAwYWdeXWi4/jonEDU7LdlBWdM7NHgTOAvma2GfiBu7d4z1cRkXzk7tz89HI2bN/DI189iYlH9OWLE4enZdspSwTufnkLy4elatsiItlm1j83MW/JFr519lFMPCK9l1jpymIRkYCtKN/Jj/60kk8cVcx1ZxyZ9u0rEYiIBGhndS3XzlrEoT0684tLj6cgdpY4DbLixjQiIrnI3bnxiaVsrarhsasn0Kd750Di0IhARCQgD7y6nhdWbmPGeSM5YWifwOJQIhARCcCijTuY+dxqzh7dj6+cmp6zg5qiQ0MiImkyd3E5ty9Yw5aqasygqFsnbv/cWMzSPy8QTSMCEZE0iK0oGnLYs6+ev62uCDo0JQIRkXSIV1F0X10oZRVFk6FEICKSBumuKJoMJQIRkTTo16swbnuqKoomQ4lARCTF6upDdO/SoVF7KiuKJkOJQEQkxe766zusrdzD5ScOTltF0WTo9FERkRT625oK7v7bu1xaMphbLx4TdDhxaUQgIpIiW6qqueGxJYw8vCe3TD4m6HCapEQgIpICtfUhps8uo7be+c0V4yns1HiOIFPo0JCISAr87LnVlG2q4u4p4xhR3CPocJqlEYGISDt7fsX73P/qeqaePJRJYwYEHU6LlAhERNrRpg/2cuOTSxk7qDc3XzAq6HASokQgItJOamrruW72IgrMuHvKeLp0zNx5gWipvHn9g8AkoMLdj4203Q5cCOwH1gJfdveqVMUgIpIODVVFyyPlIr562nAG9+kWcFSJS+WI4CHg3Ji2vwDHuvsY4G3gphRuX0Qk5aKrijaY9cYm5i4uDzCq5KQsEbj7K8COmLYX3L0u8vQNYFCqti8ikg7xqopW19ZnRFXRRAU5R3AV8FxTC81smpmVmllpZWVlGsMSEUlcJlcVTVQgicDMvgvUAbOa6uPu97p7ibuXFBcXpy84EZEEuXuTF4plQlXRRKX9gjIz+xLhSeQz3d3TvX0RkfbyROlmqmvr6Vhg1IX+/XGWKVVFE5XWEYGZnQt8G/iMu+9N57ZFRNrTqq0f8b15Kzj1yL7cdsmYjKwqmqhUnj76KHAG0NfMNgM/IHyWUBfgL5GbNb/h7tekKgYRkVTYVVPLdbPK6N21E3dddjx9e3Th4hOy99yXlCUCd788TvMDqdqeiEg6uDsz5ixn0469zP7qSfTt0SXokNpMVxaLiCThD29sZP7yrXzr7KM5acShQYfTLpQIREQStPS9Kn7855WcOfIwrj59RNDhtBslAhGRBOzcG54XOKxnIXd+fiwFBRZ0SO1G9yMQEWlBKOR884klVOyq4YlrJlLUrXPQIbUrjQhERFpw39/X8ddVFXz3/FEcP7go6HDanUYEIiJxNFQU3VJVjQNjB/Vi6sRhQYeVEhoRiIjEiK4o2nC98Jptu5m3ZEugcaWKEoGISIx4FUVrakNZVVE0GUoEIiIxcqGiaDKUCEREYvTpHv+soGyqKJoMJQIRkSjbPqqhprae2KsEsq2iaDKUCEREIurqQ3xt9mJCDjPOOzqrK4omQ6ePiohE3PHC27y5YQd3XXo8F40byNWfODLokNJCIwIREWDhqm3c8/Jappw0JGe/+TdFiUBE8t57O/Zyw+NLGd2/F9+fNDrocNJOiUBE8tr+uhDTZ5cRCjm/vXJ8k/cgzmWaIxCRvPbTZ1exdPNO7rlyPEMP7R50OIHQiEBE8tb8ZVt56LUNXHXKcM49tn/Q4QQmZYnAzB40swozWxHV1sfM/mJm70R+H5Kq7YuINGdd5W6+M2cZ44YUMeO8kUGHE6hUjggeAs6NaZsBLHT3jwELI89FRNJm7uJyJt66kE/d+TJ79tfxmbED6Nwxvw+OpGzv3f0VYEdM82Tg4cjjh4GLUrV9EZFYDVVFt+ysAcAdbnt+DXMXlwccWbDSnQb7ufvWyOP3gX5p3r6I5LF4VUWra+tztqpoogIbD7m7w4FS342Y2TQzKzWz0srKyjRGJiK5Kt+qiiYq3Ylgm5n1B4j8rmiqo7vf6+4l7l5SXFyctgBFJDft2VdHhyZuOJ+rVUUTle5E8AwwNfJ4KjAvzdsXkTzk7tz89HLqQt5oYjiXq4omKpWnjz4KvA4cbWabzewrwEzgLDN7B/h05LmISErNfnMT85Zs4ZtnHcVtl4zJm6qiiUrZlcXufnkTi85M1TZFRGKtKN/JLc+s5PSjirn+k0dSUGB5/8EfK79PnhWRnLazupbrZpVxaI/O3HXp8RQ0MUeQ71RrSERykrtz4xNL2VJVzWNXT2jy9pOiEYGI5KgHXl3PCyu3MeO8kZwwtE/Q4WQ0JQIRyTmLNn7IzOdWc/bofnzl1OFBh5PxlAhEJKfs2LOf6bPL6F9UyO2fG4uZ5gVaojkCEcl6cxeXc/uCNWypqqZzxwLq6kPMvf5UenftFHRoWUEjAhHJag2F5MqrqnFgX12IggJjbeXuoEPLGkoEIpLV4hWSq633vC8klwwlAhHJaiok13ZKBCKS1foXFcZtz/dCcslQIhCRrHZM/16N2lRILjlKBCKStf62poK/rKpgwvA+KiTXBjp9VESy0paqam54bAkjD+/JQ1edSGGnDkGHlLU0IhCRrLO/LsT1s8uorXd+c8V4JYE20ohARLLObc+vZvGmKu6eMo4RxT2CDifraUQgIlnl+RXvc/+r65l68lAmjRkQdDg5QYlARLLGxg/2cOOTSxk7qDc3XzAq6HByhhKBiGSFmtp6rp9dhgF3TxlPl46aF2gvmiMQkazw4z+vZEX5R9z/xRIG9+kWdDg5JZBEYGb/BXwVcGA58GV3rwkiFhHJXA1VRcsj5SI+NbKYT4/uF3BUuSfth4bMbCDwdaDE3Y8FOgCXpTsOEcls0VVFG7y29gPmLi4PMKrcFNQcQUegq5l1BLoBWwKKQ0QyVLyqojW1IVUVTYG0JwJ3LwfuADYBW4Gd7v5CbD8zm2ZmpWZWWllZme4wRSRgqiqaPkEcGjoEmAwMBwYA3c3syth+7n6vu5e4e0lxcXG6wxSRgDV1dzFVFW1/CSUCM/tGIm0J+jSw3t0r3b0WeAqY2Mp1iUgOWrnlI3bV1FIQc7thVRVNjURHBFPjtH2pldvcBEwws24Wvqv0mcCqVq5LRHLMrpparp9dxqE9unDL5GNUVTQNmj191MwuB6YAw83smahFPYEdrdmgu//TzJ4EyoA6YDFwb2vWJSK5xd2ZMWc5m3bs5dH/N4ETh/fhCxOGBR1WzmvpOoLXCE/o9gXujGrfBSxr7Ubd/QfAD1r7ehHJTb9/fSPzl29lxnkjOXF4n6DDyRvNJgJ33whsBE5OTzgikq+WvlfFT+av5MyRhzHttBFBh5NXErqy2Mx2Eb4KGKAz0AnY4+6N7xEnIpKknXtruW5WGYf1LOTOz4+lIHaWWFIqoUTg7j0bHkcmeCcDE1IVlIjkj1DI+eYTS6jYVcMT10ykqFvnoEPKO0lfR+Bhc4FzUhCPiOSZ+/6+jr+uquC754/i+MFFQYeTlxI9NHRx1NMCoARQkTgRaZN/bdjBbQvWcMFx/Zk6cVjQ4eStRKuPXhj1uA7YQPjwkIhIUhoqim6pqsYM+nTrxMxLjiN81FmCkOgcwZdTHYiI5L6GiqINxeTcYde+ehauqtCFYgFKtMTECDP7k5lVmlmFmc0zM53fJSJJiVdRdF+dKooGLdHJ4tnA40B/woXingAeTVVQIpKbVFE0MyWaCLq5+x/cvS7y8whQmMrARCT39OsV/2NDFUWDlehk8XNmNgP4I+ELyy4FnjWzPgDu3qq6QyKSP+rqQ3Tr3PiG86ooGrxEE8HnI7+vjmm/jHBi0HyBiDTrjhfeZt32PXxhwhBeXF3JlqpqBhR15cZzjtZEccASTQSjYm8ub2aFuuG8iCRi4apt3PPyWqacNIQfX3QcPw46IDlIonMEryXYJiJykPd27OWGx5dyzIBefH/S6KDDkThauh/B4cBAwjeaHwc0XPHRi/BN50VEmrS/LsT02WWEQs5vrhhPYafGcwQSvJYODZ1D+E5kg4CfR7XvAm5OUUwikiN++uwqlm7eyT1Xjmfood2DDkea0NL9CB4GHjazS9x9TppiEpEcMH/ZVh56bQNXnTKcc4/tH3Q40oxEJ4uPNbNjYhvd/UftHI+I5ID12/fwnTnLGDekiBnnjQw6HGlBoolgd9TjQmASuuG8iMRRU1vPtY8somMH4+4p4+ncMelq95JmiRadi75fMWZ2B7CgtRs1syLgfuBYwtchXOXur7d2fSISvIaqouWRchHTTh/OQF0xnBVam6q7EZ5Abq1fAs+7+0hgLBpdiGS1hqqi5VE1g/7w+ibmLi4PMCpJVKI3plnOv+9ZXAAcBq27JsTMegOnEz4bCXffD+xvzbpEJDPEqypaXVvP7QvW6KrhLJDoHMEk4BDgNKAIeNbdF7Vym8OBSuB3ZjYWWAR8w933RHcys2nANIAhQ4a0clMikg6qKprdEj00NBn4A9AX6ET4Q/xrrdxmR2A88Ft3HwfsAWbEdnL3e929xN1LiouLW7kpEUk1d2/yQjFVFc0OiY4IvgpMaPjWbmY/A14HftWKbW4GNrv7PyPPnyROIhCR7DD7zU1U19bTscCoC/mBdlUVzR6JjggMiD4AWM+/y00kxd3fB94zs4b/IWcCK1uzLhEJ1oryndzyzEpOP6qY2y8Zw8CirhgwsKgrt158nOYHskSiI4LfAf80s6cjzy8CHmjDdr8GzDKzzsA6QPdEFskyO6truW5WGX26d+auS4+nT/fOfPaEtpxMKEFJ9DqCn5vZS8CpkaYvu/vi1m7U3ZcAJa19vYgEy9359pNL2VJVzWNXT6BP985BhyRtkOiIAHcvA8pSGIuIZIkHXl3Pgre28d8XjOKEoX2CDkfaSNd+i0hSFm38kJnPreas0f34yqnDgw5H2oESgYgkbMee/UyfXUb/okLu+NxYzFp1zohkmIQPDYlIfguFnBseX8IHu/cz59qJ9O7aKeiQpJ1oRCAiCfnty2t5aU0l37twNMcN6h10ONKONCIQkSbFVhQdP7iIK09SyZdcoxGBiMQVr6Loyvc/Yt6SLQFGJamgRCAiccWrKFpTG+L2BWsCikhSRYlAROJSRdH8oUQgInE1dbWwKormHiUCEWlkS1U11fvrGlWWVEXR3KREICIHqa0PMX12GWbGTeePVEXRPKDTR0XkID97bjVlm6q4e8o4Jo0ZwLTTjwg6JEkxjQhE5IDnV7zP/a+uZ+rJQ5k0ZkDQ4UiaKBGICACbPtjLjU8uZeyg3tx8waigw5E0UiIQEWpq67lu9iIMuHvKeLp0jH8PYslNmiMQEX4yfyUryj/ivi+WMLhPt6DDkTTTiEAkz81bUs4jb2zi6tNHcNbofkGHIwEIbERgZh2AUqDc3ScFFYdIPmooJtdwlfDwvt34lq4PyFtBjgi+AawKcPsieSm6mJwDDmytqmH+sq1BhyYBCSQRmNkg4ALg/iC2L5LP4haTq1MxuXwW1IjgLuDbQKipDmY2zcxKzay0srIyfZGJ5DgVk5NYaU8EZjYJqHD3Rc31c/d73b3E3UuKi4vTFJ1I7ivu2SVuu4rJ5a8gRgSnAJ8xsw3AH4FPmdkjAcQhknd21dQScm/UrmJy+S3ticDdb3L3Qe4+DLgMeNHdr0x3HCL5xt2ZMWc5H+6tZfonj1AxOTlAF5SJ5Infv76R+cu38p1zR3LtGUfwrXNGBh2SZIhAE4G7vwS8FGQMIvlg6XtV/GT+Ss4ceRhXnz4i6HAkw+jKYpEcV7V3P9fNKuOwnoXc+fmxFBTE3m5G8p0ODYnksFDI+ebjS6nYVcMT10ykqFv8209KftOIQCSH3ff3dSxcXcF3zx/F8YOLgg5HMpQSgUiOenP9Dm5bsIbzjzucqROHBR2OZDAlApEctH33Pr72aBmDD+nKzEvGYKZ5AWma5ghEckR0RdHOHQuoqw/xp6+dRq/CTkGHJhlOIwKRHBBbUXRfXYiCAuPtbbuCDk2ygBKBSA6IV1G0tt5VUVQSokQgkgNUUVTaQolAJAf0LyqM266KopIIJQKRHDDq8F6N2lRRVBKlRCCS5Rau2sbC1RWcfEQfVRSVVtHpoyJZ7L0de7nh8aUcM6AXv/vSiRR26hB0SJKFNCIQyVL760JMn11GKOT85orxSgLSahoRiGSpnz67iqWbd3LPleMZemj3oMORLKYRgUgWmr9sKw+9toGrThnOucf2DzocyXJKBCJZZv32PXxnzjLGDSlixnm6y5i0nRKBSBapqa3n2kcW0bGDcfeU8XTuqLewtJ3mCESyyA+feYvV7+/id1/6OAN1sZi0k7QnAjMbDPwe6Ac4cK+7/zLdcYhki4aqouWRchGfHnUYnxx5WMBRSS4JYlxZB3zT3UcDE4DrzWx0AHGIZLzoqqINXn13O3MXlwcYleSatCcCd9/q7mWRx7uAVYAufxSJI15V0ZrakKqKSrsKdKbJzIYB44B/xlk2zcxKzay0srIy3aGJZARVFZV0CCwRmFkPYA7wn+7+Uexyd7/X3UvcvaS4uDj9AYpkgN5d499dTFVFpT0FkgjMrBPhJDDL3Z8KIgaRTLeifCe7amopiLndsKqKSntLeyKw8F20HwBWufvP0719kWyws7qW62aVUdyzkB9NPkZVRSWlgriO4BTgC8ByM1sSabvZ3Z8NIBaRjOPufPvJpWypquaxqydwwtA+XDlhWNBhSQ5LeyJw91cBa7GjSJ568B8bWPDWNr57/ihOGNon6HAkD+j6dJEMsmjjh9z67CrOGt2Pr542POhwJE8oEYhkiB179jN9dhn9iwq54z/GEp5OE0k91RoSyQChkHPD40v4YPd+5lw7kd7d4p82KpIKGhGIZIDfvryWl9ZU8r0LR3PcoN5BhyN5RolAJGCvr/2AO19Yw4VjB3DlSUOCDkfykA4NiQSgoaLolqpqzODQHp259eLjNC8ggdCIQCTNoiuKOhBy+Ki6jr+u3BZ0aJKnlAhE0ixeRdF9daooKsFRIhBJM1UUlUyjRCCSZof16hK3XRVFJShKBCJpVFsfojDODedVUVSCpEQgkkY/e241G3dUM/XkoaooKhlDp4+KpMnzK97n/lfXM/Xkodwy+VhumRx0RCJhGhGIpMGmD/Zy45NLGTOoNzdfMCrocEQOokQgkmI1tfVcN3sRBvx6yni6dOwQdEgiB9GhIZEU+8n8lawo/4j7vljC4D7dgg5HpBGNCERSaN6Sch55YxNXnz6Cs0b3CzockbiUCERS5N2K3dz01HI+PuwQvqVTQyWDKRGIpED1/nqum7WIwk4d+NXl4+nUQW81yVyBzBGY2bnAL4EOwP3uPjOIOETaW7iq6GrKq2oAuOYTIzi8d2HAUYk0L+1fU8ysA/Br4DxgNHC5mY1Odxwi7e3fVUVrDrQ9/NpG5i4uDzAqkZYFMV49EXjX3de5+37gj4AurZGMNXdxOafMfJHhM+ZzyswXG32w768L8a8NO/jevBWNqopW19arqqhkvCAODQ0E3ot6vhk4KbaTmU0DpgEMGaK7Nkn7i745zICirtx4ztGNyjw0fMtv+IAvr6pmxlPLWLd9N50KCnhj/Qcs2vghNbWhJrejqqKS6TL2OgJ3vxe4F6CkpMSTfX0ib/Jk+2qdubPOeB/wNz21HOBA/5raem59blWjb/k1tSH+Z+G7AIzq34vLTxzChBGH8sNn3mLrzhpiqaqoZLogEkE5MDjq+aBIW7tJ5E2ebF+tM7PX6e48XVbOzXOXH/h2Xl5VzXfmLGPbRzWccmRf9tXVU1MbYl9dPT/601txD+N8+8ll/HLhO2zftY9d++pozuLvncUh3Tv/+/X76w+KE1RVVLKDuSf9ZbttGzTrCLwNnEk4AfwLmOLubzX1mpKSEi8tLU14G6fMfJHyOMPxjgXG8L7dD2pbv30PdaHG/waxfRPtF8Q6h0X13ZBgv2T6Ntdv6KHdiF6y6YO9cft2KDAGH9L1QN/NH1ZTH6+fwWG9CnEHx3GH7bv3EacrAJ07FFDvHnddrTVpTH/69uhCcc8u3PfKOqqqaxv1GVjUlX/M+FSj9mRGOSKpZmaL3L2kpX5pHxG4e52ZTQcWED599MHmkkBrNHVMti7kfKxfj4Pa3qnYnVDfRPsFsc6j+/U88PzdBPsl07e5fiP79wKg4Zbr6yr3xO1bH3LGDi460HfjB3vj93M49ci+mIFhFBTAo2++F7cvwFWnDqdjgVFQYPzPwnea7Pe/XziBwk4d6NKxgMJOHZj2+1Iqdu1r1G9gUVfunjL+oOfJfMu/aNxAffBL1glkjsDdnwWeTdX6BxR1jTsiGFjUld9cccJBbU2NHmL7JtoviHX++op/f3AtSbBfMn2b7Tfl4HUu3tR0319eNu7A839t+LDJfrd/buxBba+8vb3JvjPOG3ng+ZxFm5vsd84xhx/UdvP5oxL6gG/4UNe3fMllOXm5443nHE3XTgdXeGzqW1yifbXO3FrnReMGcuvFxyV0c5iLxg3kHzM+xfqZF/CPGZ9SEpCck7FnDbVFMt/iEu2rdebWOhv660NdJIDJ4tZIdrJYREQSnyzOyUNDIiKSOCUCEZE8p0QgIpLnlAhERPKcEoGISJ7LirOGzKwS2NjKl/cFtrdjOJkg1/Yp1/YHcm+fcm1/IPf2Kd7+DHX34pZemBWJoC3MrDSR06eySa7tU67tD+TePuXa/kDu7VNb9keHhkRE8pwSgYhInsuHRHBv0AGkQK7tU67tD+TePuXa/kDu7VOr9yfn5whERKR5+TAiEBGRZigRiIjkuZxNBGa/DiPEAAAEVElEQVT2OTN7y8xCZlYS1T7MzKrNbEnk554g40xUU/sTWXaTmb1rZmvM7JygYmwLM/uhmZVH/V3ODzqm1jCzcyN/h3fNbEbQ8bQHM9tgZssjf5esKwNsZg+aWYWZrYhq62NmfzGzdyK/DwkyxmQ1sU+tfg/lbCIAVgAXA6/EWbbW3Y+P/FyT5rhaK+7+mNlo4DLgGOBc4Ddm1qHxy7PCL6L+Lim7g12qRP7dfw2cB4wGLo/8fXLBJyN/l2w87/4hwu+NaDOAhe7+MWBh5Hk2eYjG+wStfA/lbCJw91XuviboONpLM/szGfiju+9z9/XAu8CJ6Y1OIk4E3nX3de6+H/gj4b+PBMjdXwF2xDRPBh6OPH4YuCitQbVRE/vUajmbCFow3MwWm9nLZnZa0MG00UAg+u7umyNt2Wi6mS2LDHuzaqgekUt/i2gOvGBmi8xsWtDBtJN+7r418vh9oF+QwbSjVr2HsjoRmNlfzWxFnJ/mvoVtBYa4+zjgBmC2mfVKT8TNa+X+ZI0W9u+3wBHA8YT/RncGGqxEO9XdxxM+5HW9mZ0edEDtycPn0OfCefStfg9l9T2L3f3TrXjNPmBf5PEiM1sLHAUEPgnWmv0ByoHBUc8HRdoyTqL7Z2b3AX9OcTipkDV/i2S4e3nkd4WZPU34EFi8ubdsss3M+rv7VjPrD1QEHVBbufu2hsfJvoeyekTQGmZW3DCZamYjgI8B64KNqk2eAS4zsy5mNpzw/rwZcExJi7wZG3yW8OR4tvkX8DEzG25mnQlP4j8TcExtYmbdzaxnw2PgbLLzbxPrGWBq5PFUYF6AsbSLtryHsnpE0Bwz+yzwK6AYmG9mS9z9HOB04EdmVguEgGvcvd0mXVKlqf1x97fM7HFgJVAHXO/u9UHG2kq3mdnxhIfoG4Crgw0nee5eZ2bTgQVAB+BBd38r4LDaqh/wtJlB+PNitrs/H2xIyTGzR4EzgL5mthn4ATATeNzMvkK4xP3ng4sweU3s0xmtfQ+pxISISJ7Lu0NDIiJyMCUCEZE8p0QgIpLnlAhERPKcEoGISJ5TIhCJYmavpWCdw8xsSnuvV6S9KBGIRHH3iSlY7TBAiUAylhKBSBQz2x35fYaZvWRmT5rZajObZZGrqiL1+W+L1Oh/08yOjLQ/ZGb/EbsuwhcvnRapEf9f6d4nkZYoEYg0bRzwn4TvLTACOCVq2U53Pw64G7irhfXMAP4eqRH/i5REKtIGSgQiTXvT3Te7ewhYQvgQT4NHo36fnO7ARNqTEoFI0/ZFPa7n4NpcHudxHZH3lJkVAJ1TGp1IO1EiEGmdS6N+vx55vAE4IfL4M0CnyONdQM+0RSaSpJytPiqSYoeY2TLCo4bLI233AfPMbCnwPLAn0r4MqI+0P6R5Ask0qj4qkiQz2wCUuPv2oGMRaQ86NCQikuc0IhARyXMaEYiI5DklAhGRPKdEICKS55QIRETynBKBiEie+z+wkJrjsD6xBAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cx.plot_f(lambda x: elu(x, 0.3), frange=(-15, 15, 1), xlabel=\"input\", ylabel=\"output\", title=\"elu(alpha=0.3)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## selu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function selu in module conx.activations:\n",
      "\n",
      "selu(x)\n",
      "    Scaled Exponential Linear Unit activation function.\n",
      "    \n",
      "    >>> selu(0)\n",
      "    0.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(selu)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5253505110740662"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "selu(0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEWCAYAAABmE+CbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHdVJREFUeJzt3XmcVPWZ7/HP0003NAq0DI3sYhRBIwqmNW5JVDRoQtSoSdSJWR00LqNxRZ3ca5YbY8w6N7lJSHKj4zruOmYh5kZjjCba7CLiqKCs0i4NLfRWVc/9o6qgKKrpamr51fJ9v1716qrTVaeeQvt865zfOb/H3B0REZGa0AWIiEhpUCCIiAigQBARkQQFgoiIAAoEERFJUCCIiAigQBDZbWY20czczAb04zV/M7PpifuXmtnNhatQpH8UCCJFYmafANrdfWFi0S+BfzazkQHLEtlGgSBSPBcCtycfuHsn8Hvgc8EqEkmhQBBJMLNrzWytmbWb2Qozm2FmNWY2x8xeNbO3zexeMxvey+tXmdmJKY9vNLM7EvfrgROAv6S97Eng4wX6SCL9okAQAcxsMnAJcLi7DwFmAquAS4HTgY8AY4B3gZ/uxltMAmLuviZt+XLg0N0sWySvsh4ME6lwUWAgcJCZtbr7KgAzuxC4JLkhN7MbgTfM7Lx+rr8RaM+wvB0YtrtFi+ST9hBEAHd/BbgcuBHYaGb3mNkYYB/gITNrM7M24t/oo8De/XyLd4EhGZYPATbtduEieaRAEElw97vc/VjiIeDAzcBq4BR3b0y5DXL3tRlWsQUYnPJ4VMr9VwAzs7FprzkQWJy/TyGy+xQIIsTHEMzsBDMbCHQCHUAM+Dnwv8xsn8TzmszstF5Wswg428zqzKwZOCv5C3fvBv5EfCwi1UeIn2kkEpwCQSRuIPAd4C1gAzASuA74MfAo8Eczawf+Dnywl3V8DdiP+OGhrwN3pf3+F8C2sQczGwR8DLgtb59CJAemBjkixWNmfyM+SL3QzC4Fxrv7NaHrEgEFgoiIJOiQkYiIAAoEERFJCHphmpmtIn5hThSIuHtzyHpERKpZKVypfLy7v5XNE0eMGOETJ04scDkiIpVl/vz5b7l7U1/PK4VAyNrEiRNpaWkJXYaISFkxs9ezeV7oMQQnfn73fDObnekJZjbbzFrMrKW1tbXI5YmIVI/QgXCsux8GnAJcbGYfTn+Cu89192Z3b25q6nOPR0REdlPQQEjOB+PuG4GHgCNC1iMiUs2CBYKZ7WFmQ5L3gY8CL4SqR0Sk2oUcVN6b+LTCyTrucvc/BKxHRKSqBQsEd38NdYoSEdmlhxeu5ZZ5K1jX1sGYxgaunjmZ06enz6KeH2V12qmISDV5eOFarntwKR09UQDWtnVw3YNLAQoSCqHPMhIRkV7cMm/FtjBI6uiJcsu8FQV5PwWCiEiJWtfW0a/luVIgiIiUoK3dEeoGZN5Ej2lsKMh7KhBERErM1u4IX7r1ebojMepqbYffNdTVcvXMyQV5Xw0qi4iUkK3dEb58awvPrXyHH589DXd0lpGISLVJhsE/Vr7NDz49jdOmxTf8hQqAdDpkJCJSAjq6ozuEQbFCIJUCQUQksI7uKF+69fmgYQAKBBGRoFLD4PufPjRYGIACQUQkmPQw+OT0cUHrUSCIiARQamEACgQRkaLr6I7y5dtKKwxAp52KiBRVMgyefe1tflBCYQDaQxARKZpSDgPQHoKISEGl9jOoH1BDVyTGDz9TemEACgQRkYJJ72fQlZibyLA+XhmGDhmJiBRIpn4GPVEvWD+DXCkQREQKpNj9DHKlQBARKYCO7ij1Re5nkCsFgohInnX2RPmX/2jZNmaQqpD9DHIVfFDZzGqBFmCtu88KXY+ISC46e6Kcf1sLf3v1Lb7/qUOprbGi9TPIVfBAAC4DlgNDQxciIpKL1DD43lmHcuYH4qeWlmoApAt6yMjMxgEfB34Vsg4RkVylhsEtKWFQTkKPIfwIuAaI9fYEM5ttZi1m1tLa2lq8ykREspQeBmeVYRhAwEAws1nARnefv6vnuftcd2929+ampqYiVScikp3kAHK5hwGE3UM4BjjVzFYB9wAnmNkdAesREemXZBg8/cpbfPfMQ8o6DCBgILj7de4+zt0nAmcDf3b3z4aqR0SkP9LD4FPN40OXlLPQYwgiImWnEsMASuO0U9z9SeDJwGWIiPQpNQxurqAwAO0hiIhkLT0MPl1BYQAlsocgIlKKUnsZjB42iCENdbz8ZntFhgEoEEREMkrvZbBuUyds6uTsw8dXZBiADhmJiGSUqZcBwF//+60A1RSHAkFEJINy62WQDwoEEZEMRg8blHF5qfYyyAcFgohIms6eKEMa6nZaXsq9DPJBgSAikqKzJ8oFt89nxYZ2zj58PGMbGzBgbGMDN50xtWymst4dOstIRCQhGQZ/ebmVm8+cymcOnxC6pKLSHoKICAoDUCCIiNDZE+XCO6o7DECBICJVLhkGT65o5TtnVG8YgAJBRKpYZ0+Ur6SEwdlHVG8YgAJBRKpUMgyeWNHKTQoDQIEgIlWoK7JjGJyjMAAUCCJSZboiUS68XWGQiQJBRKpGahh8+5MKg3S6ME1EKlZ6P4NhDXUs39DOtz85lXM/qDBIp0AQkYqUqZ/Buk2dfKp5nMKgFzpkJCIVqbd+Bs+88naAasqDAkFEKlI19jPIlQJBRCpSNfYzyFWwQDCzQWb2nJktNrNlZvb1ULWISGXpikRpHFx9/QxyFXIPoQs4wd0PBaYBJ5vZkQHrEZEK0BWJctEdC3hxfTtnfWBcVfUzyFWws4zc3YH3Eg/rEjcPVY+IlL+uSJSL71zA/3tpI986/WA+e+Q+oUsqK0HHEMys1swWARuBx939HxmeM9vMWsyspbW1tfhFikhZSIbBn5Zv5JsKg90SNBDcPeru04BxwBFmdnCG58x192Z3b25qaip+kSJS8tLD4DyFwW4pibOM3L0NeAI4OXQtIlJeuiMxhUGehDzLqMnMGhP3G4CTgJdC1SMi5ac7EuOiO+fHw+C09ysMchRy6orRwG1mVks8mO5198cC1iMiZWSnMDhqYuiSyl7Is4yWANNDvb+IlK94GCxQGORZSYwhiIhka3sYvMk3FAZ5pUAQkbKRHgafUxjklaa/FpGSld7PYPge9bywbrPCoEAUCCJSknrrZ3DmYWMVBgWiQ0YiUpJ662fw99feCVBNdVAgiEhJUj+D4lMgiEhJUj+D4lMgiEjJ6Y7EGL5H/U7L1c+gsBQIIlJSuiMxLrlrAS+s28wZh41VP4Mi0llGIlIyuiMxLr17AX988U2+fur7+fzRE0OXVFW0hyAiJSEZBvOWvcmNnzhIYRCAAkFEguuJ7hgGXzhm39AlVSUFgogE1RONjxkoDMJTIIhIMKlh8D8VBsEpEEQkiPQw+KLCIDgFgogUXU80xqV3LVQYlBgFgogUVTIM/rBsA/9jlsKglCgQRKRo0sPgS8cqDEqJLkwTkYLZoZ9B4yBG7FnPkjWbFQYlSoEgIgWxUz+Dtk7WtXVy+rQxCoMSpUNGIlIQvfUzeH7VuwGqkWwECwQzG29mT5jZi2a2zMwuC1WLiOSf+hmUn5B7CBHgSnc/CDgSuNjMDgpYj4jk0ehG9TMoN8ECwd3Xu/uCxP12YDmgeW1FKkBPNEbTnupnUG5KYgzBzCYC04F/hK1ERHLVE41x+T2LWLxmM6dPG6N+BmUk+FlGZrYn8ABwubtvzvD72cBsgAkTJhS5OhHpj2QY/Hbpev7t4wdy/ofeF7ok6Yes9hAyDfjmYxDYzOqIh8Gd7v5gpue4+1x3b3b35qamplzfUkQKRGFQ/rI9ZPT5DMu+kMsbm5kBvwaWu/sPclmXiIQVURhUhF0eMjKzc4BzgX3N7NGUXw0B3snxvY8BzgOWmtmixLLr3f13Oa5XRIooEo1xWSIMbviYwqCc9TWG8AywHhgBfD9leTuwJJc3dvenActlHSISViQa47L/3B4G//JhhUE522UguPvrwOvAUcUpR0TKxbYwWLKe6z82RWFQAbI6y8jM2gFPPKwH6oAt7j60UIWJSOlKD4PZH94vdEmSB1kFgrsPSd5PDAafRvzqYhGpMpFojMsVBhWp3xemedzDwMwC1CMiJSwZBo8tWc91pygMKk22h4zOSHlYAzQDnQWpSERKRno/g5FDBrJo9SauO2UKF3xEYVBpsr1S+RMp9yPAKuKHjUSkQvXWz+DUQ0YrDCpUtmMIXyx0ISJSWnrrZzD/jbYA1UgxZDt1xfvM7L/MrNXMNprZI2amc8xEKpj6GVSfbAeV7wLuBUYDY4D7gLsLVZSIhKd+BtUn20AY7O63u3skcbsDyPx/i4iUvUg0xqghO/+Jq59BZcs2EH5vZnPMbKKZ7WNm1wC/M7PhZja8kAWKSHFFojGuuHcxC1a3MeuQ0epnUEWyPcvo04mfF6QtP5v4FcwaTxCpAJFojCvvW8yji9dx7clT+MpxOpuommQbCAe6+w7XHZjZoPRlIlK+kmHwyKJ1XHPyZIVBFcr2kNEzWS4TkTKUHgYXHbd/6JIkgL76IYwi3vi+wcyms3266qHA4ALXJiJFEI25wkCAvg8ZzSTeGW0ckNrVrB24vkA1iUiRRGPOFfcu4pFF67h6psKg2vXVD+E24DYzO9PdHyhSTSJSBOlhcPHxCoNql+2g8sFm9v70he7+jTzXIyJFEI05VyoMJE22gfBeyv1BwCxgef7LEZFCS4bBwwoDSZPt5Hap/ZQxs+8B8wpSkYgUjMJAdqXfDXISBhMfaBaRMhGNOVfdt1hhIL3KtkHOUrb3VK4BRgLfLFRRIpIfqQ1uBtXV0tET5aqPHqAwkIyyHUOYBewFfAhoBH7n7vNzfXMz+7+JdW9094NzXZ+IbJfe4KajJ8qAGmPcXrqESDLL9pDRacDtwAigDviNmV2ah/e/FTg5D+sRkTSZGtxEYs4t81YEqkhKXbZ7COcDR7r7FgAzuxl4Fvjfuby5uz9lZhNzWYeIZKYGN9Jf2e4hGJD6VSPK9mksCsrMZptZi5m1tLa2FuMtRcpeNOYMqqvN+Ds1uJHeZLuH8BvgH2b2UOLx6cCvC1PSjtx9LjAXoLm52ft4ukjVi8acq+9bvG3MIBLb/mejBjeyK9leh/ADM3sSODax6IvuvrBgVYnIbonGnKvvX8yDC9dy5UkHMH744G1nGY1pbODqmZPV4EZ6le0eAu6+AFhQwFpEJAfbwmDBWq446QAunTEJQAEgWdvdC9PywszuJj44PdnM1pjZl0PWI1Ku0sPgXxNhINIfWe8hFIK7nxPy/UUqQTTmXHP/EoWB5CzoHoKI5CYZBg8sWMNXT1QYSG4UCCJlKj0MLjtRYSC5USCIlKFozLn2AYWB5JcCQaTMJMPg/vlruPzESQoDyRsFgkgZSQ+Dy088IHRJUkEUCCJlIhZz5igMpICCnnYqIruW2s+gob6Wrd1RLpuhMJDCUCCIlKj0fgZbu+NzE+07Yo/AlUml0iEjkRKlfgZSbAoEkRKlfgZSbAoEkRIUizkN9epnIMWlQBApMbHEqaXJMYNU6mcghaRAECkhsZgz58El3Dd/Df86YxLf+9ShjG1swICxjQ3cdMZUTWctBaOzjERKRDIM7m2Jh8FXT5yEmSkApGi0hyBSAnoLA5FiUiCIBBaLOdc9uDQeBifsrzCQYBQIIgElw+A/W1bHw+CkAxQGEowCQSQQhYGUGgWCSACxmHP9Q/EwuFRhICVCgSBSZMkwuOf5eBhcoTCQEqFAECkihYGUsqCBYGYnm9kKM3vFzOaErEWk0GIx54aH42FwyfEKAyk9wS5MM7Na4KfAScAa4Hkze9TdXwxVk0i+ZepncMnx+3PlRxUGUnpC7iEcAbzi7q+5ezdwD3BawHpE8irZz2BtWwfO9n4G+zXtoTCQkhQyEMYCq1Mer0ksE6kIvfUz+N4fXw5UkciulfygspnNNrMWM2tpbW0NXY5I1tTPQMpNyEBYC4xPeTwusWwH7j7X3ZvdvbmpqaloxYnkau+hAzMuVz8DKVUhA+F5YJKZ7Wtm9cDZwKMB6xHJm55ojIa6nRvcqJ+BlLJggeDuEeASYB6wHLjX3ZeFqkckn743bwUr397K547aR/0MpGwE7Yfg7r8DfheyBpF8e/zFN/nFU69x3pH78I3TDuYbOndOykTJDyqLlJPV72zlynsXMXXsMP5t1oGhyxHpFwWCSJ50RaJcfNcCHPjpuYcxcMDOYwgipUwtNEXy5Nu/Xc6SNZv4xXkfYMI/DQ5djki/aQ9BJA8eW7KO2559nfOP3ZeZ7x8VuhyR3aJAEMnRa63vMeeBpRw2oZFrT5kSuhyR3aZAEMlBZ0+Ui+5cQF2t8ZNzD6OuVn9SUr40hiCSgxsfXcZLG9q59YuH6wpkKXv6OiOymx6Yv2Zbb4PjJo8MXY5IzhQIIrthxYZ2bnh4KUe+bziXnzgpdDkieaFAEOmnLV0RLrpzPnsOrOPfz57OAI0bSIXQ/8ki/eAe74m88q0t/Ps50xg5dFDokkTyRoEg0g93P7eaRxat44qTDuDo/UaELkckrxQIIll6Ye0mbvyvZXz4gCYuOm7/0OWI5J1OOxXZhYcXruWWeStY19ZBTY2xR30tP/rMNGpq1BNZKo/2EER68fDCtVz34FLWtnXgQDTmdEViPPWyWrlKZVIgiPTilnkr6OiJ7rCsKxLjlnkrAlUkUlgKBJFerGvr6NdykXKnQBDJYMmaNnobJtAUFVKpFAgiae6fv4azfv4sQwbVMXDAjn8iDXW1XD1zcqDKRApLgSCS0BONceOjy7jqvsU077MXf77qOG4+8xDGNjZgwNjGBm46YyqnTx8bulSRgtBppyLAW+91cdGdC3hu5Tucf+y+zDllCgNqazh9+lgFgFQNBYJUvSVr2rjg9vm8u7WbH589jdOmKQCkOgU5ZGRmnzKzZWYWM7PmEDWIwPbxghoz7r/waIWBVLVQewgvAGcAvwj0/lLleqIxvvXYi9z27Oscvd8/8ZNzD2P4HvWhyxIJKkgguPtyADNd/i/F19rexcV3LuC5VTuOF4hUu5IfQzCz2cBsgAkTJgSuRsrd4tVtXHiHxgtEMilYIJjZn4BRGX51g7s/ku163H0uMBegubnZ81SeVInUyemGDa6jvaOHUcMauP/Cozl47LDQ5YmUlIIFgrufWKh1i2QjOTldcj6itq091Bh85bj9FAYiGejAqVSs7/7hpZ0mp4s5/OzJVwNVJFLaQp12+kkzWwMcBfzWzOaFqEMq05auCL986jXWberM+HtNTieSWaizjB4CHgrx3lK5Nnf28B/PrOLXT6/k3a09DBxQQ1ckttPzNDmdSGYlf5aRSF/e2dLNb/62klufWUV7Z4QZU0Zy8Qn788bbW3cYQwBNTieyKwoEKVsb2zv51V9XcsffX2drd5RTDh7Fxcfvv23A+LAJewFsO8toTGMDV8+crLmJRHqhQJCSlnraaHKDfsS+w5n71Gvc/dwb9ERjnHroGC46fn8O2HvITq/X5HQi2VMgSMlKP210bVsHV963GHenxowzDhvLV47bn31H7BG4UpHKoECQkpWpp3E05uxRX8u8r36YcXsNDlSZSGVSIEjJefu9Lp5c0craXk4P3dodVRiIFIACQYJzd5at28wTL23kzys2smh1G+5QY/ELydLptFGRwlAgSEFlGhQ+ffpYtnZHePq/3+KJFRt54qVWNmyOX0R26LhhXDZjEjOm7M0rb7Zz/cMv6LRRkSJRIEjBZBoUvvr+xfzsL6+ysnUL3dEYew4cwIcmjeD4KSM5bnITI4cM2vb6qeOGYTWm00ZFikSBIAXxzpZuvvXbF3caFO6JOq9ufI/PHz2RE6aM5PCJw6kf0PsMKjptVKR4FAiyS70d8knqikR5deMWXtqwmZc2tMdv6zezsb2r13VGY87XZh1UjPJFpB8UCNKrTId8rrl/CY+/uIHamhpe2rCZ11q3EEmM/NbX1jBp7z05dtIIDhw1lJ//5VXe3tK903o1KCxSmhQIFa6vb/hJW7oibNjcyYZNnazf1Mmbmzv5P0+8stMhn+5ojN8u3cDYxgamjBrCSQftzZRRQ5kyaggTR+xBXUoryqYhAzWXkEgZUSAUWLYb5EK8/uGFa5nz4BI6e+IzfiYHdX+/dD2Ng+tZv7mTDZs6WL+pk/bOSNY1GfC3OSf0+bxknRoUFikP5l4+XSmbm5u9paWlX68JvUHO9A35pjOmZrWOhxas4bqHlm7boAPUD6jhsx+cwJTRQ9nc0cOmjp5tP5O3zZ0RNnX00LqL4/h7Dx3IqKGDGDVsEKOHNbD30EGMHhZ/nFw+4/t/yXhx2NjGhqwCQURKg5nNd/fmPp9XyYGwuxtkdycacx5auJavPfLCDhvkgQNquPKjB/CRA0bSHYnRHY3Rk3Lrjnh8WSTGNx97kbaOnp3WP7i+lpMPHkVnT5SO7ihbu6Px+8lbd/y2pTu602vTmcHQQXUMa4jfhjYM2Hb/7udWZ34NsPI7H+9z3bkGmoiUBgUCcMx3/pzxG26NwYg9BxKNOZFYfOOfvEVisYxXx+bbuL0aaKirpaG+lkF1tTTU1TK4Pv5zUOLnr59emfG1Bjx1zfEMbahjyMAB1NRYxuf19vn78w0/1z0sEQkv20Co6DGE3lolxhxmHDiS2hpjQE0NtTW27TYged+M7z/+cq/r/tk/H0ZdbQ11A2qoqzUGDqiJP07c6mtr+PQvnt12BW6qsY0NPH1t3xvkP7ywIeMGfUxjA+OH9z2Xz9UzJ+c8qKvrAESqR0UHwpjGhl6/Id90xiF9vv6e51f3+vpTpo7u8/VzTpmS0wY51w26BnVFpD8qOhBy3aCG3iDnY4Oub/gikq2KHkOAsGcZiYiUAg0qi4gIkH0g9D6rWAGZ2S1m9pKZLTGzh8ysMUQdIiKyXZBAAB4HDnb3Q4CXgesC1SEiIglBAsHd/+juybkS/g6MC1GHiIhsF2oPIdWXgN/39kszm21mLWbW0traWsSyRESqS8FOOzWzPwGjMvzqBnd/JPGcG4AIcGdv63H3ucBciA8qF6BUEREh4FlGZvYF4AJghrtvzfI1rcDrhayrgEYAb4UuIiB9fn3+av78EPbfYB93b+rrSUECwcxOBn4AfMTdq+I4kJm1ZHPaV6XS59fnr+bPD+XxbxBqDOEnwBDgcTNbZGY/D1SHiIgkBJm6wt33D/G+IiLSu1I4y6hazA1dQGD6/NWt2j8/lMG/QVlNXSEiIoWjPQQREQEUCCIikqBACMDMrjQzN7MRoWsppmqd1NDMTjazFWb2ipnNCV1PMZnZeDN7wsxeNLNlZnZZ6JpCMLNaM1toZo+FrmVXFAhFZmbjgY8Cb4SuJYCqm9TQzGqBnwKnAAcB55jZQWGrKqoIcKW7HwQcCVxcZZ8/6TJgeegi+qJAKL4fAtcAVTeaX6WTGh4BvOLur7l7N3APcFrgmorG3de7+4LE/XbiG8Wq6jBlZuOAjwO/Cl1LXxQIRWRmpwFr3X1x6FpKwC4nNawgY4HVKY/XUGUbxCQzmwhMB/4RtpKi+xHxL4Gx0IX0paJ7Koewq0n9gOuJHy6qWPma1FAqi5ntCTwAXO7um0PXUyxmNgvY6O7zzey40PX0RYGQZ+5+YqblZjYV2BdYbGYQP1yywMyOcPcNRSyxoHr7/EmJSQ1nEZ/UsBoOm60Fxqc8HpdYVjXMrI54GNzp7g+GrqfIjgFONbOPAYOAoWZ2h7t/NnBdGenCtEDMbBXQ7O5VMwNklU5qOID4APoM4kHwPHCuuy8LWliRWPzbz23AO+5+eeh6QkrsIVzl7rNC19IbjSFIMVXdpIaJQfRLgHnEB1TvrZYwSDgGOA84IfHffFHi27KUIO0hiIgIoD0EERFJUCCIiAigQBARkQQFgoiIAAoEERFJUCCIZGBmzxRgnRPN7Nx8r1ckXxQIIhm4+9EFWO1EQIEgJUuBIJKBmb2X+HmcmT1pZvcnejncmbj6FjNbZWbfNbOlZvacme2fWH6rmZ2Vvi7gO8CHEhdnfbXYn0mkLwoEkb5NBy4n3s/gfcSvvk3a5O5TiV+F/aM+1jMH+Ku7T3P3HxakUpEcKBBE+vacu69x9xiwiPihn6S7U34eVezCRPJJgSDSt66U+1F2nCXYM9yPkPjbMrMaoL6g1YnkiQJBJDefSfn5bOL+KuADifunAnWJ++3EJ/cTKUnqhyCSm73MbAnxvYhzEst+CTxiZouBPwBbEsuXANHE8ls1jiClRrOdiuymauxpIZVNh4xERATQHoKIiCRoD0FERAAFgoiIJCgQREQEUCCIiEiCAkFERAD4/+putA5ob3rhAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cx.plot_f(selu, frange=(-5, 5, 0.5), xlabel=\"input\", ylabel=\"output\", title=\"selu()\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## softplus"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function softplus in module conx.activations:\n",
      "\n",
      "softplus(x)\n",
      "    Softplus activation function.\n",
      "    \n",
      "    >>> round(softplus(0), 1)\n",
      "    0.7\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(softplus)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9740769863128662"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "softplus(0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VPW5x/HPkxBWgSCEXQyogIiyGC2Vqri0YNVitVrR2+vWoi1aW1st2t7aenur11rr1uXa1moVcBdttVWrV72tVg2ETRZlN2ELYFgTsj33j5nQZJiQSTIzZ5bv+/XKKzO/+eX8nsOQefI7v3OeY+6OiIhkr5ygAxARkWApEYiIZDklAhGRLKdEICKS5ZQIRESynBKBiEiWUyIQCTOzr5vZZjPbbWa92/DzD5vZT+IQRyczW2pmA8LPf25mX2/vdkWao0QgAphZHnA38Dl3P8Tdt5mZm9mRAYQzHXjL3TeGn98F3GJmHQOIRbKAEoFISD+gM/BB0IEA1wCPNjwJJ4TlwBcCi0gymhKBZCQz+56ZlZnZLjNbYWZnhA+53GNmG8Jf94TbhgMrwj9aYWavm9lb4ecLw4eKvmxmk8ys1MxuMbOtZrbWzC5tZvzLzezvEW37Zxhm9vnw4Z9d4Ti/G24fAgwD3o3Y5BvA2fH51xFpqkPQAYjEm5mNAK4FTnD3DWZWCOQC3wcmAGMBB54HfuDu/2FmxwBrgHx3rw1vx4Ex7r4y/HwS0B/oAwwKb+slMyt29xW0zu+Bi9z9/8ysFzA03H4ssLohhkaWARe0cgyRmGhGIJmoDugEjDKzPHdf6+6rgEuB29x9i7uXAz8GvtKG7f+Hu+9z9zeBF4GL2rCNmnB8Pdz9E3efH27PB3ZF6b8r/JpI3CkRSMYJ/wX/LeBHwBYze9zMBgIDgXWNuq4Lt7XGJ+6+p53bgNBf958H1pnZm2b26YbtA92j9O8OVLRhHJEWKRFIRnL32e7+GeBwQoeB/hvYEH7eYEi4rTV6mVm3GLaxB+ja8MTM+kfE9767TwX6AnOBJ8MvLQKGmlnkYdujgYWtjFUkJkoEknHMbISZnW5mnYAqoBKoB+YAPzCzAjPrA/wQeOwgm9pMaOE20o/NrKOZnQycAzwVpc9C4BgzG2tmnQnNThri62hml5pZT3evAXaG48PdS4GVwIkR2zsV+EtL+y7SFkoEkok6AXcAW4FNhP7qvhn4CVBM6K/uxcD8cFtzfgQ8YmYVZtawDrCJ0OGbDcAs4Bp3Xx75g+7+IXAb8DfgI+DvEV2+Aqw1s52EThdtfPbR/9Bo7SJ8YdkoQjMHkbgz3ZhGJDbhs4Yec/fBCR6nE1ACnOHuG83s58Aqd/9VIseV7KXTR0VSjLvvIzQDaHj+nQDDkSygQ0MiIllOh4ZERLKcZgQiIlkuLdYI+vTp44WFhUGHISKSVubNm7fV3Qta6pcWiaCwsJDi4uKgwxARSStmtq7lXjo0JCKS9ZQIRESynBKBiEiWUyIQEclySgQiIlkuYYnAzB4ysy1mtiTKa98J37avT6LGFxFJZ3NLyph4x+sMnfkiE+94nbklZQkbK5EzgoeBKZGNZnYY8DlgfQLHFhFJW3NLyrj52cWUVVTiQFlFJTc/uzhhySBhicDd3wK2R3npF8BNhG4WIiIiEX728goqa+qatFXW1PGzl1t7a+zYJHWNwMymAmXu3uKdlsxsupkVm1lxeXl5EqITEUkNGyoqW9XeXklLBGbWFbiF0F2hWuTuD7p7kbsXFRS0eIW0iEjGyO+aF7V9YH6XhIyXzBnBEcBQYKGZrQUGA/Mj7+UqIpLN1m7dw959tZg1be+Sl8uNk0ckZMykJQJ3X+zufd290N0LgVJgvLtvSlYMIiKpbF9tHdfOmU/njh344TmjGJTfBQMG5Xfh9vOP5bxxgxIybsKKzpnZHGAS0MfMSoFb3f33iRpPRCTd3f7ScpaU7eS3/17EZ0f144qJQ5MybsISgbtPa+H1wkSNLSKSbv66ZBMPv72WKycO5bOj+iV1bF1ZLCISsI+37+Wmpxdy3OCezDxrZNLHVyIQEQlQdW09180pwR0emDaejh2S/7GcFjemERHJVHe9soIFH1fwq0vHM6R310Bi0IxARCQgry/fzINvreYrEw7n88cOCCwOJQIRkQBs3FHJd55cyNEDevD9s48ONBYdGhIRSZK5JWX87OUVbKioJC83B3B+eck4OuflBhqXZgQiIkkQWVG0uq4eBxaV7gg6NCUCEZFkiFZRtKbOE1ZRtDWUCEREkiDZFUVbQ4lARCQJBuR3jtqeqIqiraFEICKSBMcNyj+gLZEVRVtDiUBEJMHeWbWNV5ZuoujwfAbld05KRdHW0OmjIiIJtG33Pq5/vITC3t145MpP0a1T6n3spl5EIiIZor7e+faTC6morOHhK05MySQAOjQkIpIw//PWat76sJwfnjOKUQN7BB1Os5QIREQSYN667dz1ygrOPnYAl35qSNDhHJQSgYhInFXsrea62SWhBeELjsUib0CcYlLzgJWISJpyd7771CLKd+/jma+fRI/OeUGH1CLNCERE4uihf6zlb8s2c/NZR3Pc4AOvHUhFibx5/UPAOcAWdx8dbvsZcC5QDawCrnD3ikTFICKSDI2rijpwzMAeXDGxMOiwYpbIGcHDwJSItleB0e5+HPAhcHMCxxcRSbjIqqIAq8p38/yCDYHG1RoJSwTu/hawPaLtFXevDT/9JzA4UeOLiCRDtKqiVTX1KVFVNFZBrhFcCfyluRfNbLqZFZtZcXl5eRLDEhGJXSpXFY1VIInAzL4P1AKzmuvj7g+6e5G7FxUUFCQvOBGRVujTvVPU9lSoKhqrpJ8+amaXE1pEPsPdvYXuIiIpa/e+WojyMZYqVUVjldQZgZlNAW4CvuDue5M5tohIPLk7P3huMdv2VHPtaUcwKL9LylUVjVUiTx+dA0wC+phZKXArobOEOgGvhq+0+6e7X5OoGEREEuWp4lLmLtjADZ8dzjfPOIrvTh4ZdEhtlrBE4O7TojT/PlHjiYgky4ebd/HDF5Zw0hG9mXHakUGH0266slhEpBUqq+uYMWs+h3TqwD0XjyU3J7XrCMVCtYZERFrh1heWsLJ8N49e+Sn6do9+H+J0oxmBiEiMnisp5cniUmZMOpLPHNUn6HDiRolARCQGq8t38/3nlnBCYS++deZRQYcTV0oEIiItqKqpY8bsEjp1yOG+aePokJtZH51aIxARacF/vbiMZRt38tDlRQzomT5XDMdKiUBEJIqG0tJl4ZpBp40o4PSR/QKOKjEya34jIhIHjUtLN3hn9TbmlpQFGFXiKBGIiETIhNLSraFEICISIRNKS7eGEoGISIRe3TpGbU+n0tKtoUQgItJIWUUlldW1WETliHQrLd0aSgQiImE1dfV8c04JuTk53HLW0WldWro1dPqoiEjYz1/5kHnrPuH+aeM4d8xAvnbKsKBDSgrNCEREgDdWbOE3b67ikk8N4dwxA4MOJ6mUCEQk623eWcUNTy5kZP/u/PCcUUGHk3RKBCKS1WrD6wJVNXU8cMl4OuflBh1S0mmNQESy2n2vr+TdNdu5+6IxHNn3kKDDCYRmBCKStd5euZX7X/+ILx0/mPPHDw46nMAoEYhIVirftY/rn1jAEQWHcNvUY4IOJ1AJOzRkZg8B5wBb3H10uO1Q4AmgEFgLXOTunyQqBhGRSHNLyrjz5eVsqKgC4MqJhXTtmN1HyRM5I3gYmBLRNhN4zd2PAl4LPxcRSYqGqqINSQDgvtdWZmxV0VglLBG4+1vA9ojmqcAj4cePAOclanwRkUjRqopW1tRlbFXRWCV7jaCfu28MP94ENHuXBzObbmbFZlZcXl6enOhEJKNlW1XRWAW2WOzuDvhBXn/Q3YvcvaigoCCJkYlIJqqvdzp2iP6Rl6lVRWOV7ESw2cwGAIS/b0ny+CKSpX7399Xsq60nL7dpWdFMrioaq2QngheAy8KPLwOeT/L4IpKF5q//hDv/uoKzRvfnzguOy5qqorFK5Omjc4BJQB8zKwVuBe4AnjSzq4B1wEWJGl9EBGDH3hqum11C/56dueOC4+jZJY8vZvHFY9EkLBG4+7RmXjojUWOKiDTm7tz49EK27KriqWtOomeXvKBDSkm6slhEMtYjb6/llaWb+d6UkYw9LD/ocFKWEoGIZKTFpTv46UvLOWNkX676zNCgw0lpSgQiknF2VdVw7Zz59DmkI3ddOAaLvAGxNJHdBTZEJOO4OzOfXUzpJ5U8MX0Cvbp1DDqklKcZgYhklDnvfcyLizbync8Np6jw0KDDSQuaEYhI2ptbUsbPXl7BhopKHBjRvzvXnHJE0GGlDc0IRCStNVQULQsnAYB12/bwwsINgcaVTpQIRCStRasoWlVTn/UVRVtDiUBE0poqirafEoGIpLWC7p2itmd7RdHWUCIQkbRVWV1HTpRLBFRRtHWUCEQkbf34Tx+wedc+rjl1mCqKtoNOHxWRtPT8gjIef/9jZpx2BDdOHsnMs44OOqS0pRmBiKSdNVv3cMuziyk6vBffPnN40OGkPSUCEUkrVTV1zJg1n7wOOdw3bRwdcvUx1l46NCQiaeWnLy1j6cadPHR5kc4MihOlUhFJGy8t3sgf31nH104eyukj+wUdTsZQIhCRtLB+216+9/QixhyWz42TRwYdTkZRIhCRlFddW891c+aDwQPTxtGxgz664imQNQIz+zbwVcCBxcAV7l4VRCwikroaqoqWhctFXDHxcA47tGvAUWWepKdVMxsEfBMocvfRQC5wcbLjEJHU1riqaIPH3ytlbklZgFFlpqDmVx2ALmbWAegKqF6siDQRrapoZU2dqoomQEyJwMyuj6UtFu5eBtwFrAc2Ajvc/ZUo259uZsVmVlxeXt6WoUQkjamqaPLEOiO4LErb5W0Z0Mx6AVOBocBAoJuZ/VtkP3d/0N2L3L2ooKCgLUOJSBrr1in6EqauHYi/gy4Wm9k04BJgqJm90Oil7sD2No55JrDG3cvDYzwLnAQ81sbtiUiGefPDcnbvqyU3x6ir9/3tqiqaGC2dNfQ2ocM3fYCfN2rfBSxq45jrgQlm1hWoBM4Aitu4LRHJMJt3VnHDEwsY0a87Xz15KPf87SM2VFQyML8LN04eoaqiCXDQRODu64B1wKfjNaC7v2tmTwPzgVqgBHgwXtsXkfRVV+9c/3gJe6vr+OWl4ziyb3cuLDos6LAyXkzXEZjZLth/X+iOQB6wx917tGVQd78VuLUtPysimeu+1z7in6u38/MLx3Bk3+5Bh5M1YkoE7r7/HTEzI7TYOyFRQYlI9nl75Vbue/0jLhg/mAuOHxx0OFml1dcReMhcYHIC4hGRLFS+ax/XP7GAYX26cdvUY4IOJ+vEemjo/EZPc4AiQCUhRKTd6uudG55cwM7KGh696sRmTxuVxIn1X/zcRo9rgbWEDg+JiLTLr99cxf99tJXbzz+Wkf3btOwo7RTrGsEViQ5ERLLP+2u3c/erH3LumIFcfILODgpKrIeGhgH3ElogduAd4NvuvjqBsYlIBmqoKLqhohIzOLRrHj/94mhC56FIEGJdLJ4NPAkMIFQW4ilgTqKCEpHM1LiiqAP1Drv21fHasi1Bh5bVYk0EXd39UXevDX89BnROZGAiknmiVRTdV1uviqIBi3Wx+C9mNhN4nNChoS8DL5nZoQDu3ta6QyKSRVRRNDXFmgguCn+/OqL9YkKJYVjcIhKRjNW/Z2c27jjwzHNVFA1WrIng6MhbSZpZZ91eUkRi5e707tbxgESgiqLBi3WN4O0Y20REonrk7bUs2bCT88YOZFB+FwwYlN+F288/VhVFA9bS/Qj6A4MI3VZyHNBwflcPQreYFBFp0eLSHfz0peWcMbIvv/jyWJ0qmmJaOjQ0mdCdyAYDdzdq3wXckqCYRCSD7KyqYcbs+fQ5pCN3XThGSSAFtXQ/gkeAR8zsAnd/JkkxiUiGcPf91w08MX0Cvbp1DDokiSLWxeLRZnZASUB3vy3O8YhIBpn17npeXLSRm6aMoKjw0KDDkWbEmgh2N3rcGTgHWBb/cEQkU3ywYQe3/Xkppw4v4JpTjgg6HDmIWIvONb5fMWZ2F/ByQiISkbS3e18t180uoVfXPO6+aAw5OVoXSGVtLfzdldACsohIE+7OD55bzNpte5j9tQn0PqRT0CFJC2KtPrqYf92zOAfoC/xnWwc1s3zgd8Do8HavdPd32ro9EQleQ1XRsnC5iLNG92fCsN4BRyWxiHVGcA7QCzgZyAdecvd57Rj3XuCv7v4lM+uIrkkQSWsNVUUbF5R7Y8UW5paU6WKxNBDrlcVTgUeBPkAe8Aczu64tA5pZT+AU4PcA7l7t7hVt2ZaIpIZoVUUra1RVNF3EOiP4KjDB3fcAmNl/E7o5zf1tGHMoUE4omYwB5gHXN2y7gZlNB6YDDBkypA3DiEiyqKpoeot1RmBA43Rfx7/KTbRWB2A88Gt3HwfsAWZGdnL3B929yN2LCgoK2jiUiCRDfte8qO2qKpoeYp0R/AF418yeCz8/j/ChnTYoBUrd/d3w86eJkghEJD2s3LKL3VW15FjojmMNVFU0fcQ0I3D3u4ErgO3hryvc/Z62DOjum4CPzazhf8gZwNK2bEtEglVZXceMWSX06JLHreeOUlXRNBXzdQTuPh+YH6dxrwNmhc8YWk0oyYhImvnRCx+wYvMuHrnyRE4dXsBlJw0NOiRpg7ZeUNYu7r4AKApibBGJj7klZTxR/DHfmHQEpw7XOl46i3WxWERkv1Xlu7nlucWcUNiLGz47POhwpJ2UCESkVapq6pgxaz6dOuRw37RxdMjVx0i6C+TQkIikr9v+vJTlm3bxh8tPYEBPnR6aCZTKRSRmf1q4gdnvrufqU4Zx2si+QYcjcaJEICIxWbt1Dzc/u5jxQ/L5rq4PyCg6NCQizWqoKLqhopIOuUaHHOP+S8aTp3WBjKJ3U0SiaqgoWlZRiQM1dU5dPby/ZnvQoUmcKRGISFTRKopW16miaCZSIhCRqFRRNHsoEYhIVAN6do7aroqimUeJQESiGlbQ7YA2VRTNTEoEInKAPy/awN9XbmPS8D6qKJoFdPqoiDSxduseZj6zmHFD8vntZSfoVNEsoHdYRParqqnjG7Pmk5tj3D9tnJJAltCMQET2+8mLS1m6cSe/+/ciBvfqGnQ4kiRK9yIChOoIPfbP9Uw/ZRhnjuoXdDiSREoEIsKarXuY+cwixg/J11lBWUiJQCTLNawL5HXI4QHVEcpKWiMQyXK3/Xkpyzbu5KHLi3SxWJYKLBGYWS5QDJS5+zlBxSGSjRqqipaFy0WcPrKA00dqXSBbBTkHvB5YFuD4IlmpcVXRBm+v2sbckrIAo5IgBZIIzGwwcDbwuyDGF8lm0aqKVtWoqmg2C2pGcA9wE1DfXAczm25mxWZWXF5enrzIRDKcqopKpKQnAjM7B9ji7vMO1s/dH3T3IncvKigoSFJ0IpmvZ5e8qO1aKM5eQcwIJgJfMLO1wOPA6Wb2WABxiGSdpRt2squqhhxr2q6qotkt6YnA3W9298HuXghcDLzu7v+W7DhEss3Oqhq+MWsefbp34sdTj1FVUdlP1xGIZAF358anFvLxJ5U8MX0CRYWH8pUJhUGHJSki0ETg7m8AbwQZg0g2+P3f1/DyB5v5wdlHU1R4aNDhSIrRteQiGe79tdu5/S/LmXJMf676zNCgw5EUpEQgksG27t7HtbPnc1ivLtx54XGYWcs/JFlHiUAkQ9XVO9+cU0LF3hp+denx9Ogc/bRRES0Wi2Soe/72IW+v2sadXzqOUQN7BB2OpDDNCEQy0P+u2ML9r6/koqLBXFR0WNDhSIrTjEAkQzRUFN1QUQkGA3t25rapo4MOS9KAZgQiGaBxRVEH3GHbnmr+umRT0KFJGlAiEMkA0SqK7qtVRVGJjRKBSAZQRVFpDyUCkQzQ+5COUdtVUVRioUQgkuY27ahiX00dkZeKqaKoxEqJQCSNVdXUcfVj86hzuGnKCFUUlTbR6aMiacrd+Y+5S1j4cQW/+bfxTBk9gK9POjLosCQNaUYgkqb++M46nppXyjdPP5IpowcEHY6kMSUCkTT0zqpt3PbnpZx5dF++debwoMORNKdEIJJmSj/Zy4zZ8yns3ZVffHksOZH3nRRpJSUCkTRSWV3H1Y/Oo6a2nt/+exHdVVFU4kCLxSJpwt353jOLWLpxJw9ddgLDCg4JOiTJEEoEIimuoZhcWfgq4bOP7c9pI/sGHJVkEh0aEklhjYvJNXh9+RbmlpQFGJVkmqQnAjM7zMz+18yWmtkHZnZ9smMQSRfRislV1qiYnMRXEIeGaoHvuPt8M+sOzDOzV919aQCxiKQ0FZOTZEj6jMDdN7r7/PDjXcAyQNfBi0T4ZE81uc2cGqpichJPga4RmFkhMA54N8pr082s2MyKy8vLkx2aSKD2VtdyxcPv4+507ND011TF5CTeAksEZnYI8AzwLXffGfm6uz/o7kXuXlRQUJD8AEUCUlNXz9cfm8+i0gp+eenx3HnBcSomJwkVyOmjZpZHKAnMcvdng4hBJBXV1zs3PrWQNz8s547zj2XK6P4A+uCXhArirCEDfg8sc/e7kz2+SKpyd/7zxaXMXbCBGyeP4OIThwQdkmSJIA4NTQS+ApxuZgvCX58PIA6RlPKrN1bxh3+s5cqJQ/nGpCOCDkeySNIPDbn73+GAmymJZLXH31vPz15ewXljB/KDs48mNHEWSQ6VmBAJQEPZiA0VlfTq1pHte6o5dXgBd35pjKqJStIpEYgkWUPZiIYrhrfvqcYsVEMo8lRRkWTQ/zqRJItWNsId7n1tZUARSbZTIhBJMpWNkFSjRCCSRO5Ojy7Rj8iqbIQERYlAJEmqa+uZ+cxidlTWErkerLIREiQtFoskwfY91Vzz2DzeW7Od604/kmG9u3HXqx+yoaKSgflduHHyCF09LIFRIhBJsI827+KqR4rZtLOKey8ey9SxoQ/8Lx4/OODIREKUCEQS6H9XbOGbs0volJfLE9MnMG5Ir6BDEjmAEoFIHDW+UKxHlw7sqKxl1IAe/O6yIi0GS8pSIhCJk8gLxRoWhS8/6XAlAUlpOmtIJE6iXShWrwvFJA0oEYjEwery3ZTpQjFJUzo0JNIO67ft5d7XPuK5klIM8Ch9dFhIUp0SgUgLGi8AN5zzf8LQQ3ng9Y94qriU3BzjyolDGdK7K7e/tLzJ4SFdKCbpQIlA5CAiF4DLKir5zlMLcXc65ORw6aeG8I3TjqRfj84A9Oicd0DS0IVikuqUCEQOItoCcF2907VjLq/ecCqDIg77nDdukD74Je0oEUjWinbI57xxg3B3Sj+p5J1V25pdAK6srjsgCYikKyUCySjNfbhH6xd5yOfGpxfy2D/XsnHHvv0JIMdCp4BG0gKwZJJAEoGZTQHuBXKB37n7HfEeI9YPhNb01TZTe5vRPtxvfnYxAOeOGci23fvYtLOKTTuquPWFDw445FNT58xfX8GU0f25+tRhTBjWmw/KdnDLc0u0ACwZzdyjnfCWwAHNcoEPgc8CpcD7wDR3X9rczxQVFXlxcXHMY0R+IEDol/f2849t8cOjub7aZvK2OXXsQOrqndp6p96d5xeU8eM/LaWqpn5/304dcrjq5KEUHd6LvdV1VFbX8ZMXl7GjsoZIOQZmRl20P+0jGLDmjrMP2CctAEs6MrN57l7UYr8AEsGngR+5++Tw85sB3P325n6mtYlg4h2vRz222yHHGNqnW5O2NVv3UBvlAyKyb6z94rXNwohtro2x78H6Hd67a5O2ddv2Ntt3yKH/6rt+e/R+uTnG4F6hQyQN/43KKiqjfuDmGPQ5pFP4MIuzfU911EMuifKNSUfQv2dn+vXoTP8enbn60Xls2ll1QL9B+V34x8zTkxeYSALFmgiCODQ0CPi40fNS4FORncxsOjAdYMiQIa0aoLkrOWvrnaP6HdKk7aMtu2PqG2u/eG1zRL/uTdpWxtj3YP1GDujRpG1V+Z5m+44a2AOz0N1TVm+N3q+u3hl7WD4N91gxM9Zv3xu1b73DGUf3BYwcg1nvro/aD+D6M44iN8fIzTE65Bi3/2V51H4GPDdjIl075tIlL5cLf/NOsx/uN00Z2aRt5lkjo85IdMhHslHKLha7+4PAgxCaEbTmZwfmd4k6IxiU34VfXXp8k7bmZg+RfWPtF69t/vLS8U3aFsTY96D9LonY5vrm+z7QqO/8dZ802+/ei8c1aXtvzfZm+95+/nH7n7+xorzZft/+7PAmbX98Z13UvgPzuzD2sPz9z1vz4d5waEeHfESCqTVUBhzW6PngcFvc3Dh5BF3ycpu0NfeBEGtfbTP1t3neuEHcfv6xDMrvgtGQfA5cx2jc/x8zT2fNHWfzj5mnKwlI1gpiRvA+cJSZDSWUAC4GLonnAK35ay/Wvtpm6m+zoa8+0EVaJ+mLxQBm9nngHkKnjz7k7v91sP6tXSwWEZHUXizG3V8CXgpibBERaUr3IxARyXJKBCIiWU6JQEQkyykRiIhkuUDOGmotMysH1rXxx/sAW+MYTirItH3KtP2BzNunTNsfyLx9irY/h7t7QUs/mBaJoD3MrDiW06fSSabtU6btD2TePmXa/kDm7VN79keHhkREspwSgYhIlsuGRPBg0AEkQKbtU6btD2TePmXa/kDm7VOb9yfj1whEROTgsmFGICIiB6FEICKS5TI2EZjZhWb2gZnVm1lRo/ZCM6s0swXhr98EGWesmtuf8Gs3m9lKM1thZpODirE9zOxHZlbW6H35fNAxtYWZTQm/DyvNbGbQ8cSDma01s8Xh9yXtygCb2UNmtsXMljRqO9TMXjWzj8LfewUZY2s1s09t/h3K2EQALAHOB96K8toqdx8b/romyXG1VdT9MbNRhO7pcAwwBfiVmeUe+ONp4ReN3pe0q04b/nf/JXAWMAqYFn5/MsFp4fclHc+7f5jQ70ZjM4HX3P0o4LXw83TyMAfuE7TxdyhjE4G7L3P3FUHHES8H2Z+pwOPuvs/d1wArgROTG52EnQisdPfV7l4NPE7o/ZEAuftbwPaI5qnAI+H5/SU0AAADS0lEQVTHjwDnJTWodmpmn9osYxNBC4aaWYmZvWlmJwcdTDsNAj5u9Lw03JaOrjWzReFpb1pN1cMy6b1ozIFXzGyemU0POpg46efuG8OPNwH9ggwmjtr0O5TWicDM/mZmS6J8HeyvsI3AEHcfB9wAzDazHsmJ+ODauD9po4X9+zVwBDCW0Hv080CDlcY+4+7jCR3ymmFmpwQdUDx56Bz6TDiPvs2/Q4HcoSxe3P3MNvzMPmBf+PE8M1sFDAcCXwRry/4Quu/zYY2eDw63pZxY98/Mfgv8OcHhJELavBet4e5l4e9bzOw5QofAoq29pZPNZjbA3Tea2QBgS9ABtZe7b2543NrfobSeEbSFmRU0LKaa2TDgKGB1sFG1ywvAxWbWycyGEtqf9wKOqdXCv4wNvkhocTzdvA8cZWZDzawjoUX8FwKOqV3MrJuZdW94DHyO9HxvIr0AXBZ+fBnwfICxxEV7fofSekZwMGb2ReB+oAB40cwWuPtk4BTgNjOrAeqBa9w9bosuidLc/rj7B2b2JLAUqAVmuHtdkLG20Z1mNpbQFH0tcHWw4bSeu9ea2bXAy0Au8JC7fxBwWO3VD3jOzCD0eTHb3f8abEitY2ZzgElAHzMrBW4F7gCeNLOrCJW4vyi4CFuvmX2a1NbfIZWYEBHJcll3aEhERJpSIhARyXJKBCIiWU6JQEQkyykRiIhkOSUCkUbM7O0EbLPQzC6J93ZF4kWJQKQRdz8pAZstBJQIJGUpEYg0Yma7w98nmdkbZva0mS03s1kWvqoqXJ//znCN/vfM7Mhw+8Nm9qXIbRG6eOnkcI34byd7n0RaokQg0rxxwLcI3VtgGDCx0Ws73P1Y4AHgnha2MxP4v3CN+F8kJFKRdlAiEGnee+5e6u71wAJCh3gazGn0/dPJDkwknpQIRJq3r9HjOprW5vIoj2sJ/06ZWQ7QMaHRicSJEoFI23y50fd3wo/XAseHH38ByAs/3gV0T1pkIq2UsdVHRRKsl5ktIjRrmBZu+y3wvJktBP4K7Am3LwLqwu0Pa51AUo2qj4q0kpmtBYrcfWvQsYjEgw4NiYhkOc0IRESynGYEIiJZTolARCTLKRGIiGQ5JQIRkSynRCAikuX+H+o3zP3o4wGWAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cx.plot_f(softplus, frange=(-15, 15, 1), xlabel=\"input\", ylabel=\"output\", title=\"softplus()\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## softsign"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function softsign in module conx.activations:\n",
      "\n",
      "softsign(x)\n",
      "    Softsign activation function.\n",
      "    \n",
      "    >>> softsign(1)\n",
      "    0.5\n",
      "    >>> softsign(-1)\n",
      "    -0.5\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(softsign)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.3333333432674408"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "softsign(0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEWCAYAAABBvWFzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xt8XVWZ//HP0zS3XtNL6CW9Q21pqbRDRBSUS5GCo8AoYp1xBhz8If7A8QbSqj9wUMYqKsyM16pIvRbk0lapVi4yOqJCsKVJC4VSivQ0bUMvadrck+f3x9mpJ8k5yUlyTvY5J9/365VX9l577X2e3ZA8rLX22svcHRERkYEaFnYAIiKSG5RQREQkJZRQREQkJZRQREQkJZRQREQkJZRQREQkJZRQRFLIzD5sZvvN7JiZTejjudvM7LwUxVFoZtvNbEqw/1Uz+3Aqri2SiGkeikhqmFk+cBQ4y92fDcocmOvuOwc5lo8AC939umB/CvAUcLK7Nw9mLDJ0qIUikjqTgCJgW9iBANcBP+rYcfdq4Hng0tAikpynhCKSgJndbGYRM6szsx1mtjToSrrLzPYGX3cFZa8DdgSnHjGzx83sd8H+s0EX2HvNbKKZ/dLMjpjZITP7vZkNCz5vt5ldGGwXm9kaMztsZs+Z2afMbE9MbLvN7EYz22pmtWZ2r5kVBcdmAHOAP3e5pSeAv0/fv5gMdcPDDkAkE5nZPOAG4A3uvtfMZgF5wGeAs4DFgAPrgc+6+/8zs4XAy0CJu7cG13Hg9I4uLzP7IrAHKA0+6qzgOl3dCswimhhGAhvj1LkSuBhoBP4AXA18G1gE7OqIIcZzwLv78M8g0idqoYjE1wYUAgvMLN/dd7v7S8A/Abe5+wF3rwH+HfjnPly3BZgCzHT3Fnf/vccfyLwS+A93P+zue4D/ilPnv9x9r7sfAn5BNMkBlAB1cerXBcdE0kIJRSSOoEXxMeBzwAEzW2tmU4GpwCsxVV8JypJ1B7AT+I2Z7TKzFQnqTQVejdl/NU6dfTHb9cCoYPswMDpO/dHAkT7EKtInSigiCbj7T939HGAm0W6pLwF7g/0OM4KyZK9Z5+6fdPc5RAfIP2FmS+NUrQamxexP70PoW4HZZta1S/tU4Nk+XEekT5RQROIws3lmdoGZFRIdo2gA2oGfAZ81s1IzmwjcAvy4h0vtJzoO0nHdd5jZKWZmQC3RrrX2OOfdB6w0s3FmVkZ0PCcpQRfZTuDMLofOBX6V7HVE+koJRSS+QmAV8BrRrqWTgJXAF4AKoq2ASuAvQVkinwPWBE91XQnMBR4FjgF/BL7p7r+Nc95tRAfvXw7q3w809SH+7xAzthPMQ1kArOvDNUT6RBMbRbJAMMt9ubufm2T9QmAzsNTdq83sq8BL7v7NdMYpQ5sSikgGCloUc4i2YuYCDwNfd/e7Qg1MpAeahyKSmQqIdlvNJvpk1lpArQvJaGqhiIhISmhQXkREUiLULi8zuxt4B3DA3U+Lc9yA/wTeTnTi1tXu/pfg2FXAZ4OqX3D3Nb193sSJE33WrFkpil5EZGh45plnXnP30t7qhT2Gcg/wdeCHCY5fQnRAci7wRuBbwBvNbDzRdx2VE51w9oyZbXD3wz192KxZs6ioqEhR6CIiQ4OZvdJ7rZC7vNz9d8ChHqpcBvzQo/4ElARPvywDHnH3Q0ESeYToS/JERCQkmT6GUkbndxjtCcoSlXdjZteaWYWZVdTU1KQtUBGRoS7TE8qAuftqdy939/LS0l67AEVEpJ8yPaFE6PxSvGlBWaJyEREJSaYnlA3Av1jUWUBtsJTpJuCi4MV544CLgjIREQlJ2I8N/ww4D5gYLG96K5AP4O7fJrpK3duJvjm1HvhAcOyQmX0eeDq41G3BIkMiIhJYtznCHZt2sPdIA1NLirlp2TwuXxJ3uDklhtRM+fLyctdjwyKSifryxz+Zuus2R1j5YCUNLW0nyorz8/jiuxb1OamY2TPuXt5bvbDnoYiI5LT+/PGPHGlg5YOVAEnVvfmBrUSONHDWnPEca2qjvqmVz/1iW6dkAtDQ0sYdm3akrZWihCIi0kfJtiYSJQp356KFkzna2EJtQwtfeHh73D/+n36okseeP8CxxhbqGls51tTKi/uP0dalZ6mptZ07Nu1IKva9Rxr6ede9U0IREQn0tzWxImghLJ5ewuH6Zg4fb+ZwfQvf+d1LcRPFx+97lmRWY65vbqMqUsvoouGMKhzOjJEjeH5fXcL6a/71TEYV5jGiYDhX/+Ap9h/tvibb1JLiJP4l+kcJRURy2kBaEzc/sJVn9xxhTukoXqtr4rVjTTzwzB4aWzuv2tzYhxZChxWXzGdscT5jivK5ZX0VB483d6tTVlLMb288r1PZ2aseJxKnlVFWUsy5r/vbXLuVl5wadwzlpmXz+hRnXyihiEhW6ndr4sGt7D/awKlTxlJd20B1bSPVRxpZtyVCU5dE0dTazg/+sBsAMxg/oqBbMom19tqzGD+ygJIR+ZQUF3D+V55I+Mf/unNPPrHf0tae9B//m5bNS6pux7/FYD7lpYQiIlmnp0Hsd7x+CtW1jew+eJxbN3QfmG5saeeLv+rcmigdXdgtmXQw4M+fXsr4kQUMzxvWYwvhrDkTOpWl449/X+umM4F0pceGRSRjJNs9dfaqx4gcaexWPnyYYQYtbb3/XbvvQ29iytgiJo0pomB4z4niDysu6BRjXx7HHey5IOmgx4ZFJKskanW0tLUzf/IYntt3lOer63h+39G4yQSgtd35v+edzMwJI5g5YSQfW7uFfUe71y0rKebM2eM7laWrK2mwWwlhUkIRkYxwx6YdcZ+Iuun+rSf2i/KHMW/yGEYU5FHf3Nb1EpSVFPOpi+ef2F9xyfykxyYyuSspWyihiEjaJer2qa1voeKVQzy1+1Dc7qYO3/qnv2P+lDHMGD+CvGGWsNtJrYlwaQxFRNIq3h//vGHGSaMK2FfXhDvk5xmG0dzWfWC86xhG7HWzfWwiW2gMRUQywn9sfK5bV1Zbu3OovoWPX/g63jBrPEtmlPDrqn19mjeh1kTmUUIRkX7pqYWw88AxHt5azcOVezlQ1322NkBzazv/tnTuif0w5k1IaimhiEifJZpV/ptt+9j12nGe31eHGbxh1njGFudT29DS7RrxXgGiVkd2U0IRkT6L90RWU2s7G6v28YZZ4/jcOxdwyaIpTBpTlPQAumQ/JRQR6bNEb6w14OfXvblTmbqyhg4lFBFJ2uHjzXz397sSHk/0Jlt1ZQ0NYS8BfDHwn0Ae8D13X9Xl+J3A+cHuCOAkdy8JjrUBlcGxv7r7pYMTtcjQc6S+me/9/mXueXI3x5tbOX36WJ6rruv0/it1Y0loCcXM8oBvAG8D9gBPm9kGd9/eUcfdPx5T/yPAkphLNLj74sGKV2SoiH16a/LYIl5fNoYnXzpEXVMrf79oCv+2dC7zJo/WPBDpJswWypnATnffBWBma4HLgO0J6r8PuHWQYhMZkroOoFfXNlJd28jrp43hy1eczvzJY07UVTeWdDUsxM8uA16N2d8TlHVjZjOB2cDjMcVFZlZhZn8ys8vTF6bI0BHv6S2Ag8daOiUTkXiyZVB+OXC/u8f+lz7T3SNmNgd43Mwq3f2lriea2bXAtQAzZswYnGhFslSi92mlcx1yyR1htlAiwPSY/WlBWTzLgZ/FFrh7JPi+C3iCzuMrsfVWu3u5u5eXlpbGqyIy5Lk733xiZ8Lj6VyHXHJHmAnlaWCumc02swKiSWND10pmNh8YB/wxpmycmRUG2xOBs0k89iIiPWhqbeOTP3+WL/96B0uml1CU3/nPgp7ekmSF1uXl7q1mdgOwiehjw3e7+zYzuw2ocPeO5LIcWOudX4t8KvAdM2snmhRXxT4dJiLJOXisiQ/96BkqXjnMxy6cy0eXzmX9lr16ekv6Ra+vFxmiduyr45o1T1NT18RX3nM67zx9atghSYbS6+tF5ISuc0b+ftFkfvrUqxQX5HHvh97E4uklYYcoOUAJRSTHxXsz8Orfv8zUsUXc/+E3a8BdUibMQXkRGQSJ5pY4enpLUksJRSTHJZpDsq+2cZAjkVynhCKS4xK1QtQ6kVRTQhHJcTctm0eeWacyzS2RdFBCEclxk8YU0ebO6KLhGFBWUswX37VIc0sk5fSUl0gOa2lr59YNVUwbV8yjnziXovy8sEOSHKYWikgOW/Pkbl7Yf4xb3rFAyUTSTglFJEcdqGvkrkdf5Lx5pbxtwaSww5EhQAlFJEet2vg8za3t3PrOhViXQXmRdFBCEclBT+8+xIObI/yft85m9sSRYYcjQ4QSikiOaW1r55b125g6tojrzz8l7HBkCFFCEckxP/nzX3mu+iiffccCRhToQU4ZPEooIjnktWNNfPU3Ozj7lAlcctrksMORIUYJRSSHfPnXz1Pf3Ma/X6qBeBl8SigiOWLzXw9zX8UerjlnNqecNDrscGQIUkIRyQFt7c4t67cxaUwhH1k6N+xwZIgKNaGY2cVmtsPMdprZijjHrzazGjPbEnx9MObYVWb2YvB11eBGLpIZ1m2OcPaqxzn50xupjNRy0YJJjCrUQLyEI7T/8swsD/gG8DZgD/C0mW1w9+1dqt7r7jd0OXc8cCtQTnSdoGeCcw8PQugiGaHrSowA9z+zhzNmjteLHyUUYbZQzgR2uvsud28G1gKXJXnuMuARdz8UJJFHgIvTFKdIRoq3EmNDSzt3bNoRUkQy1IWZUMqAV2P29wRlXb3bzLaa2f1mNr2P52Jm15pZhZlV1NTUpCJukYyQaCXGROUi6Zbpg/K/AGa5++uJtkLW9PUC7r7a3cvdvby0tDTlAYqERSsxSqYJM6FEgOkx+9OCshPc/aC7NwW73wPOSPZckVx307J5FOR1/hXWSowSpjATytPAXDObbWYFwHJgQ2wFM5sSs3sp8FywvQm4yMzGmdk44KKgTGTIuHxJGRfMPwlAKzFKRgjtKS93bzWzG4gmgjzgbnffZma3ARXuvgH4NzO7FGgFDgFXB+ceMrPPE01KALe5+6FBvwmRkLW2O3NKR/L4J88LOxSRcJcAdveNwMYuZbfEbK8EViY4927g7rQGKJLhqiK1vHHO+LDDEAEyf1BeRBKoqWti39FGTps6NuxQRAAlFJGsVbW3FoDTypRQJDMooYhkqao90YSysGxMyJGIRCmhiGSpykgtsyeOZExRftihiABKKCJZqypSq+4uyShKKCJZ6OCxJvbWNrJI3V2SQZRQRLJQ1d6jgAbkJbMooYhkoapIMCCvR4YlgyihiGShyj21zJwwgrHFGpCXzKGEIpKFKjUgLxlICUUkyxw+3kzkSAOLlFAkwyihiGSZymD8RAlFMo0SikiWOfHKFQ3IS4ZRQhHJMlWRWmaMH8HYERqQl8yihCKSZSojterukoykhCKSRY7UN/PqoQa9EFIykhKKSBapikRnyKuFIpko1IRiZheb2Q4z22lmK+Ic/4SZbTezrWb2mJnNjDnWZmZbgq8NXc8VyUUdT3hpQF4yUWhLAJtZHvAN4G3AHuBpM9vg7ttjqm0Gyt293sw+DHwZeG9wrMHdFw9q0CIhq9pby7RxxYwbWRB2KCLdhNlCORPY6e673L0ZWAtcFlvB3X/r7vXB7p+AaYMco0hGqdKAvGSwMBNKGfBqzP6eoCyRa4BfxewXmVmFmf3JzC5PdJKZXRvUq6ipqRlYxCIhqm1o4ZWD9XrlimSs0Lq8+sLM3g+UA+fGFM9094iZzQEeN7NKd3+p67nuvhpYDVBeXu6DErBIGmzTDHnJcGG2UCLA9Jj9aUFZJ2Z2IfAZ4FJ3b+ood/dI8H0X8ASwJJ3BioTtxIC8EopkqDATytPAXDObbWYFwHKg09NaZrYE+A7RZHIgpnycmRUG2xOBs4HYwXyRnFMZqaWspJjxGpCXDBVal5e7t5rZDcAmIA+42923mdltQIW7bwDuAEYBPzczgL+6+6XAqcB3zKydaFJc1eXpMJGcE11DXhMaJXOFOobi7huBjV3KbonZvjDBeU8Ci9IbnUjmONrYwu6D9Vxxhh50lMylmfIiWWBbRGvIS+ZTQhHJAlV6wkuygBKKSBaojNQydWwRE0YVhh2KSEJKKCJZoEpryEsWUEIRyXB1jS3seu24EopkPCUUkQy3fa9eWS/ZQQlFJMNphrxki6QSipl9NJkyEUm9qkgtk8cUUTpaA/KS2ZJtoVwVp+zqFMYhIglUakBeskSPM+XN7H3APwKzu6yKOBo4lM7ARASONbWy67XjXHp6Tys7iGSG3l698iRQDUwEvhpTXgdsTVdQIhK1fe9R3GHRNL3DSzJfjwnF3V8BXgHeNDjhiEisKq0hL1kkqZdDmlkd0LE4VQGQDxx3d/1vk0gaVUVqOWl0ISeNKQo7FJFeJZVQ3H10x7ZF3yN/GXBWuoISkahKrSEvWaTP81A8ah2wLA3xiEigvrmVl2qO6QkvyRrJdnm9K2Z3GNH13RvTEpGIANEB+XbXDHnJHskusPXOmO1WYDfRbi8RSZOOGfKLpimhSHZIdgzlA+kOREQ6q4ocpXR0IZM0IC9ZItlXr8wxs1+YWY2ZHTCz9WY2Z6AfbmYXm9kOM9tpZiviHC80s3uD4382s1kxx1YG5TvMTOM5kjPWbY5w9qrHeeAvezja0MK6zZGwQxJJSrKD8j8F7gOmAFOBnwM/G8gHm1ke8A3gEmAB8D4zW9Cl2jXAYXc/BbgT+FJw7gJgObAQuBj4ZnA9kay2bnOElQ9WEjnSAEBTazsrH6xUUpGskGxCGeHuP3L31uDrx8BA2+FnAjvdfZe7NwNr6T4ucxmwJti+H1ga89jyWndvcveXgZ3B9USy2h2bdtDQ0taprKGljTs27QgpIpHkJZtQfmVmK8xslpnNNLNPARvNbLyZje/nZ5cBr8bs7wnK4tZx91agFpiQ5LkAmNm1ZlZhZhU1NTX9DFVkcOwNWibJlotkkmSf8roy+P6hLuXLic6gH/B4Srq4+2pgNUB5ebn3Ul0kVFNLik90d3UtF8l0ybZQTnX32bFfMWX9TSYRYHrM/rSgLG4dMxsOjAUOJnmuSNa5adk8ivM7DwcW5+dx07J5IUUkkrxkE8qTSZb1xdPAXDObbWYFRFs7G7rU2cDf1mK5Anjc3T0oXx48BTYbmAs8NcB4REJ3+ZIybrts4Yn9spJivviuRVy+RK+vl8zX23ook4mOTRSb2RLAgkNjgBED+WB3bzWzG4BNQB5wt7tvM7PbgAp33wB8H/iRme0kuv7K8uDcbWZ2H7Cd6ETL6929Le4HiWSZU04aBcC3338GF582OeRoRJLX2xjKMqIrM04DvhZTXgd8eqAf7u4bgY1dym6J2W4E3pPg3NuB2wcag0imqdIMeclSva2HsgZYY2bvdvcHBikmkSGtMlLL+JEFTB2rGfKSXZJ9yus0M1vYtdDdb0txPCJDXmXkKKeVjSU65UokeyQ7KH8MOB58tRGd3T4rTTGJDFmNLW28uL+ORWVau06yT7Ivh4xdTx4z+wrRwXQRSaHn99XR2u56Zb1kpT4vsBUYQXSgXkRSqOOV9VpUS7JRsgtsVfK3NeWHAScBn09XUCJD1bZILSUj8inTzHjJQskOyr8DGAe8BSgBNrr7M2mLSmSI6lhDXgPyko2S7fK6DPgRMBHIB35gZh9JW1QiQ1BTaxsv7K9Td5dkrWRbKB8EznL34wBm9iXgj8B/pyswkaFmx746Wto0IC/ZK9kWihF9XLhDG397DYuIpMCJNeSVUCRLJdtC+QHwZzN7KNi/nOh7tkQkRaoitYwtzmfaOA3IS3ZKdh7K18zsCeCcoOgD7r45bVGJDEGVkVpOKxujAXnJWsm2UHD3vwB/SWMsIkNWU2sbO/bV8a/nzA47FJF+6+/ERhFJoRf3H9OAvGQ9JRSRDKABeckFSigiGaAyUsuYouHMGD+gdetEQqWEIpIBqiK1emW9ZL1QEoqZjTezR8zsxeD7uDh1FpvZH81sm5ltNbP3xhy7x8xeNrMtwdfiwb0DkdRpbm3n+eo6dXdJ1gurhbICeMzd5wKPBftd1QP/4u4LgYuBu8ysJOb4Te6+OPjakv6QRdLjhf11NLe1s1AJRbJcWAnlMmBNsL2G6ETJTtz9BXd/MdjeCxwASgctQpFBUqUBeckRYSWUSe5eHWzvAyb1VNnMzgQKgJdiim8PusLuNLPCHs691swqzKyipqZmwIGLpFrV3lpGFw5npgbkJculLaGY2aNmVhXn67LYeu7u/G2tlXjXmUL0TccfcPf2oHglMB94AzAeuDnR+e6+2t3L3b28tFQNHMk8lZGjLCwbw7BhGpCX7Jb0TPm+cvcLEx0zs/1mNsXdq4OEcSBBvTHAw8Bn3P1PMdfuaN00mdkPgBtTGLrIoGlpa+e56qNc9aaZYYciMmBhdXltAK4Ktq8C1netYGYFwEPAD939/i7HpgTfjej4S1VaoxVJkxf3H6O5tV1roEhOCCuhrALeZmYvAhcG+5hZuZl9L6hzJfBW4Oo4jwf/JFiWuJLool9fGNzwRVKjSmvISw5JW5dXT9z9ILA0TnkF0cW8cPcfAz9OcP4FaQ1QZJBURmoZVTic2RNGhh2KyIBpprxIiCojtSyYqgF5yQ1KKCIhaQ0G5DX/RHKFEopISHbWHKOptV0JRXKGEopISCr3aEBecosSikhIqiK1jCjIY/ZEDchLblBCEQlJZaSWhVPHkKcBeckRSigiIWhta2d79VF1d0lOUUIRCcFLNcdpbNGAvOQWJRSREGgNeclFSigiIegYkJ9TOirsUERSRglFJARVkVoWTNGAvOQWJRSRQdbW7mzbqwF5yT1KKCKDbFfNMRpa2pRQJOcooYgMMg3IS65SQhEZZJWRWoryh3FyqWbIS25RQhEZZB0D8sPz9OsnuUX/RYsMoo4BeXV3SS5SQhEZRC+/dpz65jYWKqFIDgoloZjZeDN7xMxeDL6PS1CvLWY9+Q0x5bPN7M9mttPM7jWzgsGLXqR/1m2O8J5vPwnAVzbtYN3mSMgRiaRWWC2UFcBj7j4XeCzYj6fB3RcHX5fGlH8JuNPdTwEOA9ekN1yRgVm3OcLKBys5XN8CwIG6JlY+WKmkIjklrIRyGbAm2F4DXJ7siWZmwAXA/f05XyQMd2zaQUNLW6eyhpY27ti0I6SIRFIvrIQyyd2rg+19wKQE9YrMrMLM/mRmHUljAnDE3VuD/T1AWaIPMrNrg2tU1NTUpCR4kb7ae6ShT+Ui2Wh4ui5sZo8Ck+Mc+kzsjru7mXmCy8x094iZzQEeN7NKoLYvcbj7amA1QHl5eaLPEUmryWOLqK5t7FY+taQ4hGhE0iNtCcXdL0x0zMz2m9kUd682synAgQTXiATfd5nZE8AS4AGgxMyGB62UaYA6oiWjnVI6sltCKc7P46Zl80KKSCT1wury2gBcFWxfBazvWsHMxplZYbA9ETgb2O7uDvwWuKKn80Uyxba9tfzhpYOcc8oEykqKMaCspJgvvmsRly9J2FsrknXS1kLpxSrgPjO7BngFuBLAzMqB69z9g8CpwHfMrJ1o4lvl7tuD828G1prZF4DNwPcH+wZEkuHu3Lp+GyUjCvjGP57B2BH5YYckkjahJBR3PwgsjVNeAXww2H4SWJTg/F3AmemMUSQVHtocoeKVw3zp3YuUTCTnaaa8SJrUNbbwHxuf5/TpJbznjOlhhyOSdmF1eYnkvLsefZGDx5u4++pyhmllRhkC1EIRSYMX9tdxz5O7Wf6GGbx+WknY4YgMCiUUkRRzd25ZX8XoouF6LFiGFCUUkRT75dZq/rTrEDdeNI/xI/XeUhk6lFBEUuh4Uyu3P/wcp5WN4X1nzgg7HJFBpUF5kRT678d3su9oI9/4p78jTwPxMsSohSKSIi/VHOP7/7uL95wxjTNmxl3iRySnqYUiMgDrNke4Y9MO9h5poGD4MIYZ3HzJ/LDDEgmFWigi/dSxaFbkSAMONLW20+7wvy++FnZoIqFQQhHpp3iLZrW0uRbNkiFLCUWkn7RolkhnSigi/TRxVGHcci2aJUOVEopIP6zfEuHQ8Sa6PhisRbNkKFNCEemD9nbna4+8wEfXbuGMmeP5/OWnadEskYAeGxZJUkNzGzfe/ywPb63mijOmcfs/nEbh8Dzef9bMsEMTyQhKKCJJ2H+0kWt/WMHWSC0rL5nPtW+dg5lmwovECiWhmNl44F5gFrAbuNLdD3epcz5wZ0zRfGC5u68zs3uAc4Ha4NjV7r4lzWHLEBI7YXHi6EKaWtpobXdW/3M5b1swKezwRDJSWGMoK4DH3H0u8Fiw34m7/9bdF7v7YuACoB74TUyVmzqOK5lIKnWdsFhT10RdYyvXn3+KkolID8JKKJcBa4LtNcDlvdS/AviVu9enNSoR4k9YdOCnf/5rOAGJZImwEsokd68OtvcBvf1v33LgZ13KbjezrWZ2p5nFnxAAmNm1ZlZhZhU1NTUDCFmGgrrGFiKasCjSL2kbQzGzR4HJcQ59JnbH3d3MvIfrTAEWAZtiilcSTUQFwGrgZuC2eOe7++qgDuXl5Qk/R4a2usYW1jy5m+/+/uWEdTRhUaRnaUso7n5homNmtt/Mprh7dZAwDvRwqSuBh9y9JebaHa2bJjP7AXBjSoKWnBY70D61pJibls3jwgWTgkSyiyP1LSydfxKLp5fwzSde6tTtpQmLIr0L67HhDcBVwKrg+/oe6r6PaIvkhJhkZETHX6rSFajkho6B9o4kETnSwI0/f5aCPKO+pZ0L5p/ER5fO5fTpJQBMHz+iW/LRhEWRnoWVUFYB95nZNcArRFshmFk5cJ27fzDYnwVMB/6ny/k/MbNSwIAtwHWDE7Zkq3gD7a3tTt4wY931Z7M4SCQdLl9SpgQi0kehJBR3PwgsjVNeAXwwZn830O232t0vSGd8klsOHmtKONDe3NreLZmISP9oprxkvXhjI299XSmbtu3j4a3V/HHXwYTnaqBdJHWUUCSrxRsb+cR9W3CPzh2ZNWEEHz73ZIoL8vj64y/S0NJ+4lwNtIuklhKKZKx4LY/YcY3jTa18/pfbu42NtDuMKhzOvR86iwVHL8vEAAAJc0lEQVRTxpx451ZZSbEG2kXSSAlFMlK8lseKB7dSuecIw4YZT718iKq9R2lrjz+16HhTKwunju1UpoF2kfRSQpFB1Vuro8OXNz3freXR2NLO9/+wm4K8YSyeXsJ1585h7VOvcvB4c7fzNTYiMviUUGTQxGt1rHywkmNNLcyeOIrnqo/y/L46nt93lL1HGuNew4Ctn7uIovw8AOaeNLrTNUFjIyJhUUKRlOit5eHurPpV91ZHQ0sbn1237cT+xFGFnDplNKMK8zjW1LkuRFseHckEOPEZGhsRCZ+5D53XW5WXl3tFRUXYYWSNZLunurY8APLzjLecMpH84cN45WA9rxys75ZMYv34mjcyb/JoSkcXJrxmcX6eltgVCYGZPePu5b3VUwtlCEomUSQaFH/1cD3zJ4+huraBvUca2VfbwMbKfTS3tXc6v6XNeXxHDSeXjmTWhJG8+eSJPPCXPdQ2tNBVWUkx58yd2KlMLQ+R7KOEkuH60kroT2sicqSBmx/Yyl8PHWfh1LHU1DXx2rEmvtXl5YgQHRT/6m9eOLGfn2dMGlPULZl0MOCxT553Yv/108b2abxDT2WJZBcllBQajD/+Kx+sBOhUP1GSqIzUMm/yaA4fb+ZwfQuHjzezfkuExtbOCaCptZ2vPfJi0ve5/vqzmVJSxMSRhQwbZpy96vG4rzbp+qSVWh0iuU1jKL0YyDhCvD7/nupdsmgyxxpbOdbUSl1jK1f/4CleO9b9kdgRBXmcc8pEjja2UNvQygv76mjr5edYkDeMcSPz2X+0KfG9Xn82paMLmTCygKVf/Z+4SaKspJg/rOj8KjWNd4jkNo2hpEBPLYS3L5pCQ3Mb9S2t1De3cfvG5+I+wXTL+ip2HjjG8eZWjje18otn93Z6/UdHvY/du4WP3ZtcXPXNbbxysJ6xxfmUlRTxXPXRuPUM+N2nzmf8yAJGFORhlrg1UVZS3OkliTctm5d095RaHiICaqH0KNEf374aZjCyYDgjCvN6bCHctGweowqHM7poOKMKh/PphyrjtlC6thJ6ShIDaU0k2zoTkdymFkoK9LSG+I0XvY7iguGMKMijOD+P2365nUPxZmyPLeIPKy448T6pnv74X3/+KZ3K6pvbkmolpKs1oUFxEekLJZQeTC0pTvjH/4YL5nYrj/dH/VMXzz+RTCA9f/z72uWkRCEi6aAurx70dbA51U95iYhkgmS7vEJJKGb2HuBzwKnAmcFKjfHqXQz8J5AHfM/dVwXls4G1wATgGeCf3b17f1MX6XzKS0QkV2X6GEoV8C7gO4kqmFke8A3gbcAe4Gkz2+Du24EvAXe6+1oz+zZwDfCtdASq7iERkeQMC+ND3f05d9/RS7UzgZ3uvitofawFLrPogMQFwP1BvTXA5emLVkREkhFKQklSGfBqzP6eoGwCcMTdW7uUi4hIiNLW5WVmjwKT4xz6jLuvT9fnxonjWuBagBkzZgzWx4qIDDlpSyjufuEALxEBpsfsTwvKDgIlZjY8aKV0lCeKYzWwGqKD8gOMSUREEsjkLq+ngblmNtvMCoDlwAaPPpb2W+CKoN5VwKC1eEREJL6wHhv+B+C/gVLgCLDF3ZeZ2VSijwe/Paj3duAuoo8N3+3utwflc4gO0o8HNgPvd/fE7zT52+fWAK/0M+yJwGv9PDdT5do96X4yX67dU67dD8S/p5nuXtrbiUNqYuNAmFlFMs9hZ5NcuyfdT+bLtXvKtfuBgd1TJnd5iYhIFlFCERGRlFBCSd7qsANIg1y7J91P5su1e8q1+4EB3JPGUEREJCXUQhERkZRQQhERkZRQQumFmb3HzLaZWbuZlceUzzKzBjPbEnx9O8w4k5XofoJjK81sp5ntMLNlYcU4EGb2OTOLxPxc3h52TP1hZhcHP4edZrYi7HhSwcx2m1ll8HPp2zoSGcDM7jazA2ZWFVM23sweMbMXg+/jwoyxrxLcU79/h5RQetfxqv3fxTn2krsvDr6uG+S4+ivu/ZjZAqJvI1gIXAx8M1hCIBvdGfNz2Rh2MH0Vs3TDJcAC4H3BzycXnB/8XLJx7sY9RH83Yq0AHnP3ucBjwX42uYfu9wT9/B1SQulFkq/azxo93M9lwFp3b3L3l4GdRJcQkMEXd+mGkGMa8tz9d8ChLsWXEV1CA7JwKY0E99RvSigDM9vMNpvZ/5jZW8IOZoASLReQjW4ws61Bcz6ruiACufSziOXAb8zsmeAt4LlgkrtXB9v7gElhBpNC/fodUkIh+qp9M6uK89XT/xVWAzPcfQnwCeCnZjZmcCLuWT/vJ2v0cn/fAk4GFhP9GX011GAl1jnu/ndEu/KuN7O3hh1QKgUvrs2FeRj9/h0KawngjNKfV+0HL6NsCrafMbOXgNcBoQ829nPpgETLBWScZO/PzL4L/DLN4aRD1vws+sLdI8H3A2b2ENGuvXhjk9lkv5lNcfdqM5sCHAg7oIFy9/0d2339HVILpZ/MrLRj0Dp4+/FcYFe4UQ3IBmC5mRWa2Wyi9/NUyDH1WfBL3eEfiD6EkG3iLt0QckwDYmYjzWx0xzZwEdn5s+lqA9ElNCBHltIYyO+QWii96PKq/YfNbIu7LwPeCtxmZi1AO3Cdu6dscCtdEt2Pu28zs/uA7UArcL27t4UZaz992cwWE+162A18KNxw+s7dW83sBmATf1u6YVvIYQ3UJOAhM4Po352fuvuvww2pb8zsZ8B5wEQz2wPcCqwC7jOza4gujXFleBH2XYJ7Oq+/v0N69YqIiKSEurxERCQllFBERCQllFBERCQllFBERCQllFBERCQllFBE0sDMnkzDNWeZ2T+m+roiqaKEIpIG7v7mNFx2FqCEIhlLCUUkDczsWPD9PDN7wszuN7PnzewnFszuC9YH+XKwRshTZnZKUH6PmV3R9VpEJ9G9JVij4uODfU8ivVFCEUm/JcDHiK5tMgc4O+ZYrbsvAr4O3NXLdVYAvw/WqLgzLZGKDIASikj6PeXue9y9HdhCtOuqw89ivr9psAMTSSUlFJH0a4rZbqPzO/Q8znYrwe+mmQ0DCtIanUiKKKGIhOu9Md//GGzvBs4Iti8F8oPtOmD0oEUm0kd627BIuMaZ2VairZj3BWXfBdab2bPAr4HjQflWoC0ov0fjKJJp9LZhkZCY2W6g3N1fCzsWkVRQl5eIiKSEWigiIpISaqGIiEhKKKGIiEhKKKGIiEhKKKGIiEhKKKGIiEhK/H+ib48cAyxT+AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cx.plot_f(softsign, frange=(-15, 15, 1), xlabel=\"input\", ylabel=\"output\", title=\"softsign()\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## relu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function relu in module conx.activations:\n",
      "\n",
      "relu(x, alpha=0.0, max_value=None)\n",
      "    Rectified Linear Unit activation function.\n",
      "    \n",
      "    >>> relu(1)\n",
      "    1.0\n",
      "    >>> relu(-1)\n",
      "    0.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(relu)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "relu(0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VOX59/HPlRB2JCiRVQwisikIpmrFWtyKVtRWrRVsq2JLbUWtWnnQttrHvvpo1Vq3bmgttBUQN+TXWtGfPNb+qtUGwioiiCAJW1jCGkKW6/fHTHQIE5gkM3Nm+b5fr7xy5j5nzn0dx8nFfe5zrmPujoiIZK+coAMQEZFgKRGIiGQ5JQIRkSynRCAikuWUCEREspwSgYhIllMiEGkCM/upmf2lCdsPNrNiM7Pw6/fMbEjiIhRpOiUCkcT6GfCQf3bDzkPAvQHGI3IQJQKRCGbWKo776gGcDcyOaJ4DnG1m3ePVj0hLKRFI1jOzNWb2f8xsMbDHzPqY2QtmVm5mH5vZzY28b5SZlUbZ13nhl+cDC9x9X/368PJ8YHSCDkekyZQIRELGAhcBRwIvAYuAXsC5wA/MrDl/uE8CVkRpXw4Ma2acInGnRCAS8pi7rwNOBArc/V533+/uq4Engauasc98YFeU9l3hdSIpIW7nQ0XS3Lrw72OBnmZWEbEuF/hnM/a5HegUpb0TUBGlXSQQSgQiIfVX9awDPnb3/jG8Zw/Qvv6FmeUCBRHrFwPXRHnfICDmS1BFEk2nhkQO9B6wKzx53M7Mcs3sRDP7XJRtPwTamtlFZpYH/BhoE7H+dWCEmbWtbwgvnxJeJ5ISlAhEIrh7LTAGOBn4GNgCPAV0jrLtDuD74fVlhEYIpRHrNwHzgEsj3nYx8Ka7r0/QIYg0menBNCKJY2aDgWnAqe7uZvYucL27Lw04NJFPKRGIiGQ5nRoSEclySgQiIllOiUBEJMulxX0EXbt29cLCwqDDEBFJK/Pnz9/i7gWH2y4tEkFhYSHFxcVBhyEiklbMbG0s2+nUkIhIllMiEBHJckoEIiJZTolARCTLKRGIiGS5hCUCM3vazDab2UE1VczsdjNzM+uaqP5FRNLZ7JIyRt4/j76T/8bI++cxu6QsYX0lckQwFbigYaOZHQN8CfgkgX2LiKSt2SVl3PniEsoqKnGgrKKSO19ckrBkkLBE4O5vAduirPoVMInPHgQiIiIRHpy7gsrq2gPaKqtreXButEdgt1xS5wjM7FKgzN0XxbDtBDMrNrPi8vLyJEQnIpIa1ldUNqm9pZKWCMysPXAXcHcs27v7FHcvcveigoLD3iEtIpIx8tvnRW3vmd8uIf0lc0TQD+gLLDKzNUBvYIGZdU9iDCIiKW3Nlj3srarB7MD2dnm53DF6QEL6TFoicPcl7n60uxe6eyGhR/qNcPeNyYpBRCSVVdXUMnHGAtq2bsXdYwbTK78dBvTKb8d9l53EV4b3Ski/CSs6Z2YzgFFAVzMrBe5x9z8kqj8RkXR33ysfsLRsJ09+q4jzB3fjupF9k9JvwhKBu489zPrCRPUtIpJuXl26kalvr2H8yL6cP7hbUvvWncUiIgFbt20vk55fxNDenZl84cCk969EICISoP01ddw0owR3eGLsCFq3Sv6f5bR4MI2ISKZ66LUVLFxXwW+uHkGfo9oHEoNGBCIiAZn3wSamvLWab55+LF8+qUdgcSgRiIgEYMOOSm6ftYhBPY7gRxcNCjQWnRoSEUmS2SVlPDh3BesrKsnLzQGcX48bTtu83EDj0ohARCQJGlYU3V9bhwOLS3cEHZoSgYhIMkSrKFpd6wmrKNoUSgQiIkmQ7IqiTaFEICKSBD3y20ZtT1RF0aZQIhARSYKhvfIPaktkRdGmUCIQEUmwdz7aymvvb6To2Hx65bdNSkXRptDloyIiCbR1dxW3zCyh8KgOTBt/Gh3apN6f3dSLSEQkQ9TVObfOWkRFZTVTrzs1JZMA6NSQiEjC/P6t1bz1YTl3jxnM4J5HBB1Oo5QIREQSYP7abTz02gouOqkHV5/WJ+hwDkmJQEQkzir27uem6SWhCeHLT8IaPoA4xaTmCSsRkTTl7vzwucWU767ihe+dwRFt84IO6bA0IhARiaOn/7WG/16+iTsvHMTQ3gffO5CKEvnw+qeBMcBmdz8x3PYgcDGwH/gIuM7dKxIVg4hIMkRWFXVgSM8juG5kYdBhxSyRI4KpwAUN2l4HTnT3ocCHwJ0J7F9EJOEaVhUF+Kh8Ny8vXB9oXE2RsETg7m8B2xq0vebuNeGX/wZ6J6p/EZFkiFZVdF91XUpUFY1VkHME44G/N7bSzCaYWbGZFZeXlycxLBGR2KVyVdFYBZIIzOxHQA3wTGPbuPsUdy9y96KCgoLkBSci0gRdO7WJ2p4KVUVjlfTLR83sWkKTyOe6ux9mcxGRlLW7qgai/BlLlaqisUrqiMDMLgAmAZe4+95k9i0iEk/uzo9fWsLWPfuZeHY/euW3S7mqorFK5OWjM4BRQFczKwXuIXSVUBvg9fCddv929xsSFYOISKI8V1zK7IXrue38E7j53P78cPTAoENqtoQlAncfG6X5D4nqT0QkWT7ctIu75yzljH5HcePZxwcdTovpzmIRkSao3F/Ljc8soGObVjxy1cnk5qR2HaFYqNaQiEgT3DNnKavKd/Pn8adxdKfozyFONxoRiIjE6KWSUmYVl3LjqOM5s3/XoMOJGyUCEZEYrC7fzY9eWsrnCrvwg/P6Bx1OXCkRiIgcxr7qWm6cXkKbVjk8NnY4rXIz60+n5ghERA7j539bzvINO3n62iJ6dE6fO4ZjpUQgIhJFfWnpsnDNoLMHFHDOwG4BR5UYmTW+ERGJg8jS0vXeWb2V2SVlAUaVOEoEIiINZEJp6aZQIhARaSATSks3hRKBiEgDXTq0jtqeTqWlm0KJQEQkQllFJZX7a7AGlSPSrbR0UygRiIiEVdfWcfOMEnJzcrjrwkFpXVq6KXT5qIhI2C9f+5D5a7fz+NjhXDysJ98567igQ0oKjQhERIA3V2zmd//4iHGn9eHiYT2DDieplAhEJOtt2rmP22YtYmD3Ttw9ZnDQ4SSdEoGIZLWa8LzAvupanhg3grZ5uUGHlHSaIxCRrPbYvFW8+/E2Hr5yGMcf3THocAKhEYGIZK23V23h8XkrueKU3lw2onfQ4QRGiUBEslL5ripueXYh/Qo6cu+lQ4IOJ1AJOzVkZk8DY4DN7n5iuO1I4FmgEFgDXOnu2xMVg4hIQ7NLynhg7gesr9gHwPiRhbRvnd1nyRM5IpgKXNCgbTLwhrv3B94IvxYRSYr6qqL1SQDgsTdWZWxV0VglLBG4+1vAtgbNlwLTwsvTgK8kqn8RkYaiVRWtrK7N2KqisUr2HEE3d98QXt4INPqUBzObYGbFZlZcXl6enOhEJKNlW1XRWAU2WezuDvgh1k9x9yJ3LyooKEhiZCKSierqnNatov/Jy9SqorFKdiLYZGY9AMK/Nye5fxHJUk/9z2qqaurIyz2wrGgmVxWNVbITwRzgmvDyNcDLSe5fRLLQgk+288CrK7hgSHceuHxo1lQVjVUiLx+dAYwCuppZKXAPcD8wy8yuB9YCVyaqfxERgB17q7lpegndO7flF1cMpXO7PL6axTePRZOwRODuYxtZdW6i+hQRieTuTHphEZt27uP5751B53Z5QYeUknRnsYhkrGlvr2Husk1MvnAgJx+TH3Q4KUuJQEQy0pLSHfy/Vz7g3IFHc/2ZfYMOJ6UpEYhIxtm1r5qJMxZwVMfWPPS1YVjDBxDLAbK7wIaIZBx3584Xl1C6vZKZE06nS4fWQYeU8jQiEJGMMuO9dfx18QZuO/8EPld4ZNDhpAWNCEQk7c0uKePBuStYX1GJAwO6deR7X+wXdFhpQyMCEUlr9RVFy8JJAGDttr3MWbQ+0LjSiRKBiKS1aBVF91XXZX1F0aZQIhCRtKaKoi2nRCAiaa2gU5uo7dleUbQplAhEJG1V7q8lJ8otAqoo2jRKBCKStu796zI27qziu188ThVFW0CXj4pIWnp5YRkz3lvH90f1Y9IFA7nzwkFBh5S2NCIQkbTz8ZY93PXiEoqO7cJt558QdDhpT4lARNLKvupabnxmAXmtcnhs7HBa5erPWEvp1JCIpJX7XlnO+xt28tS3inRlUJwolYpI2vj7kg1Me2ct3z6zL+cN7hZ0OBlDiUBE0sK6bXuZ9MJihh2Tz6QLBgYdTkZRIhCRlLe/po6JM0oAeGLscFq30p+ueApkjsDMbgW+DTiwBLjO3fcFEYuIpK76qqJl4XIR1408lmOObB9wVJkn6WnVzHoBNwNF7n4ikAtclew4RCS1RVYVrTfzvVJml5QFGFVmCmp81QpoZ2atgPaA6sWKyAGiVRWtrK5VVdEEiCkRmNktsbTFwt3LgIeAT4ANwA53fy3K/ieYWbGZFZeXlzenKxFJY6oqmjyxjgiuidJ2bXM6NLMuwKVAX6An0MHMvtFwO3ef4u5F7l5UUFDQnK5EJI11aBN9ClP3DsTfISeLzWwsMA7oa2ZzIlZ1ArY1s8/zgI/dvTzcx4vAGcBfmrk/Eckw//iwnN1VNeTmGLV1/mm7qoomxuGuGnqb0OmbrsAvI9p3AYub2ecnwOlm1h6oBM4Fipu5LxHJMJt27uO2ZxcyoFsnrj+zL4++sZL1FZX0zG/HHaMHqKpoAhwyEbj7WmAt8Pl4deju75rZ88ACoAYoAabEa/8ikr5q65xbZpawd38tT4wbTv9unbjyc8cEHVbGi+k+AjPbBZ8+F7o1kAfscfcjmtOpu98D3NOc94pI5np83kr+vXobD14xlP7dOgUdTtaIKRG4+6efiJkZocne0xMVlIhkn7c/2sKjb6zksuG9uOKU3kGHk1WafB+Bh8wGRicgHhHJQlt2V/GDmQvp27UDP/vKiYT+vSnJEuupocsiXuYARYBKQohIi9XVObc+u5CKymqmjT+10ctGJXFi/S9+ccRyDbCG0OkhEZEW+d1bH/HPlVv4+VdPZFCPZk07SgvFOkdwXaIDEZHs85812/jlax9y0dAejDu1T9DhZK1YTw0dBzxKaILYgXeAW919dQJjE5EMVF9RdH1FJWbQpX0e9192kuYFAhTrZPF0YBbQg1BZiOeAGYkKSkQyU2RFUQfqHHZX1fLG8s1Bh5bVYk0E7d39z+5eE/75C9A2kYGJSOaJVlG0qqZOFUUDFutk8d/NbDIwk9Cpoa8Dr5jZkQDu3ty6QyKSRVRRNDXFmgiuDP/+boP2qwglhuPiFpGIZKzunduyYcfBV56romiwYk0Egxo+StLM2urxkiISK3enoFPrgxKBKooGL9Y5grdjbBMRieov/17L4tKdXDK0B73y22FAr/x23HfZSaooGrDDPY+gO9CL0GMlhwP113cdQegRkyIih7Vs/Q5+9tfljBpQwCNXDScnR5eKppLDnRoaTehJZL2BhyPadwF3JSgmEckgu6tqmDi9hC4d8nj4ypOVBFLQ4Z5HMA2YZmaXu/sLSYpJRDKEu3PXi0tYu3UPM75zOkd2aB10SBJFrJPFJ5rZkIaN7n5vnOMRkQwyq3gdcxat5/bzT+C0444KOhxpRKyJYHfEcltgDLA8/uGISKZYsXEX98xZxsjjj+L7Zx8fdDhyCLEWnYt8XjFm9hAwNyERiUja27u/honTF9CxTR6/+vrJ5GpeIKU1t/B3e0ITyCIiB7nn5WWsKt/Nn8efxtGdVI0m1cVafXQJnz2zOAc4GvhZczs1s3zgKeDE8H7Hu/s7zd2fiASvvqpoWbhcxJcGH82Z/bsGHJXEItYRwRigC/AFIB94xd3nt6DfR4FX3f0KM2uN7kkQSWv1VUUjC8q9tXILs0vKdLNYGoj1zuJLgT8DXYE84I9mdlNzOjSzzsBZwB8A3H2/u1c0Z18ikhqiVRXdV62qouki1hHBt4HT3X0PgJn9gtDDaR5vRp99gXJCyWQYMB+4pX7f9cxsAjABoE8fPblIJJWpqmh6i3VEYEBkuq/ls3ITTdUKGAH81t2HA3uAyQ03cvcp7l7k7kUFBQXN7EpEkiG/fV7UdlUVTQ+xjgj+CLxrZi+FX3+F8KmdZigFSt393fDr54mSCEQkPazduoe9VTWYgftn7aoqmj5iGhG4+8PAdcC28M917v5Iczp0943AOjOr/z/kXOD95uxLRIJVVVPLxOkltMnL5ScXDVJV0TQV830E7r4AWBCnfm8CnglfMbSaUJIRkTRz/98/YEnZDn7/zVMYPaQ748/UM6rSUXNvKGsRd18IFAXRt4jEx9xlG/njv9Zw3chCRg/pHnQ40gKxThaLiHyqdPte7nhuESf16szkCwcGHY60kBKBiDRJdW0dN80owR2eGDecNq1ygw5JWiiQU0Mikr4eem0FJZ9U8MS44Rx7VIegw5E40IhARGL2/z/YzO//sZqrT+vDmKE9gw5H4kSJQERismFHJbfNWsjA7p34yZjBQYcjcaRTQyLSqPqKousrKsnLzcFxnr96BG3zNC+QSTQiEJGo6iuKllVU4sD+2joAlpTuCDYwiTslAhGJKlpF0epaV0XRDKREICJRqaJo9lAiEJGoeuRHf8SkKopmHiUCEYnq5N75B7WpomhmUiIQkYO8u3orry7byCl98umV31YVRTOcLh8VkQNs3V3FzTNLOPaoDky7/jQ6ttGfiUynEYGIfKquzrn9uUVs31vNE+OGKwlkCSUCEfnUk/9czZsryvnJmMEM6dk56HAkSZQIRASA+Wu38+DcFXz5pO5847Q+QYcjSaREICJU7N3PzTNK6JHflvsuG4qZBR2SJJFOAIpkOXfnjucXs3nXPp6/4Qw6t8sLOiRJMo0IRLLc1LfX8Pr7m5h84SCGHXPwvQOS+QIbEZhZLlAMlLn7mKDiEMlGkVVFHRjS8wjGjywMOiwJSJAjgluA5QH2L5KVGlYVBfiofDcvL1wfaFwSnEASgZn1Bi4Cngqif5FsFq2q6L7qOlUVzWJBjQgeASYBdY1tYGYTzKzYzIrLy8uTF5lIhlNVUWko6YnAzMYAm919/qG2c/cp7l7k7kUFBQVJik4k8xV0ahO1XVVFs1cQI4KRwCVmtgaYCZxjZn8JIA6RrLOnqgZ3P6hdVUWzW9ITgbvf6e693b0QuAqY5+7fSHYcItnG3fnx7KVs3bOfiWf3o1d+O1UVFUA3lIlkjefml/JSSRm3nncCt5zXnx+OHhh0SJIiAk0E7v4m8GaQMYhkg5WbdnH3y0s5o99RTDzn+KDDkRSjO4tFMlzl/lpunL6Ajm1a8cjXTyY3R3WE5EA6NSSS4X46ZxkrN+/mT+NP5egjoj+HWLKbRgQiGWx2SRnPFq/j+6P68YX+ugxbolMiEMlQq8t386OXlvC5wi7cet4JQYcjKUyJQCQD7auu5cbpJbRulcNjY4fTKldfdWmc5ghEMtDP/7ac5Rt28vS1RfTorDuG5dCUCEQyRH1p6bJwzaCzBxRwzsBuAUcl6UDjRZEMEFlaut47q7cyu6QswKgkXSgRiGQAlZaWllAiEMkAKi0tLaFEIJIBunRoHbVdpaUlFkoEImmurKKSyv01WIPKESotLbFSIhBJY9W1ddw8o4TcnBzuunCQSktLs+jyUZE09vDrHzJ/7XYeGzucS4b15DtnHRd0SJKGNCIQSVNvrtjMb9/8iLGn9uGSYT2DDkfSmBKBSBratHMft81axIBunbjn4sFBhyNpTolAJM3U1jk3zyihcn8tv756OG3zcoMOSdKc5ghE0syjb6zk3Y+38dDXhnH80Z2CDkcygEYEImnk7VVbeHzeSi4f0ZsrTukddDiSIZQIRNJE+a4qbnl2Icd17cC9lw4JOhzJIEk/NWRmxwB/AroBDkxx90eTHYdIuphdUsYDcz9gfcU+AK49o5AObXRWV+IniBFBDXC7uw8GTgduNDNd9iASRX1V0fokAPDEvFWqKipxlfRE4O4b3H1BeHkXsBzQ7Y8iUUSrKlpZXauqohJXgc4RmFkhMBx4N8q6CWZWbGbF5eXlyQ5NJCWoqqgkQ2CJwMw6Ai8AP3D3nQ3Xu/sUdy9y96KCgoLkBygSMHendavoX1FVFZV4CiQRmFkeoSTwjLu/GEQMIqnuqX9+TFVNHXm5B5YVVVVRibekJwIzM+APwHJ3fzjZ/Yukg5JPtvOLVz9g9JBuPHD5UFUVlYQK4hq0kcA3gSVmtjDcdpe7vxJALCIpZ8feaiZOL6HbEW154PJhdG6fx1dH6OYxSZykJwJ3/x/ADruhSBZydya9sIhNO/cx64bP07l9XtAhSRbQncUiKeRP76xl7rJNTLpgACP6dAk6HMkSSgQiKWJp2Q5+/rflnDPwaL59ph4wI8mjRCCSAnbtq2bi9AUc2aE1D31tGDk5OnsqyaOCJSIBc3fuemkp67ZXMuM7p3Nkh9ZBhyRZRiMCkYDN/M86/mvRem47/wRO7Xtk0OFIFtKIQCQAs0vKeHDuCtZXVOLAgG4d+d4X+wUdlmQpjQhEkqy+omhZOAkArN26lzmL1gcal2QvJQKRJItWUXRfTZ0qikpglAhEkkwVRSXVKBGIJFlBpzZR21VRVIKiRCCSRPuqa8mNcouAKopKkJQIRJLo//7X+2zYWcWEs/qqoqikDF0+KpIkcxatZ8Z7n3DDF/sx+cKB3PVlPapbUoNGBCJJsGbLHu56cQmnHNuF2790QtDhiBxAiUAkwapqapk4YwG5OcZjY4eTl6uvnaQWnRoSSbD7XvmApWU7efJbRfTSlUGSgvRPE5EEenXpRqa+vYbxI/ty/uBuQYcjEpUSgUiCrNu2l0nPL2Jo785MvnBg0OGINEqJQCQB9tfUMXFGCe7wxNgRtG6lr5qkrkDmCMzsAuBRIBd4yt3vj3cfkdUde+a3447RAxq9TjvWbbVP7TPWfZaFy0Vce8ax9DmqfdRtRVKFufvht4pnh2a5wIfA+UAp8B9grLu/39h7ioqKvLi4OOY+6qs7Rhb2apeXG/WmnVi31T61z3jtUyRZzGy+uxcddrsAEsHngZ+6++jw6zsB3P2+xt7T1EQw8v55n/6LLFKrHKNv1w4HtH28ZQ81dQf/N2i4bazbaZ/aZ0O98tvxr8nnHNQukmixJoIgTg31AtZFvC4FTmu4kZlNACYA9OnTp0kdNFbFsabO6d+t4wFtKzfvjmnbWLfTPrXPhlRVVFJdyt5H4O5TgCkQGhE05b0989tFHRH0ym/Hb64+5YC2xkYPDbeNdTvtU/tsSFVFJdUFcSlDGXBMxOve4ba4uWP0ANrl5R7Q1lh1x1i31T61z3jtUyTVBDEi+A/Q38z6EkoAVwHj4tlB/cRcLFd6xLqt9ql9xmufIqkm6ZPFAGb2ZeARQpePPu3uPz/U9k2dLBYRkdSeLMbdXwFeCaJvERE5kG53FBHJckoEIiJZTolARCTLKRGIiGS5QK4aaiozKwfWNvPtXYEtcQwnFWTaMWXa8UDmHVOmHQ9k3jFFO55j3b3gcG9Mi0TQEmZWHMvlU+kk044p044HMu+YMu14IPOOqSXHo1NDIiJZTolARCTLZUMimBJ0AAmQaceUaccDmXdMmXY8kHnH1Ozjyfg5AhERObRsGBGIiMghKBGIiGS5jE0EZvY1M1tmZnVmVhTRXmhmlWa2MPzzuyDjjFVjxxNed6eZrTKzFWY2OqgYW8LMfmpmZRGfy5eDjqk5zOyC8OewyswmBx1PPJjZGjNbEv5c0q4MsJk9bWabzWxpRNuRZva6ma0M/+4SZIxN1cgxNfs7lLGJAFgKXAa8FWXdR+5+cvjnhiTH1VxRj8fMBhN6psMQ4ALgN2aWe/Db08KvIj6XtKtOG/7v/mvgQmAwMDb8+WSCs8OfSzpedz+V0Hcj0mTgDXfvD7wRfp1OpnLwMUEzv0MZmwjcfbm7rwg6jng5xPFcCsx09yp3/xhYBZya3Ogk7FRglbuvdvf9wExCn48EyN3fArY1aL4UmBZengZ8JalBtVAjx9RsGZsIDqOvmZWY2T/M7AtBB9NCvYB1Ea9Lw23paKKZLQ4Pe9NqqB6WSZ9FJAdeM7P5ZjYh6GDipJu7bwgvbwS6BRlMHDXrO5TWicDM/tvMlkb5OdS/wjYAfdx9OHAbMN3MjkhOxIfWzONJG4c5vt8C/YCTCX1Gvww0WIl0pruPIHTK60YzOyvogOLJQ9fQZ8J19M3+DgXyhLJ4cffzmvGeKqAqvDzfzD4CTgACnwRrzvEQeu7zMRGve4fbUk6sx2dmTwJ/TXA4iZA2n0VTuHtZ+PdmM3uJ0CmwaHNv6WSTmfVw9w1m1gPYHHRALeXum+qXm/odSusRQXOYWUH9ZKqZHQf0B1YHG1WLzAGuMrM2ZtaX0PG8F3BMTRb+Mtb7KqHJ8XTzH6C/mfU1s9aEJvHnBBxTi5hZBzPrVL8MfIn0/GwamgNcE16+Bng5wFjioiXfobQeERyKmX0VeBwoAP5mZgvdfTRwFnCvmVUDdcAN7h63SZdEaex43H2Zmc0C3gdqgBvdvTbIWJvpATM7mdAQfQ3w3WDDaTp3rzGzicBcIBd42t2XBRxWS3UDXjIzCP29mO7urwYbUtOY2QxgFNDVzEqBe4D7gVlmdj2hEvdXBhdh0zVyTKOa+x1SiQkRkSyXdaeGRETkQEoEIiJZTolARCTLKRGIiGQ5JQIRkSynRCASwczeTsA+C81sXLz3KxIvSgQiEdz9jATsthBQIpCUpUQgEsHMdod/jzKzN83seTP7wMyesfBdVeH6/A+Ea/S/Z2bHh9unmtkVDfdF6OalL4RrxN+a7GMSORwlApHGDQd+QOjZAscBIyPW7XD3k4AngEcOs5/JwD/DNeJ/lZBIRVpAiUCkce+5e6m71wELCZ3iqTcj4vfnkx2YSDwpEYg0ripiuZYDa3N5lOUawt8pM8sBWic0OpE4USIQaZ6vR/x+J7y8BjglvHwJkBde3gV0SlpkIk2UsdVHRRKsi5ktJjRqGBtuexJ42cwWAa8Ce8Lti4HacPtUzRNIqlH1UZEmMrM1QJG7bwk6FpF40KkhEZEspxGBiEiW04hARCTLKRGIiGQ5JQIRkSwqLge3AAAAEklEQVSnRCAikuWUCEREstz/AjhXB41WJA1JAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cx.plot_f(relu, frange=(-15, 15, 1), xlabel=\"input\", ylabel=\"output\", title=\"relu()\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## tanh"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function tanh in module conx.activations:\n",
      "\n",
      "tanh(x)\n",
      "    Tanh activation function.\n",
      "    \n",
      "    >>> tanh(0)\n",
      "    0.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(tanh)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.46211716532707214"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tanh(0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEWCAYAAABBvWFzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X2cXGV99/HPN5tNsoGEJCSEJJBskPAgokldUYtalECotSRai2Bto9UXtpU+aKWG2lu9qdQorejdYgtVHuoD+ASYKjYiD7VVETYGiKAxIe6GLIFsJg8mu5tNdvd3/zFnw2Qzm53dnZkzM/t9v17zmnOuc50zv8My88t1znWuSxGBmZnZaI1LOwAzM6sNTihmZlYUTihmZlYUTihmZlYUTihmZlYUTihmZlYUTihmFURSSDr9GNtnSfqFpIZk/ZuSfrt8EZoNzgnFbJQktUhaWqaPWwXcFhFdyfongY+X6bPNjskJxaxKSJoIrAS+1F8WEY8AUyU1pRaYWcIJxWwUJH0RmA/8p6T9kv5G0tclPSdpr6QfSDonp/5tkm6U9B1J+yT9RNKLBhx2qaRNkvYkdZWUvxLYExHbBtR/CPidUp2jWaGcUMxGISL+ENgK/G5EHB8RnwK+CywCTgJ+Cnx5wG6XA/8XmA5sBq4bsP1NwCuAlwKXAcuS8nOBjXnC+DnwslGfjNkoOaGYFVlE3BIR+yKiG/gY8DJJJ+RUuTsiHomIHrLJZvGAQ6yOiD0RsRV4MGf7NGBfno/cl2wzS5UTilkRSaqTtFrS05J+DbQkm2bmVHsuZ7kTOH7AYQbbvhuYkudjpwB7Rhy0WZE4oZiNXu6Q3W8HlgNLgROAxqRcjN4TwBl5ys8GHi/C8c1GxQnFbPSeB05LlqcA3UAGmAz8QxE/5xFgmqR5A8p/i+x9G7NUOaGYjd4ngL+TtAeYAbQCbcBTwMPF+pCIOAjcBryjv0zSK4D9Sfdhs1TJE2yZVQ9Js4D/AZZERJekbwJfiIh7Uw7NzAnFzMyKw5e8zMysKJxQzMysKJxQzMysKManHUA5zZw5MxobG9MOw8ysqqxbt25nRMwaqt6YSiiNjY00NzenHYaZWVWR1FpIPV/yMjOzonBCMTOzonBCMTOzonBCMTOzonBCMTOzoki1l5ekW8jOTrcjIl6SZ7uAzwJvJDsvxDsj4qfJtpXA3yVVPx4Rt5cnajMDuGd9G9ev3cize7qYO62Bq5edyYolAwdCHl5dHzO9YxZDqmN5SXodsB/4j0ESyhuBPyebUF4JfDYiXilpBtAMNJGdi2Id8PKI2H2sz2tqagp3GzY7tkJ+hO5Z38Y1d22g61Dv4bKG+jo+8ZZzR1zXx0zvmEORtC4imoasl/bgkJIagW8PklBuAh6KiDuS9Y3ABf2viHhvvnqDcUIxO7Z8P0ITx4/j/UsX8Zunz6TrYC+dh3r5668+xq7OQ0ftP3XSeP70gtPpS35XIoKbfrCFfQd6jqo7ZdJ4/vj8hYdnJ7v1h7/KX2/ieN51fuMRZbf+sIV93UPXLbTeWDrmvGkN/HDVG44qP5ZaSSjfJju/9v8m6/cDHyKbUCZFxMeT8v8DdEXEP+Y5xpXAlQDz589/eWtrQc/nmI05e7sOccH1D7I7T6JImwbMd3msn63cuoXWG0vHFPCr1b8z+EHyHrewhFLzT8pHxM3AzZBtoaQcjlkq8l3GWr54Lpt27OeBX+zgwV/soLl1N719g39FvrCyiYb6OiZNqONPvriOHfu6j6oz54RJPPjBC4Dsj9s4id+6/kGe3XPgqLoD/6V8/uoHaNvTNWS94dT1MY+uN3daw1FlxVLpvbzagFNz1k9JygYrN7MB+i9jte3pIoC2PV389dcfZ8m193HxDT9g9Xd/wd6uQ7z3dacx8/gJeY8xb1oDF549m988fSa/MX86f/vGs2morzuiTkN9HR+65Cwm1dcxqb6OiePrqK8bx98sOytv3auXnXlE2dXLziyo3nDq+piFHbNYKr2Fsga4StKdZG/K742I7ZLWAv8gaXpS72LgmrSCNKtk16/deMQ9EYDevuDAoV7+4c3n8vqzZjHnhOy/Ws+YPSXvjdyBP0L9N3UL6UFUaF0fM71jFkvavbzuIHs/ZCbwPPBRoB4gIv4t6Tb8L8AlZLsNvysimpN9/xj42+RQ10XErUN9nm/K21i0cNV3yPctH+xaerm7mlrlq4p7KBFxxRDbA3jfINtuAW4pRVxmtWTutIZhXUtfsWSeE4iNSKXfQzGzUfrgxWcc1VOo1NfSbWxyQjGrcRPr64iA6ZPrEdkb7CN5uM1sKJV+U97MRqGnt49/+t5GTj/peNb+1euoG6ehdzIbIbdQzGrY3evbeLq9gw9efIaTiZWcE4pZjeru6eUz39/ES085gWXnnJx2ODYGOKGY1ag7frKVtj1dXL3sTDTwrrxZCTihmNWgzoM9/MuDm3nVaTN4zekz0w7HxggnFLMadOsPW9i5/yBXLzvLrRMrGycUsxqzt/MQN/330yw9+yRevmD60DuYFYkTilmNuekHT7Ovu4e/vtgPLlp5OaGY1ZAd+w5w6w9buPRlczl7ztS0w7ExxgnFrIbc+MBmDvb28f6lZ6Qdio1BTihmNeKZXZ185ZGtXNZ0Ko0zj0s7HBuDnFDMasRn79+EJP7ywkVph2JjlBOKWQ3Y9Pw+7vrpNla+egEnnzAp7XBsjPLgkGZVrH8yrLY9XQhYcKIvdVl6Um2hSLpE0kZJmyWtyrP9BkmPJa9fStqTs603Z9ua8kZulr7cueIBArjuOz/nnvVt6QZmY1ZqLRRJdcCNwEXANuBRSWsi4qn+OhHx/pz6fw4syTlEV0QsLle8ZpUm31zxXYd6uX7tRs91YqlIs4VyHrA5IrZExEHgTmD5MepfAdxRlsjMqsCzeab1PVa5WamlmVDmAc/krG9Lyo4iaQGwEHggp3iSpGZJD0taMdiHSLoyqdfc3t5ejLjNKsJgc8IPVm5WatXSy+ty4BsRkdu+XxARTcDbgc9IelG+HSPi5ohoioimWbNmlSNWs7K4etmZTBx/5FfYc8VbmtJMKG3AqTnrpyRl+VzOgMtdEdGWvG8BHuLI+ytmNW/Fknm841ULADxXvFWENLsNPwoskrSQbCK5nGxr4wiSzgKmAz/OKZsOdEZEt6SZwPnAp8oStVkFmZM8c/LYRy7mhMn1KUdjY11qCSUieiRdBawF6oBbIuJJSdcCzRHR3xX4cuDOiIic3c8GbpLUR7aVtTq3d5jZWNGS6eCEhnonE6sIqT7YGBH3AvcOKPvIgPWP5dnvR8C5JQ3OrAq0ZjppPHFy2mGYAdVzU97M8mjNdPrpeKsYTihmVepQbx9te7pY4BaKVQgnFLMq1ba7i96+cAvFKoYTilmVasl0APgeilUMJxSzKtWa6QRgvhOKVQgnFLMq1ZrpZPKEOmYdPzHtUMwAJxSzqtWa6WDBicchKe1QzAAnFLOq1ZLpYMEMX+6yyuGEYlaFevuCZ3Z1sWCmE4pVDicUsyr03K8PcLC3j0Z3GbYK4oRiVoVad2a7DPuhRqskTihmVagl6TLshxqtkjihmFWh1l0dTBg/jjlTJ6UditlhTihmVah1ZyfzZ0xm3Dh3GbbK4YRiVoXcZdgqkROKWZWJCLbu8rD1VnlSTSiSLpG0UdJmSavybH+npHZJjyWv9+RsWylpU/JaWd7IzdLTvr+bzoO9NPoZFKswqc3YKKkOuBG4CNgGPCppTZ6pfL8aEVcN2HcG8FGgCQhgXbLv7jKEbpaqw4NC+pKXVZg0WyjnAZsjYktEHATuBJYXuO8y4L6I2JUkkfuAS0oUp1lFadnZP2y9L3lZZUkzocwDnslZ35aUDfR7kp6Q9A1Jpw5zXyRdKalZUnN7e3sx4jZL1dZdndSNE/OmN6QditkRKv2m/H8CjRHxUrKtkNuHe4CIuDkimiKiadasWUUP0KzcWjKdzJvWQH1dpX99baxJ8//INuDUnPVTkrLDIiITEd3J6ueBlxe6r1mtyg5b7/snVnnSTCiPAoskLZQ0AbgcWJNbQdKcnNVLgZ8ny2uBiyVNlzQduDgpM6t5rZlO3z+xipRaL6+I6JF0FdlEUAfcEhFPSroWaI6INcBfSLoU6AF2Ae9M9t0l6e/JJiWAayNiV9lPwqzM9nQeZG/XIbdQrCKlllAAIuJe4N4BZR/JWb4GuGaQfW8BbilpgGYVxoNCWiXzXT2zKtKa6e8y7BaKVR4nFLMq0prpRIJT/VCjVSAnFLMq0pLp4OSpk5hUX5d2KGZHcUIxqyJbM52+IW8VywnFrIq0uMuwVTAnFLMqsb+7h537u5nvFopVKCcUsyrxQg8vt1CsMjmhmFWJrYefQXELxSqTE4pZlfBDjVbpnFDMqkRrpoOZx0/g+ImpDnBhNignFLMq0ZrxPPJW2ZxQzKqEh623SueEYlYFDhzqZfuvD7BghlsoVrmcUMyqwLbdnURA40y3UKxyOaGYVYGWne7hZZUv1YQi6RJJGyVtlrQqz/YPSHpK0hOS7pe0IGdbr6THkteagfua1ZKW5KHGBR5l2CpYav0PJdUBNwIXAduARyWtiYincqqtB5oiolPSnwKfAt6WbOuKiMVlDdosJa2ZTqZOGs+0yfVph2I2qDRbKOcBmyNiS0QcBO4EludWiIgHI6IzWX0YOKXMMZpVhNZdnTTOPA5JaYdiNqg0E8o84Jmc9W1J2WDeDXw3Z32SpGZJD0taMdhOkq5M6jW3t7ePLmKzlLRmOpjvy11W4ariprykdwBNwPU5xQsiogl4O/AZSS/Kt29E3BwRTRHRNGvWrDJEa1Zch3r72La7y4NCWsVLM6G0AafmrJ+SlB1B0lLgw8ClEdHdXx4Rbcn7FuAhYEkpgzVLy7N7uujtCz/UaBUvzYTyKLBI0kJJE4DLgSN6a0laAtxENpnsyCmfLmlisjwTOB/IvZlvVjM8KKRVi9R6eUVEj6SrgLVAHXBLRDwp6VqgOSLWkL3EdTzw9eRm5NaIuBQ4G7hJUh/ZpLh6QO8ws5rxwjwobqFYZUt12NKIuBe4d0DZR3KWlw6y34+Ac0sbnVllaM100lBfx6wpE9MOxeyYquKmvNlY1j8opLsMW6VzQjGrcC2ZTt+Qt6rghGJWwfr6gq27Ot1l2KqCE4pZBXvu1wc42NPnHl5WFZxQzCrY4UEhfcnLqkBBCUXSXxZSZmbF1Xr4GRQnFKt8hbZQVuYpe2cR4zCzPFoznUyoG8ecExrSDsVsSMd8DkXSFWTHylo4YM6RKcCuUgZmZtkuw6fMaKBunLsMW+Ub6sHGHwHbgZnAP+WU7wOeKFVQZpbVknEPL6sex0woEdEKtAKvLk84ZtYvItia6eBVp81IOxSzghQ09IqkfUAkqxOAeqAjIqaWKjCzsW7n/oN0HOz1tL9WNQpKKBExpX9Z2fEflgOvKlVQZvbCoJALZvqSl1WHYT+HEln3AMtKEI+ZJfq7DPseilWLQi95vSVndRzZ2RMPlCQiMwOyLZRxgnnT3GXYqkOhw9f/bs5yD9BC9rKXmZVIS6aTedMbmDDeA1pYdSj0Hsq7Sh2ImR2pNdPhy11WVQodeuU0Sf8pqV3SDknfknTaaD9c0iWSNkraLGlVnu0TJX012f4TSY05265JyjdK8v0cqxn3rG/j/NUP8Pi2vfy0dTf3rG9LOySzghTalv4K8DVgDjAX+Dpwx2g+WFIdcCPw28CLgSskvXhAtXcDuyPidOAG4JPJvi8mOwf9OcAlwOeS45lVtXvWt3HNXRto29MFQMfBXq65a4OTilWFQhPK5Ij4YkT0JK8vAZNG+dnnAZsjYktEHATu5Oj7MsuB25PlbwAX5nRbvjMiuiPiV8Dm5HhmVe36tRvpOtR7RFnXoV6uX7sxpYjMCldoQvmupFWSGiUtkPQ3wL2SZkga6WO884Bncta3JWV560RED7AXOLHAfQGQdKWkZknN7e3tIwzVrDyeTVomhZabVZJCe3ldlry/d0D55WSfoB/1/ZRSiYibgZsBmpqaYojqZqmaO63h8OWugeVmla7QFsrZEbEw95VTNtJk0gacmrN+SlKWt46k8cAJQKbAfc2qztXLzqSh/sjbgQ31dVy97MyUIjIrXKEJ5UcFlg3Ho8AiSQslTSDb2lkzoM4aXpiL5a3AAxERSfnlSS+whcAi4JFRxmOWuhVL5vGJt5zLhLrsV3PetAY+8ZZzWbEk7xVds4oy1HwoJ5O9N9EgaQnQPynDVGBUI9ZFRI+kq4C1QB1wS0Q8KelaoDki1gBfAL4oaTPZ+VcuT/Z9UtLXgKfIPmj5vojozftBZlVmxZJ5XHfvz3n9mbP41FtflnY4ZgUb6h7KMrIzM54CfDqnfB/wt6P98Ii4F7h3QNlHcpYPAL8/yL7XAdeNNgazStPR3UP7vm4W+KFGqzJDzYdyO3C7pN+LiG+WKSazMW3rLg8KadWp0F5eL5F0zsDCiLi2yPGYjXmHh60/0fOgWHUpNKHsz1meBLwJ+HnxwzGzlmTYeicUqzaFDg6ZO588kv6R7M10Myuy1kwHJx43gSmT6tMOxWxYRjou9mSyN+rNrMhaM51unVhVKnSCrQ28MKf8OOAk4O9LFZTZWNaa6eS8hSMd0cgsPYXeQ3kTMB14LTANuDci1pUsKrMxqrunl2f3drmFYlWp0Etey4EvAjOBeuBWSX9esqjMxqhndnUR4S7DVp0KbaG8B3hVRHQASPok8GPgn0sVmNlY1N9leL5bKFaFCm2hCMgd2qSXF4ZhMbMi6e8y7BaKVaNCWyi3Aj+RdHeyvoLsOFtmVkRbMx1MmTSe6ZPdZdiqT6HPoXxa0kPAa5Kid0XE+pJFZTZGtSRdhrMTk5pVl0JbKETET4GfljAWszGvNdPBOfNOSDsMsxEZ6YONZlZkPb19bNvdRaNvyFuVckIxqxDP7jlAT1942HqrWk4oZhWipX+U4RluoVh1SiWhSJoh6T5Jm5L36XnqLJb0Y0lPSnpC0ttytt0m6VeSHktei8t7BmbF1/8MSuNMt1CsOqXVQlkF3B8Ri4D7k/WBOoE/iohzgEuAz0ialrP96ohYnLweK33IZqXVmulkUv04TpoyMe1QzEYkrYSyHLg9Wb6d7HMtR4iIX0bEpmT5WWAHMKtsEZqVWUumkwUzjnOXYataaSWU2RGxPVl+Dph9rMqSzgMmAE/nFF+XXAq7QdKg/6STdKWkZknN7e3tow7crFRaMx0eFNKqWskSiqTvS/pZntfy3HoREbwwNH6+48whOzDluyKiLym+BjgLeAUwA/jQYPtHxM0R0RQRTbNmuYFjlamvL9i6q9P3T6yqFfxg43BFxNLBtkl6XtKciNieJIwdg9SbCnwH+HBEPJxz7P7WTbekW4EPFjF0s7J7ft8Bunv6mO8eXlbF0rrktQZYmSyvBL41sIKkCcDdwH9ExDcGbJuTvIvs/ZeflTRasxJr2elBIa36pZVQVgMXSdoELE3WkdQk6fNJncuA1wHvzNM9+MvJLJIbyM7R8vHyhm9WXFt3Jc+g+B6KVbGSXfI6lojIABfmKW8mO/cKEfEl4EuD7P+GkgZoVmYtmU7q68TcaQ1ph2I2Yn5S3qwCtGY6OHX6ZOrGucuwVS8nFLMK0JoMW29WzZxQzFIWEUlC8Q15q25OKGYpy3QcZH93j1soVvWcUMxSdnhQSLdQrMo5oZilrDWTfQbFLRSrdk4oZilryXQyTnDKdCcUq25OKGYpa810MHdaAxPG++to1c3/B5ulrDXT6fsnVhOcUMxS5mHrrVY4oZilaG/XIXZ3HnJCsZrghGKWoq2He3j5kpdVPycUsxS1+BkUqyFOKGYp6n+o0RNrWS1wQjFLUWumk9lTJ9IwoS7tUMxGLZWEImmGpPskbUrepw9Srzdncq01OeULJf1E0mZJX01mdzSrOh4U0mpJWi2UVcD9EbEIuD9Zz6crIhYnr0tzyj8J3BARpwO7gXeXNlyz0mjJdLDAl7usRqSVUJYDtyfLt5OdF74gyTzybwD655kf1v5mlaLzYA879nXTONMtFKsNaSWU2RGxPVl+Dpg9SL1JkpolPSypP2mcCOyJiJ5kfRswb7APknRlcozm9vb2ogRvVgxbd3lQSKstJZtTXtL3gZPzbPpw7kpEhKQY5DALIqJN0mnAA5I2AHuHE0dE3AzcDNDU1DTY55iVXcvOJKHMcAvFakPJEkpELB1sm6TnJc2JiO2S5gA7BjlGW/K+RdJDwBLgm8A0SeOTVsopQFvRT8CsxA53GXYLxWpEWpe81gArk+WVwLcGVpA0XdLEZHkmcD7wVEQE8CDw1mPtb1bpWnd1MuO4CZzQUJ92KGZFkVZCWQ1cJGkTsDRZR1KTpM8ndc4GmiU9TjaBrI6Ip5JtHwI+IGkz2XsqXyhr9GZF4EEhrdaU7JLXsUREBrgwT3kz8J5k+UfAuYPsvwU4r5QxmpVay85OXtGY9xEss6rkJ+XNUtDd08v2vV1+qNFqihOKWQq27e6iL6Bxpi95We1wQjFLwQuDQrqFYrXDCcUsBa3JPCiNvilvNcQJxSwFrZlOpkwcz4zjPK6p1Q4nFLMUtGQ6mH/iZLJD05nVBicUsxS0Zjo9S6PVHCcUszLr6e1j2+5OP9RoNccJxazMtu89wKHecAvFao4TilmZtXhQSKtRTihmZfZCl2G3UKy2OKGYlVlrpoNJ9eM4acrEtEMxKyonFLMya8l0Mn/GZMaNc5dhqy1OKGZltjXT6UEhrSY5oZiVUV9f0Lqrw0OuWE1yQjErox37ujlwqI/5bqFYDUoloUiaIek+SZuS96NmGZL0ekmP5bwOSFqRbLtN0q9yti0u/1mYDV//KMNuoVgtSquFsgq4PyIWAfcn60eIiAcjYnFELAbeAHQC38upcnX/9oh4rCxRm43CPevbeO8X1wHwwa8/zj3r21KOyKy40kooy4Hbk+XbgRVD1H8r8N2I6CxpVGYlcs/6Nq65awN7ug4B8Pyvu7nmrg1OKlZT0koosyNie7L8HDB7iPqXA3cMKLtO0hOSbpA0aId+SVdKapbU3N7ePoqQzUbu+rUb6TrUe0RZ16Ferl+7MaWIzIqvZAlF0vcl/SzPa3luvYgIII5xnDnAucDanOJrgLOAVwAzgA8Ntn9E3BwRTRHRNGvWrNGcktmIPbuna1jlZtVofKkOHBFLB9sm6XlJcyJie5IwdhzjUJcBd0fEoZxj97duuiXdCnywKEGblcjcaQ205Ukec6c1pBCNWWmkdclrDbAyWV4JfOsYda9gwOWuJAmh7OxEK4CflSBGs6J53+tfdFRZQ30dVy87M4VozEojrYSyGrhI0iZgabKOpCZJn++vJKkROBX47wH7f1nSBmADMBP4eBliNhuxbbuzrZOTpkxEwLxpDXziLeeyYsm8dAMzK6KSXfI6lojIABfmKW8G3pOz3gIc9Y2LiDeUMj6zYtqx7wC3/rCFS182l/93xZK0wzErGT8pb1Zin3vwaQ729vGBi85IOxSzknJCMSuhbbs7+fJPWrms6VQaZ3q4FattTihmJfSZ729CEn9x4elph2JWck4oZiWyecc+7vrpNv7oVQuYc4K7B1vtc0IxK5FP3/dLGurr+LPXu3ViY4MTilkJbNi2l3s3PMd7XnsaM46bkHY4ZmXhhGJWAtd/byPTJ9fzntcuTDsUs7JxQjErsoe3ZPjBL9v5swtOZ8qk+rTDMSsbJxSzIooIrl+7kdlTJ/KHr16QdjhmZeWEYlZED27cwbrW3fzFhYuYVF+XdjhmZeWEYlYkfX3B9Wt/yYITJ3NZ06lph2NWdk4oZkXy7Q3b+fn2X/OBi86gvs5fLRt7Uhkc0qxW3LO+jevXbuTZPV2MGyfmTJ3I7750btphmaXC/4wyG6H+eeLb9nQRQG9fkOk4xJrHn007NLNUOKGYjVC+eeIP9vZ5nngbs3zJyyyP3EtZc6c1cPWyMw9PhnWwp49HW3blndIXPE+8jV2pJBRJvw98DDgbOC+ZWCtfvUuAzwJ1wOcjon9mx4XAncCJwDrgDyPiYCliPdYPy0jr+pjpHHM49a65a8Ph1kfbni5WffMJHt6SYU/nIf538072d/fkjRk8T7yNXYqI8n+odDbQB9wEfDBfQpFUB/wSuAjYBjwKXBERT0n6GnBXRNwp6d+AxyPiX4f63Kampmhuzpu78hr4wwLZecDzTd1aaF0fM51j5q83jr9f/hIuOudkDhzqpfNgL10He/mjWx5h5/5u8jl56iRef9ZJvOGsk9jd0c1H1zxV0PmYVTNJ6yKiach6aSSUwx8uPcTgCeXVwMciYlmyfk2yaTXQDpwcET0D6x3LcBPK+asfyHtZY/w4sXDAZEm/2tlBT9/R/y0H1i203lg75sDJp1qOUXf+iZMPr2/NdOatVzdOzDlhEhHZp9ef+/UB8lQbFgFbPvFGJB0uG06Ly6xaFZpQKvkeyjzgmZz1bcAryV7m2hMRPTnlg36DJV0JXAkwf/78YQUw2LXwnr5g0ezjjyjbtGN/QXULrTfWjnnm7ClHlG0+Rt2z50yl/yd9S3tH3nq9fcF5jTNAME7iG+u25a0H8H/e9GIa6utomDCOhvrxfPjuDWQ6jr6COndawxHJBGDFknlOIGaJkiUUSd8HTs6z6cMR8a1Sfe5AEXEzcDNkWyjD2XfutIa8LZR50xr43B+8/IiywVozA+sWWm+sHfPGP/iNI8oeO1bdt79Qd/3Wwet9+m2LD6//+OnMoPXe/ZojRwQ+cKg372W0q5ededT+ZvaCknUbjoilEfGSPK9Ck0kbkDt+xSlJWQaYJmn8gPKiu3rZmTQMGI9psB+WQuv6mOkcczifvWLJPD7xlnOZN60BkU06vi9iNrRKvuT1KLAo6dHVBlwOvD0iQtKDwFvJ9vRaCZSkxdP/A1LINfJC6/qY6RxzOJ/dX98JxGx40url9Wbgn4FZwB7gsYhYJmku2e7Bb0zqvRH4DNluw7dExHVJ+Wlkk8kMYD3wjojI3y0nx3BvypuZWZX08io3JxQzs+ErNKF46BUzMysKJxQzMysKJxQzMysKJxQzMyuKMXVTXlI70DrC3WfR+kEwAAAEt0lEQVQCO4sYTiWotXPy+VS+WjunWjsfyH9OCyJi1lA7jqmEMhqSmgvp5VBNau2cfD6Vr9bOqdbOB0Z3Tr7kZWZmReGEYmZmReGEUrib0w6gBGrtnHw+la/WzqnWzgdGcU6+h2JmZkXhFoqZmRWFE4qZmRWFE8oQJP2+pCcl9UlqyilvlNQl6bHk9W9pxlmowc4n2XaNpM2SNkoackrlSiTpY5Lacv4ub0w7ppGQdEnyd9gsaVXa8RSDpBZJG5K/S9WN0irpFkk7JP0sp2yGpPskbUrep6cZ43ANck4j/g45oQztZ8BbgB/k2fZ0RCxOXn9S5rhGKu/5SHox2TlnzgEuAT4nqe7o3avCDTl/l3vTDma4kv/uNwK/DbwYuCL5+9SC1yd/l2p8duM2st+NXKuA+yNiEXB/sl5NbuPoc4IRfoecUIYQET+PiI1px1Esxzif5cCdEdEdEb8CNgPnlTc6S5wHbI6ILRFxkOzcP8tTjmnMi4gfALsGFC8Hbk+WbwdWlDWoURrknEbMCWV0FkpaL+m/Jb027WBGaR7wTM76tqSsGl0l6YmkOV9VlyAStfS3yBXA9yStk3Rl2sEUyeyI2J4sPwfMTjOYIhrRd8gJBZD0fUk/y/M61r8KtwPzI2IJ8AHgK5KmlifiYxvh+VSNIc7vX4EXAYvJ/o3+KdVgLddrIuI3yF7Ke5+k16UdUDFF9hmMWngOY8TfoUqeU75sImLpCPbpBrqT5XWSngbOAFK/2TiS8wHagFNz1k9JyipOoecn6d+Bb5c4nFKomr/FcEREW/K+Q9LdZC/t5bs3WU2elzQnIrZLmgPsSDug0YqI5/uXh/sdcgtlhCTN6r9pncxxvwjYkm5Uo7IGuFzSREkLyZ7PIynHNGzJl7rfm8l2Qqg2jwKLJC2UNIFsZ4k1Kcc0KpKOkzSlfxm4mOr82wy0BliZLK8EvpViLEUxmu+QWyhDkPRm4J+BWcB3JD0WEcuA1wHXSjoE9AF/EhFFu7lVKoOdT0Q8KelrwFNAD/C+iOhNM9YR+pSkxWQvPbQA7003nOGLiB5JVwFrgTrgloh4MuWwRms2cLckyP7ufCUi/ivdkIZH0h3ABcBMSduAjwKrga9JejfZqTEuSy/C4RvknC4Y6XfIQ6+YmVlR+JKXmZkVhROKmZkVhROKmZkVhROKmZkVhROKmZkVhROKWQlI+lEJjtko6e3FPq5ZsTihmJVARPxmCQ7bCDihWMVyQjErAUn7k/cLJD0k6RuSfiHpy0qe7kvmB/lUMkfII5JOT8pvk/TWgcci+xDda5M5Kt5f7nMyG4oTilnpLQH+iuzcJqcB5+ds2xsR5wL/AnxmiOOsAv4nmaPihpJEajYKTihmpfdIRGyLiD7gMbKXrvrdkfP+6nIHZlZMTihmpdeds9zLkWPoRZ7lHpLvpqRxwISSRmdWJE4oZul6W877j5PlFuDlyfKlQH2yvA+YUrbIzIbJow2bpWu6pCfItmKuSMr+HfiWpMeB/wI6kvIngN6k/DbfR7FK49GGzVIiqQVoioidacdiVgy+5GVmZkXhFoqZmRWFWyhmZlYUTihmZlYUTihmZlYUTihmZlYUTihmZlYU/x8mAbL/dYtdGAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cx.plot_f(tanh, frange=(-15, 15, 1), xlabel=\"input\", ylabel=\"output\", title=\"tanh()\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## sigmoid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function sigmoid in module conx.activations:\n",
      "\n",
      "sigmoid(x)\n",
      "    Sigmoid activation function.\n",
      "    \n",
      "    >>> sigmoid(0)\n",
      "    0.5\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(sigmoid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.622459352016449"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sigmoid(0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xt4XOV17/Hv0l2+ysbCtuQrYAzGxjboEC4hcRISGwjYoSnBpGnDk4bkNKRNQ53iNoE8bs+B4KRtGsiFprmUFDhAHdkhTtQUQpOQG1IkYwyIGCPZHvluyVfdtc4fM1LH45E8krW1Z0a/z/Po8ex3lvasbWlm6d3v3u9r7o6IiAhATtgJiIhI+lBREBGRPioKIiLSR0VBRET6qCiIiEgfFQUREemjoiCjkpn9jZl9M91e18wazOy6uO0FZlZtZhbb/q2ZXTISucroZLpPQSR9mFkD8Kfu/l+x7f8AnnL3J2LbtwIfcPc/CC9LyWbqKYikKTObDrwDqIxr3gS8w8ymhZOVZDsVBcl6ZvbXZhYxs2NmVm9m7zKzz5vZ9+Ji/tjMGs3skJl9Lv40Tiz2KTP7XmwfW83sQjNba2b7zWyXmb0nbl9lZrbJzA6b2XYz+2jcc4mv+6G41/3bhNTfDfzO3dt6G2KPa4Dlw/8/JaKiIFnOzOYDdwH/y93HE/0wbUiIWQB8FfggMB2YCJQn7Oom4FFgElALVBF9/5QD64BvxMU+AewGyoD3A//XzN6ZJLcFwNeAD8VizwFmxIUsAuqTHNarwOIBD1xkiFQUJNt1A4XAAjPLd/cGd38jIeb9wA/c/Rfu3gHcCyQOtv3c3avcvQt4CigFHnD3TqJFYI6ZlZjZTOAa4K/dvc3d64BvAn+cJLf3A8+4+8/cvR34HNAT93wJcCzJ9x2LPScy7FQUJKu5+3bgU8Dngf1m9oSZlSWElQG74r7nJHAoIWZf3ONW4KC7d8dtA4yL7euwu8d/mDdyes8j2eueSHjdZmB8ku8bD7QkaRc5ayoKkvXc/TF3fyswm2gP4AsJIXuIO21jZsVET+UMRRMw2cziP8xnAZEksXuAmXGvOybhdV8CLkzyfRcDW4aYn8iAVBQkq5nZfDN7p5kVAm1E/6rvSQh7GrjJzK42swKivQobyuu5+y7gl8D9ZlZkZpcCHwG+lyT8aeC9ZvbW2Ouu49T35E+Ay8ysKO54ioDLY8+JDDsVBcl2hcADwEFgL3AusDY+wN23AZ8kOjawBzgO7Afah/iaq4E5RHsN3wfu673vIMnrfgJ4LPa6zUQHqHuf3wc8B6yM+7abgOfdvWmIuYkMSDeviSQws3FEz9nPc/c3Q85lAfBd4Ap3dzP7DfARd385zLwke6koiABmdhPwLNHTRl8C3gJc5nqDyCij00ciUSuJnu5pAuYBt6kgyGiknoKIiPRRT0FERPrkhZ3AYE2ZMsXnzJkTdhoiIhmlpqbmoLuXniku44rCnDlzqK6uDjsNEZGMYmaNqcTp9JGIiPRRURARkT4qCiIi0kdFQURE+qgoiIhIn8CuPjKzbwHvBfa7+8IkzxvwZeAG4CTwYXf/XVD5iEj6qayNsL6qnqaWVspKilmzfD6rliZbeiL12NG8z+EQ5CWp3wEeAv6tn+evJzqdwDyi88x8LfaviGS4VD7EKmsjrN2wldbO6FpFkZZW1m7YCjDk2NG8z+ES6DQXZjaH6HKDyXoK3yA6BfDjse16YJm77xlonxUVFa77FERG3mD+Ao7/EAMoysthzfL5vO3CUtq7emjv6uFjj1Zz8HjHad8/aUw+a2+4mJ4ep8eh250vVtVzpLXztNgJRXn872UX4LHVU7/+/Bscbes6LW58UR4fvfY84j/uvvmLHRzrJ/aOa+b2bX/7hTdTihswtjCPO66ZExfXwLH2M8cNFFteUswL95y29He/zKzG3SvOGBdiUXiG6Bq3v4htP0t0XdvTPvHN7E7gToBZs2Zd3tiY0j0YIjJMkn3Q5+caNyycTtmkYg4f7+DwyQ4On+hgy64Wuno0p1oii1u2aaCPXUtY3qm/WAPefODGQbx+akUhI+5odvdHgEcg2lMIOR2RrNJfD6Cjq4fX9h6lblcL929+7ZSCANDZ7Wzc0kR+rjF5bAGTxhRwzriCAQvCV1YvpTAvh4K8HP7qqS1JewpTJxTy9MevJjfHyM0xzODmh15g75G202KnTyziubuX9X2QvvOLz9OUJK6spIiffyb6V3XvZ+5bH3yOppbTYxP/Ar/mgeeItLSeMW4wscOxz7KS4tPahkOYVx9FiFuflugaucnWsRWRgPT2ACItrTjR89V3P7WFtz/4HAs/X8XND73AvRu3nVYQehnw+t9fz2/+5jp+/Km38e9/eiXl/XxYlZcUc9PiMt5zyTSWzT+Xz964gOL83FNiivNzWXv9xcycPIaykmKmTiji3PFF3LPioqSxf73iIooLcinKj359pp+4zyy/qK/I5MS+PrM8eeya5fNPaVuzfH5KcYOJDWKfwyXMnsIm4C4ze4LoAPORM40niMjwerDq9B5Ad4+z50g7H75mDktmlrB4Zgm3fv2XRJL8VV1WUowlnO9Ys3z+aaeakn2I9Y5HpDJOkWrsaN7ncAlsTMHMHgeWAVOAfcB9QD6Au389dknqQ8AKopek3pFsPCGRBppFzp678/zrB7jj2y8mfT7xfHWyMYXi/Fzuv2XRWV9uKSMj9DEFd199hued6KLlIjKCtjUd4f7Nr/GL7QfJzTG6k4wBJJ6vHuxfq6uWlqsIZKiMGGgWkbO350grX6x6nQ21u5lYnM99Ny1gfGEen0sYM+jvfLU+6EcHFQWRLBR/+mbaxCIWlk3g59sP0tMDd157Hn/2jguYWJwPQF5ujk71SB8VBZEsk3j+f8+RNvYcaeOymSV8efVSZk4ec0q8egASTxPiiWSZ9VX1SS8h3Xes/bSCIJJIRUEkyzQludFpoHaReCoKIlmmdHxh0vag7oCV7KKiIJJFWju6ybHT24O8A1ayi4qCSBZZ98w29h5t52NvP4/ykmKM6PQS/d1kJpJIVx+JZIkfbGni8d/u4uNvP597rr+ItddfHHZKkoHUUxDJAo2HTrB2w1Yum1XC3e+5MOx0JIOpKIhkuPaubu56rJYcg39evZT8XL2tZeh0+kgkw33hR/VsjRzhGx+6nBmTdB+CnB39SSGSwX7yyj6+9cKbfPjqOSy/ZFrY6UgWUFEQyVBNLa2seXoLl5RNYO0NF4WdjmQJFQWRDNTV3cOfP15LZ1cPD91+GYV5uWf+JpEUaExBJIP0zn7au2bvh66cxdwpY0POSrKJegoiGSJ+PeVeT9dEqKzV0uYyfFQURDJEstlPWzu7WV9VH1JGko1UFEQyhGY/lZGgoiCSIfqb5VSzn8pwUlEQyRCfeMf5p7Vp9lMZbioKIhnCLDondun4Qs1+KoHRJakiGaKyNsJ5pWN59tNv7ysQIsNNPQWRDNDU0spv3jzMqiXlKggSKBUFkQzwgy1NANy8uCzkTCTbqSiIZIDKuiaWzCxhju5eloCpKIikudf3HePVPUdZtUS9BAmeioJImqusjZCbY9x4qYqCBE9FQSSNuTsb65q45oIplI4vDDsdGQVUFETSWE1jM5GWVp06khGjoiCSxirrIhTl5/AeraomIyTQomBmK8ys3sy2m9k9SZ6fZWY/NbNaM3vJzG4IMh+RTNLZ3cMPX9rDuxdMY1yh7jOVkRFYUTCzXOBh4HpgAbDazBYkhH0WeNLdlwK3AV8NKh+RTPOz1w/QfLJTp45kRAXZU7gC2O7uO9y9A3gCWJkQ48CE2OOJQFOA+YhklI11TZSMyefaeaVhpyKjSJBFoRzYFbe9O9YW7/PAH5nZbmAz8MlkOzKzO82s2syqDxw4EESuImnlRHsXP3llHzcumk5Bnob+ZOSE/du2GviOu88AbgAeNbPTcnL3R9y9wt0rSkv1V5Nkv/98ZS+tnd2aAVVGXJBFIQLMjNueEWuL9xHgSQB3/xVQBEwJMCeRjFBZ20R5STGXz5oUdioyygRZFF4E5pnZXDMrIDqQvCkhZifwLgAzu5hoUdD5IRnVDh5v5xfbD7JySRk5OZoRVUZWYEXB3buAu4Aq4FWiVxltM7N1ZnZzLOxu4KNmtgV4HPiwu3tQOYlkgh++tIfuHmflEp06kpEX6MXP7r6Z6AByfNu9cY9fAa4JMgeRTFNZF+GiaeOZP2182KnIKBT2QLOIxGk8dILanS0aYJbQqCiIpJFNdU2YaTEdCY+KgkiacHcq6yJcMWcyZSXFYacjo5SKgkia2NZ0lDcOnNAAs4RKs2yJhKyyNsL6qnoiLa2xFl2AJ+FRURAJUWVthLUbttLa2d3X9nfPvMqYgjwNNksodPpIJETrq+pPKQgArZ3drK+qDykjGe1UFERC1NR3yii1dpGgqSiIhKi/q4x09ZGERUVBJERrls+nKP/Ut2Fxfi5rls8PKSMZ7VQUREK0amk5d1w9t2+7vKSY+29ZpEFmCY2uPhIJWWF+Dmaw5b73MKEoP+x0ZJRTT0EkZDWNzcyfOl4FQdKCioJIiLp7nNqdLVTM0WI6kh5UFERCVL/3GMfbu6iYPTnsVEQAFQWRUNU0Hgbg8tnqKUh6UFEQCVF1YzNTJxQyY5LuS5D0oKIgEqLqhmYunz0JM63FLOlBRUEkJHuPtBFpaeVyjSdIGlFREAlJTWMzABUaT5A0oqIgEpLqxsMU5+eyoGxC2KmI9FFREAlJTWMzi2dOJD9Xb0NJH/ptFAnByY4utjUd1f0JknZUFERCULerhe4e1/0JknZUFERC8LvYIPNls1QUJL2oKIiEoLqxmQunjmPiGE2CJ+lFRUFkhPX0OL9rbNb9CZKWVBRERtjv9x/naFuX7k+QtKSiIDLCqjUJnqQxFQWREVbT0MyUcQXMPmdM2KmInCbQomBmK8ys3sy2m9k9/cTcamavmNk2M3ssyHxE0kHNTk2CJ+krsDWazSwXeBh4N7AbeNHMNrn7K3Ex84C1wDXu3mxm5waVj0g6OHCsncZDJ/mjt8wOOxWRpILsKVwBbHf3He7eATwBrEyI+SjwsLs3A7j7/gDzEQld36I6Wn5T0lSQRaEc2BW3vTvWFu9C4EIze8HMfm1mK5LtyMzuNLNqM6s+cOBAQOmKBK+6oZmCvBwWlk0MOxWRpMIeaM4D5gHLgNXAv5hZSWKQuz/i7hXuXlFaWjrCKYoMn+rGZhbPmEhBXthvPZHkgvzNjAAz47ZnxNri7QY2uXunu78JvE60SIhknbbObrY1HdFNa5LWgiwKLwLzzGyumRUAtwGbEmIqifYSMLMpRE8n7QgwJ5HQvLT7CJ3drpvWJK0FVhTcvQu4C6gCXgWedPdtZrbOzG6OhVUBh8zsFeCnwBp3PxRUTiJh0k1rkgkCuyQVwN03A5sT2u6Ne+zAp2NfIlmtpqGZ80vHMmlsQdipiPQrpZ6Cmf1FKm0iklxPj1Ozs1mL6kjaS/X00Z8kafvwMOYhktV2HDxOy8lOnTqStDfg6SMzWw3cDsw1s/hB4vHA4SATE8km1Q3RRXV005qkuzONKfwS2ANMAb4U134MeCmopESyTU1jM5PHFnDelLFhpyIyoAGLgrs3Ao3AVSOTjkh2qmls5rJZmgRP0l+qA83HzOxo7KvNzLrN7GjQyYlkg0PH29lx8AQVOnUkGSClS1LdfXzvY4v+qbMSuDKopESySU1jbDxBg8ySAQZ985pHVQLLA8hHJOvUNDZTkJvDonJNgifpL6WegpndEreZA1QAbYFkJJJlqhubWVg+gaL83LBTETmjVO9ovinucRfQwOlrI4hIgvaubrbuPsKHr5kTdioiKUl1TOGOoBMRyUYvR47Q0d2j8QTJGKlefXSemf3AzA6Y2X4z22hm5wWdnEgmq6yNcMe3XwTgvk3bqKxNnDleJP2kOtD8GPAkMB0oA54CHg8qKZFMV1kbYe2GrRxt6wJg75E21m7YqsIgaS/VojDG3R91967Y1/eAoiATE8lk66vqae3sPqWttbOb9VX1IWUkkppUB5p/ZGb3AE8ADnwA2GxmkwHcXfMgicRpamkdVLtIuki1KNwa+/djCe23ES0SGl8QiVNWUkwkSQEoKykOIRuR1KV6+uhid58b/xXXpoIgkmDN8vnk5pw6z1Fxfi5rls8PKSOR1KRaFH6ZYpuIAKuWlnPO2AIK83IwoLykmPtvWcSqpeVhpyYyoDOtpzANKAeKzWwp0PunzwRgTMC5iWSslpMd7D/Wzprl8/nEOy4IOx2RlJ1pTGE50RXWZgD/ENd+DPibgHISyXiaBE8y1ZnWU/gu8F0z+wN3/48Rykkk41U3NpOXYyyeURJ2KiKDkurVRwvN7JLERndfN8z5iGSFmsZmLimfSHGBJsGTzJLqQPNx4ETsqxu4HpgTUE4iGa2jq4ctu1qo0KkjyUCpTogXvz4zZvZFoCqQjEQy3LamI7R3aRI8yUyDXmQnZgzRwWcRSdA7yKyegmSiVBfZ2Ur0zmWIFpJzgb8LKimRTFbd0MzMycWcO0HTg0nmSXWg+b3AJOBaoATY7O41gWUlkqHcnZqdzbz1gilhpyIyJKmePloJPApMAfKBb5vZJwPLSiRD7TrcyoFj7RpPkIyVak/hT4Er3f0EgJl9AfgV8JWgEhPJRNWN0QmDK+aoKEhmSrWnYEQvRe3Vzf9MeSEiMdWNzYwvzGPeuePDTkVkSFLtKXwb+I2ZfT+2vQr412BSEslcNQ3NLJ096bQZUkUyRUo9BXf/B+AO4HDs6w53/6czfZ+ZrTCzejPbHlukp7+4PzAzN7OKVBMXSTdHWjt5ff8xXYoqGS3VngLu/jvgd6nGm1ku8DDwbmA38KKZbXL3VxLixgN/Afwm1X2LpKPanc246/4EyWxDvXktFVcA2919h7t3EF3Kc2WSuL8DvgC0BZiLSOBqGpvJzTGWzNIkeJK5giwK5cCuuO3dsbY+ZnYZMNPdfzjQjszsTjOrNrPqAwcODH+mIsOguqGZBdMnMKYg5Q64SNoJsigMyMxyiK7RcPeZYt39EXevcPeK0tLS4JMTGaTO7h7qdrXo/gTJeEEWhQgwM257Rqyt13hgIfC8mTUAVwKbNNgsmejVPUdp7exWUZCMF2RReBGYZ2ZzzawAuA3Y1Pukux9x9ynuPsfd5wC/Bm529+oAcxIJRN8keLppTTJcYEXB3buAu4hOsf0q8KS7bzOzdWZ2c1CvKxKG6sZmykuKmT6xOOxURM5KoCNi7r4Z2JzQdm8/scuCzEUkKO5OTUMzV8ydHHYqImcttIFmkWwRaWll79E2nTqSrKCiIHKWescTLpuloiCZT0VB5CzVNDYztiCXi6ZpEjzJfCoKImepuqGZpbMmkZert5NkPv0Wi5yF4+1dvLb3qO5PkKyhoiByFmp3NtPjuj9BsoeKgshZqG5oJsdgyUxNgifZQUVB5CzUNDYzf9oExhflh52KyLBQURAZou4ep3Zns9ZPkKyioiAyRK/tPcqJjm6NJ0hWUVEQGaLem9Z05ZFkExUFkSGqbmhm2oQiyks0CZ5kDxUFkSGqaWzm8tmTMLOwUxEZNioKIkOw50grkZZWnTqSrKOiIDIEWlRHspWKgsgQVDc0U5yfy8XTJ4SdisiwUlEQGYKaxmaWzCwhX5PgSZbRb7TIIFTWRrjq/mfZGjnCy5EjVNZGwk5JZFgFuhynSDaprI2wdsNWWju7ATjW3sXaDVsBWLW0PMzURIaNegoiKVpfVd9XEHq1dnazvqo+pIxEhp+KgkiKmlpaB9UukolUFERSVNbPncv9tYtkIhUFkRStWT6fvJxT714uzs9lzfL5IWUkMvxUFERSdNPiMsYU5FKUl4MB5SXF3H/LIg0yS1bR1UciKfrNm4c42tbFV1Yv5abFZWGnIxII9RREUrSxtomxBblcd/HUsFMRCYyKgkgK2jq72fzyHpYvnEZxQW7Y6YgERkVBJAXP1x/gWFsXq5Zo/ECym4qCSAo21kWYMq6Aq88/J+xURAKloiByBkfbOnn2tf2899Iy8jQBnmS5QH/DzWyFmdWb2XYzuyfJ8582s1fM7CUze9bMZgeZj8hQ/HjrXjq6enTpqYwKgRUFM8sFHgauBxYAq81sQUJYLVDh7pcCTwMPBpWPyFBt3BJhzjljWDxjYtipiAQuyJ7CFcB2d9/h7h3AE8DK+AB3/6m7n4xt/hqYEWA+IoO272gbv3zjEDcvKddazDIqBFkUyoFdcdu7Y239+Qjwo2RPmNmdZlZtZtUHDhwYxhRFBvaDLU24w6olullNRoe0GDUzsz8CKoD1yZ5390fcvcLdK0pLS0c2ORnVKusiXDpjIueVjgs7FZEREWRRiAAz47ZnxNpOYWbXAX8L3Ozu7QHmIzIo2/cf5+XIUVbq3gQZRYIsCi8C88xsrpkVALcBm+IDzGwp8A2iBWF/gLmIDNqmugg5BjddOj3sVERGTGBFwd27gLuAKuBV4El332Zm68zs5ljYemAc8JSZ1ZnZpn52JzKi3J3KuiauPn8K504oCjsdkRET6Cyp7r4Z2JzQdm/c4+uCfH2Roard1cLOwyf55DsvCDsVkRGVFgPNIulmY22EwrwcViycFnYqIiNKRUEkQWd3D8+8tIfrLp7K+KL8sNMRGVEqCiIJXth+kEMnOrhZ9ybIKKSiIJJgY10TE4ryWDZf98TI6KOiIBLnZEcXVdv2cuOl0ynM02I6MvqoKIjE+ckr+zjZ0a0b1mTUUlEQibOpronpE4u4Ys7ksFMRCYWKgghQWRvhqvuf5dnX9nOsrYtNW5rCTkkkFIHevCaSCSprI6zdsJXWzm4Ajrd3sXbDVgAtrCOjjnoKMuqtr6rvKwi9Wju7WV9VH1JGIuFRUZBRr6mldVDtItlMRUFGvXMnFCZtLyspHuFMRMKnoiCjWmd3D0V5p78NivNzWbN8fggZiYRLRUFGtS/95+s0Hm7lj6+aTXlJMQaUlxRz/y2LNMgso5KuPpJR6/n6/Xz9v99g9RWzWLdyIetWhp2RSPjUU5BRad/RNu5+cgvzp47nvpsWhJ2OSNpQUZBRp7vH+dQTdZzs6Oah25dSlK85jkR66fSRjDoPPbedX+04xIPvv5R5U8eHnY5IWlFPQUaVX+84xJeffZ1VS8r4w8tnhJ2OSNpRUZBR49Dxdv7iiVpmnzOWv3/fIsws7JRE0o6KgowKPT3OXz21heYTnTx0+1LGFerMqUgyKgoyKvzrL97kp/UH+Ox7L+aSsolhpyOStvTnkmStytoI66vqaWppxYFF5RP40JWzw05LJK2ppyBZqXc67EisIAD8fv9xNtZpnQSRgagoSFZ6sOq106bDbuvs0XTYImegoiBZ51dvHKKppS3pc5oOW2RgGlOQrLF9/3Ee+NFr/Ner+8gx6PHTYzQdtsjAVBQk48QPIJeVFPPxZefx+t7jPPbbnRTn5/KZFfMpHVfIvRu3nXIKSdNhi5yZioJklMT1lCMtrXyuchsGfOiq2fz5u+YxZVx00Zz83JxTisea5fM1HbbIGagoSEZZn2QAGaB0fCHrVi48pW3V0nIVAZFBUlGQQCWe6unvr/X+4g4db2fL7hbqdh2hblcLkX4GkA8caw/6UERGhUCLgpmtAL4M5ALfdPcHEp4vBP4NuBw4BHzA3RuGO49UP5gGE6t9phaXeKpn7YatAKfEV9ZGuGfDS7R19vTF3f1kHeue2cbhE50A5BhcOHU8YwpyOdlxek9BA8giw8Pck1yiMRw7NssFXgfeDewGXgRWu/srcTF/Blzq7h83s9uA97n7Bwbab0VFhVdXV6ecR+IHE0QHHJMtt5hqrPZ5alxRfg73XH8Ryy48l47uHto7e+jo7uZjj9Zw8HgHiYrzc7nq/HM4fKKDwyc62HX4JMl+C4vyc/j0uy9k8YwSFs2YyJiCvEEdk4j8DzOrcfeKM8YFWBSuAj7v7stj22sB3P3+uJiqWMyvzCwP2AuU+gBJDbYoXPPAc0SSXJuel2PMnTL2lLY3D56gK8l1jHk5xpy42IYU4wYTO1Dc7HPGnPKhufPQyaSxuTnGrMlj6P3v29XcSneyODOmTSzC3XHAHfYfa0t6CacBxQW5dPc4Pe50dg/P78vC8glMHlvI5DH5VPZzl7EBbz5w42ntg+n9iEhUqkUhyNNH5cCuuO3dwFv6i3H3LjM7ApwDHIwPMrM7gTsBZs2aNagk+rtZqavHmTd13Cltv99/vN/Y+XGLsWxPMW4wsQPFXTR9AhD9kATYceBE0tjuHmdReXSyNzNoOHQyeZw7V553DmbRfZrBk9W7k8Y68MG3zCLHjJwc42vPv5E0DuAfP7CYwrxcCnJzKMzP4S//X13SnkJ5STHPfPLavu0XG5qTFu7+TglpAFkkOBkx0OzujwCPQLSnMJjvLSspTvqBU15SzFc/ePkpbf31KspLinn4g5f1bdelGDeY2AHjbj91n7U7+4/959VL+7ar+/mwLS8p5ku3Lj6l7YXth/qN/dsb/2cN4011Tf3GvW/pqYvWfPbGBUlP9STeK7Bm+fyU4kQkeEFOcxEBZsZtz4i1JY2JnT6aSHTAedisWT6f4oQ1ePv7wEk1VvtMbZ+rlpZz/y2LKC8pxogWjmTn/lONE5HgBTmmkEd0oPldRD/8XwRud/dtcTGfABbFDTTf4u63DrTfwY4pQPZd1ZNJ+xSR9BD6QHMsiRuAfyJ6Seq33P3/mNk6oNrdN5lZEfAosBQ4DNzm7jsG2udQioKIyGiXDgPNuPtmYHNC271xj9uAPwwyBxERSZ2mzhYRkT4qCiIi0kdFQURE+qgoiIhIn0CvPgqCmR0AGof47VNIuFs6C2TbMWXb8UD2HVO2HQ9k3zElO57Z7l56pm/MuKJwNsysOpVLsjJJth1Tth0PZN8xZdvxQPYd09kcj04fiYhIHxUFERHpM9qKwiNhJxCAbDumbDseyL5jyrbjgew7piEfz6gaUxARkYGNtp6CiIgMQEVBRET6jIqiYGZ/aGbbzKzHzCri2ueYWauZ1cW+vh5mnqnq73hiz601s+1mVm9my8PK8WyY2efNLBL3c7kh7JyJKjYCAAAEKUlEQVSGwsxWxH4O283snrDzGQ5m1mBmW2M/l4ybrtjMvmVm+83s5bi2yWb2EzP7fezfSWHmOFj9HNOQ30OjoigALwO3AD9L8twb7r4k9vXxEc5rqJIej5ktAG4DLgFWAF81s9zTvz0j/GPcz2XzmcPTS+z//WHgemABsDr288kG74j9XDLxuv7vEH1vxLsHeNbd5wHPxrYzyXc4/ZhgiO+hUVEU3P1Vd68PO4/hMsDxrASecPd2d38T2A5cMbLZScwVwHZ33+HuHcATRH8+EiJ3/xnRtVvirQS+G3v8XWDViCZ1lvo5piEbFUXhDOaaWa2Z/beZXXvm8LRWDuyK294da8tEd5nZS7GucUZ152Oy6WcRz4H/NLMaM7sz7GSGyVR33xN7vBeYGmYyw2hI76GsKQpm9l9m9nKSr4H+OtsDzHL3pcCngcfMbMLIZDywIR5PxjjD8X0NOB9YQvRn9KVQk5V4b3X3y4ieFvuEmb0t7ISGk0ev0c+G6/SH/B4KdOW1keTu1w3he9qB9tjjGjN7A7gQCH0AbSjHQ3Qt7Jlx2zNibWkn1eMzs38Bngk4nSBkzM9iMNw9Evt3v5l9n+hpsmRjdZlkn5lNd/c9ZjYd2B92QmfL3ff1Ph7seyhregpDYWalvQOxZnYeMA8YcI3oNLcJuM3MCs1sLtHj+W3IOQ1a7I3Z631EB9YzzYvAPDOba2YFRC8A2BRyTmfFzMaa2fjex8B7yMyfTaJNwJ/EHv8JsDHEXIbF2byHsqanMBAzex/wFaAU+KGZ1bn7cuBtwDoz6wR6gI+7+7AN2ASlv+Nx921m9iTwCtAFfMLdu8PMdYgeNLMlRLvxDcDHwk1n8Ny9y8zuAqqAXOBb7r4t5LTO1lTg+2YG0c+Ox9z9x+GmNDhm9jiwDJhiZruB+4AHgCfN7CNEp+W/NbwMB6+fY1o21PeQprkQEZE+o/r0kYiInEpFQURE+qgoiIhIHxUFERHpo6IgIiJ9VBREBmBmvwxgn3PM7Pbh3q/IcFBREBmAu18dwG7nACoKkpZUFEQGYGbHY/8uM7PnzexpM3vNzP7dYndxxdYYeDC2zsBvzeyCWPt3zOz9ifsierPUtbF57v9ypI9JZCAqCiKpWwp8iuj6COcB18Q9d8TdFwEPAf90hv3cA/w8Ns/9PwaSqcgQqSiIpO637r7b3XuAOqKngXo9HvfvVSOdmMhwUVEQSV173ONuTp07zJM87iL2HjOzHKAg0OxEhoGKgsjw+EDcv7+KPW4ALo89vhnIjz0+BowfscxEBmFUzJIqMgImmdlLRHsTq2Nt/wJsNLMtwI+BE7H2l4DuWPt3NK4g6USzpIqcJTNrACrc/WDYuYicLZ0+EhGRPuopiIhIH/UURESkj4qCiIj0UVEQEZE+KgoiItJHRUFERPr8f3OkU6EXpxNWAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cx.plot_f(sigmoid, frange=(-15, 15, 1), xlabel=\"input\", ylabel=\"output\", title=\"sigmoid()\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## hard_sigmoid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function hard_sigmoid in module conx.activations:\n",
      "\n",
      "hard_sigmoid(x)\n",
      "    Hard Sigmoid activation function.\n",
      "    \n",
      "    >>> round(hard_sigmoid(-1), 1)\n",
      "    0.3\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(hard_sigmoid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6000000238418579"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hard_sigmoid(0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XuYXXV97/H3J5NJmFxgyIVLJglJSIiCIIEYLokWrTZgLVBrFehFfajYp9JjTz1p4ZzWWnr6gMZabMVaaBWLFw6tGKNG09YrM1wDQWLAmJ0byQQhe5IJuUwuM/M9f+ydcWdnLnsms2bty+f1PPPM3r/922t/VyZ7PrPWb+3fTxGBmZkZwKi0CzAzs/LhUDAzsx4OBTMz6+FQMDOzHg4FMzPr4VAwM7MeDgUrW5K2SnprQtv+mKQvncTzvyPpvcNZ08m+rqRZkkLS6IK2D0q6O3/7TEkvSBo7UvVa5Rk9cBczKxYR15T760oaA/wFcHn+uS9L+gFwC/CPyVRolc5HClbVlFOr/8+vA34WEa0FbV8GPphSPVYBavXNYpXjYknPSdor6f9JOkXS6ZK+JWmXpD3529OPPUHSDyX9raQW4CAwR9JsST+StE/SfwFTBnrh/Gt9SVKbpHZJT0k6s+A1/iB/u07S30nKStoi6dbC0zj5vv9X0qOS9kv6pqTJkr4s6dX8dmcVvO6V+ba9+e9XFu1b4et+Mv+6m4FfL9qFa4AfFbU9kf/3OKfUH4DVFoeClbt3A1cDs4GLgPeR+3/7BeAcYCbQAXym6Hm/R+40yURgG/AV4GlyYfA3QCnjAe8FTgNmAJOBP8y/VrEPkPsFfDFwCXB9L31uyNfUBJwLPJbfh0nAC8BfAUiaBHwb+If8a34K+LakyX287juABcBC4F1Fj18IbChsiIhOIAO8vs+9tprmULBy9w8RsTMidgPfBC6OiLaI+FpEHIyIfcDfAr9S9Lz7I2J9/pfg2cAbgL+MiMMR8eP8tgZylNwv5rkR0RURT0fEq730ezfw6YjYERF7gLt66fOFiNgUEXuB7wCbIuK/8/X9O7lf7JD7a39jRDwQEZ0R8VXgZ8Bv9PG6d0fE9vy/z51FjzcC+3p53r78Y2YncChYuftFwe2DwARJ4yT9s6Rtkl4Ffgw0Sqor6Lu94PY0YE9EHCho21bCaz8ArAYelLRT0ick1ffSb1rR623vpc/LBbc7erk/oWBbxbVtI3eEMdDrFj9vD7kjpWITgfZe2s0cClaRPgLMBy6LiFOBN+XbVdCncPrfl4DTJY0vaJs50ItExNGI+OuIOB+4ktypmt/vpetLwPSC+zMG3oU+7SR3WqzQTKC1l74vFb1W8T49B5xX2JAf55gL/OQkarQq5lCwSjSR3F/X7flz8H/VX+eI2AasAf5a0hhJS+j9dMxxJL1Z0oX5I5BXyZ1O6u6l60PAhyU1SWoE/nxwu3OcVcB5km6SNFrSe4DzgW/18br/Q9J0SacDt/WyreLTaouArfl/E7MTOBSsEt0NNABZ4HHguyU85ybgMmA3uRD5txKecxbwH+QC4QVyV/I80Eu/+4D/JPeX+Vpyv4w7ga4SXuM4EdFG7ojkI0Ab8GfAOyIi28frrib3V/8zwMNFj38TeI2kaQVtvwN8brB1We2QF9kxG16SrgE+FxGpX/Yp6Rbg/Ij4E0lnkAu2BRFxKOXSrEw5FMxOkqQG4M3kjhbOBL4GPB4Rf5JqYWZD4NNHVtMk/U7+A2XFX+sHsxngr8ld7bOW3KmmjyZRr1nSfKRgZmY9fKRgZmY9Km6W1ClTpsSsWbPSLsPMrKI8/fTT2YiYOlC/iguFWbNmsWbNmrTLMDOrKJJK+myKTx+ZmVkPh4KZmfVwKJiZWQ+HgpmZ9XAomJlZj8SuPpL0eXITe70SEa/r5XEBnwbeTm6e/PdFxDNJ1WNWS1asbWX56g3sbO9gWmMDy5bO5/oFvS3JUHpfb7P8tzkckrwk9X5ySyT2NRvlNcC8/NdlwD/lv5vZSVixtpXbH15Hx9HcJK2t7R3c/vA6gBN+kZTa19ss/20Ol0SnucgvRv6tPo4U/hn4YX65QSRtAK6KiJf62+bChQvDn1Mw69viu75Pa/uJS0lPHDua9y+edVzbF1q2su9w54B9S+3nbY7cNpsaG2i57S0ntPdF0tMRsXDAfimGwreAuyKiOX//e8CfR8QJv/Hz0//eAjBz5sxLt23z+iBmfZl927fp610tHX+/v7d/Yd9S+3mbI7dNAVvu+vW+N3TCdksLhYr4RHNE3AvcC7kjhZTLMStr0xobej1S6O0vy76OKor7ltrP2xy5bU5rbDihbTikefVRK8evLzud3tehNbNB+MjbzqPoj00a6utYtnT+CX2XLZ1PQ33dgH1L7edtprfN4ZLmkcJK4FZJD5IbYN470HiCmQ2sGwhg0rh69hw82u/VKsfaBrqypdR+3mZ62xwuiY0pSPoqcBUwBXiZ3Lq49QAR8bn8JamfAa4md0nq+3sbTyjmgWazvh3u7OItn/wRk8aPYeWti1HxCWqrWamPKUTEjQM8HsCHknp9s1r05cdfpLW9g4//1kUOBBsSf6LZrErsP9zJZ36QYfHcySyZNyXtcqxCORTMqsS/PLKZ3QeO8GdLX5N2KVbBHApmVaBt/2Hu+/FmrnndWbx+RmPa5VgFcyiYVYF7frCJjqNdfOTXkrlM0WqHQ8Gswu3Yc5AvPb6N3750BnPPmJB2OVbhHApmFe7T/70RBB9+67y0S7Eq4FAwq2AbX97H157Zwe9ffk5i0x5YbXEomFWwT/7nBsaNGc0fvXlu2qVYlXAomFWotS/uYfX6l7nlTXOYNH5M2uVYlXAomFWgiODj3/0Zk8eP4eYls9Mux6qIQ8GsAj2yMcvjm3fzx2+Zy/ixFTEDvlUI/28yqyAr1rbyidU/Y2f7IepGiQkOBBtm/h9lViGK1+rt6g7+8hvrGV03KrFplK32+PSRWYVYvnpDTyAc03G0i+WrN6RUkVUjh4JZhdjZy5KM/bWbDYVDwaxC9PXhNH9ozYaTQ8GsQixbOp9RRevmJLlWr9Umh4JZhbj6dWchYPzYOgQ0NTZw5zsv9CCzDStffWRWIZ7ZtoeugH+4YQG/+toz0y7HqpSPFMwqRHMmS90ocdmcyWmXYlXMoWBWIVoyWRbMaPQH1ixRDgWzCrD34FGea93L4rlT0i7FqpxDwawCPLa5jQhYMs+hYMlyKJhVgJZMlvFj6rh4RmPapViVcyiYVYCWTJbL5kymvs5vWUuW/4eZlbnW9g42Zw94PMFGhEPBrMy1ZLIALHEo2AhwKJiVuZZMlikTxnLemRPSLsVqgEPBrIxFBC2ZLEvmTkbSwE8wO0kOBbMytuHlfWT3H/F4go0Yh4JZGWvemBtPcCjYSEk0FCRdLWmDpIyk23p5fKakH0haK+k5SW9Psh6zStOSyTJn6nivmWAjJrFQkFQH3ANcA5wP3Cjp/KJufwE8FBELgBuAzyZVj1mlOdLZzRNbdvuqIxtRSR4pLAIyEbE5Io4ADwLXFfUJ4NT87dOAnQnWY1ZRnt3ezsEjXT51ZCMqyVBoArYX3N+Rbyv0MeB3Je0AVgF/3NuGJN0iaY2kNbt27UqiVrOy05zJMkpwuafKthGU9kDzjcD9ETEdeDvwgKQTaoqIeyNiYUQsnDp16ogXaZaGlkyWi6Y3clpDfdqlWA1JMhRagRkF96fn2wrdDDwEEBGPAacAPla2mrfv0FGe3d7u8QQbcUmGwlPAPEmzJY0hN5C8sqjPi8CvAkh6LblQ8Pkhq3lPbN5NV3d4PMFGXGKhEBGdwK3AauAFclcZrZd0h6Rr890+AnxA0k+ArwLvi4hIqiazStGcyXJK/SguOcdTZdvISnRdv4hYRW4AubDtowW3nwcWJ1mDWSVqyWRZNHsyY0fXpV2K1Zi0B5rNrMjLrx5i4yv7WTLXVx3ZyHMomJWZY1NlezzB0uBQMCszzZksk8aP4bVnnTpwZ7Nh5lAwKyPHpsq+4tzJjBrlqbJt5DkUzMrIpl37efnVw/58gqXGoWBWRo5Nle1QsLQ4FMzKSHOmjZmTxjFj0ri0S7Ea5VAwKxOdXd08vrnNVx1ZqhwKZmXiJzv2sv9wp08dWaocCmZloiWTRYIrzvWH1iw9DgWzMtGcyXLBtFOZNH5M2qVYDXMomKVsxdpWrrjzezy5ZTdbswdZsbZ4hnmzkZPohHhm1r8Va1u5/eF1dBztAmD/4U5uf3gdANcvKF6o0Cx5PlIwS9Hy1Rt6AuGYjqNdLF+9IaWKrNY5FMxStLO9Y1DtZklzKJilaFpjw6DazZLmUDBL0bKl86mvO37iu4b6OpYtnZ9SRVbrHApmKbp+QROXzjwdAQKaGhu4850XepDZUuOrj8xSFBFs39PB0gvO4nO/d2na5Zj5SMEsTdvaDtLa3sHieZ7awsqDQ8EsRc0ZT5Vt5cWhYJailkyWpsYGZk32VNlWHhwKZinp6g4e3dTG4rmTkbz0ppUHh4JZStbv3MvejqNeP8HKikPBLCXHxhOuPNehYOXDoWCWkpZMltecNZGpE8emXYpZD4eCWQoOHe3iqa17fOrIyo5DwSwFa7bu4Uhnty9FtbLjUDBLQXMmy+hRYtHsSWmXYnYch4JZCloyWS6ZeTrjx3qmGSsviYaCpKslbZCUkXRbH33eLel5SeslfSXJeszKwZ4DR/jpzr0eT7CylNifKZLqgHuAtwE7gKckrYyI5wv6zANuBxZHxB5JZyRVj1m5eGxzGxGwZN7ktEsxO0GSRwqLgExEbI6II8CDwHVFfT4A3BMRewAi4pUE6zErC82ZLBPGjuai6Y1pl2J2giRDoQnYXnB/R76t0HnAeZJaJD0u6ereNiTpFklrJK3ZtWtXQuWajYyWTJbL50yivs5DelZ+0v5fORqYB1wF3AjcJ+mEP58i4t6IWBgRC6dOnTrCJZoNn+27D7Kt7aDHE6xsJRkKrcCMgvvT822FdgArI+JoRGwBfk4uJMyqUounyrYyl2QoPAXMkzRb0hjgBmBlUZ8V5I4SkDSF3OmkzQnWZJaq5kyWMyaOZe4ZE9IuxaxXiYVCRHQCtwKrgReAhyJivaQ7JF2b77YaaJP0PPADYFlEtCVVk1mauvNTZS+ZO8VTZVvZSvSTMxGxClhV1PbRgtsB/Gn+y6yqvfCLV9l94IjHE6yslXSkIOnDpbSZWd+OjSc4FKyclXr66L29tL1vGOswq3rNmTbmnjGBs047Je1SzPrU7+kjSTcCNwGzJRUOEk8EdidZmFk1OdzZxZNb2rjhDTPTLsWsXwONKTwKvARMAf6uoH0f8FxSRZlVm2e2tXPoaLdPHVnZ6zcUImIbsA24YmTKMatOLZksowSXzfFU2VbeSrr6SNI+IPJ3xwD1wIGIODWpwsyqSXMmy+tnNHLqKfVpl2LWr5JCISImHrut3AXW1wGXJ1WUWTV59dBRntvRzofePDftUswGNOgPr0XOCmBpAvWYVZ3HN7XRHb4U1SpDqaeP3llwdxSwEDiUSEVmVaYlk6Whvo4FMz1VtpW/Uj/R/BsFtzuBrZy4NoKZ9aI5k2XR7EmMHV2XdilmAyp1TOH9SRdiVo1e2tvBpl0H/PkEqxilTnMxR9I3Je2S9Iqkb0iak3RxZpVsxdpWrvn0IwDc98hmVqwtnjnerPyUOtD8FeAh4GxgGvDvwFeTKsqs0q1Y28rtD6+j/eBRAF7Zd5jbH17nYLCyV2oojIuIByKiM//1JcATuJj1YfnqDXQc7TqureNoF8tXb0ipIrPSlDrQ/B1JtwEPkvsQ23uAVZImAUSE50EyK7CzvWNQ7WblotRQeHf++weL2m8gFxIeXzArMK2xgdZeAmBaY0MK1ZiVrtTTR6+NiNmFXwVtDgSzIsuWzmdU0eJqDfV1LFs6P52CzEpUaig8WmKbmQG/ftHZjB4lxo2pQ0BTYwN3vvNCrl/QlHZpZv0aaD2Fs4AmoEHSAuDY3z6nAuMSrs2sYv1keztHuoJP3/B6rrnw7LTLMSvZQGMKS8mtsDYd+FRB+z7gfydUk1nFa85kkeCKcyenXYrZoAy0nsIXgS9K+q2I+NoI1WRW8VoyWS5sOo3GcWPSLsVsUEq9+uh1ki4oboyIO4a5HrOKt/9wJ2tfbOcDb/I1GFZ5Sg2F/QW3TwHeAbww/OWYVb4nt7TR2R0s8VTZVoFKnRCvcH1mJH0SWJ1IRWYVrnljG2NHj+LSc05PuxSzQRv0Ijt548gNPptZkZZMljfMmsQp9Z4q2ypPqYvsrOOXazSPAs4A/iaposwq1Sv7DrHh5X1ct2Ba2qWYDUmpYwrvAE4H3gg0Aqsi4unEqjKrUI9tagPweIJVrFJPH10HPABMAeqBL0j648SqMqtQzRuznNZQzwXTTku7FLMhKfVI4Q+AyyPiAICkjwOPAf+YVGFmlSYiaMlkufLcydQVT3xkViFKPVIQUDg5fBe/nPLCzIAt2QPs3HuIxT51ZBWs1COFLwBPSPp6/v71wL8mU5JZZWrJZAGPJ1hlK+lIISI+Bbwf2J3/en9E3D3Q8yRdLWmDpEx+kZ6++v2WpJC0sNTCzcpNcyZLU2MD50z2XJFWuUo9UiAingGeKbW/pDrgHuBtwA7gKUkrI+L5on4TgQ8DT5S6bbNy09UdPLqpjbe/7mwkn1m1yjXUD6+VYhGQiYjNEXGE3FKe1/XS72+AjwOHEqzFLFHrWvey71Ani+f51JFVtiRDoQnYXnB/R76th6RLgBkR8e3+NiTpFklrJK3ZtWvX8FdqdpKOjSdc6amyrcIlGQr9kjSK3BoNHxmob0TcGxELI2Lh1KlTky/ObJCaN2Z57dmnMmXC2LRLMTspSYZCKzCj4P70fNsxE4HXAT+UtBW4HFjpwWarNB1Hunh62x6WzPVRglW+JEPhKWCepNmSxgA3ACuPPRgReyNiSkTMiohZwOPAtRGxJsGazIbdU1t3c6Sr259PsKqQWChERCdwK7kptl8AHoqI9ZLukHRtUq9rNtJaMlnq68Si2ZPSLsXspJV8SepQRMQqYFVR20f76HtVkrWYJaU5k+WSmaczbkyibyezEZHaQLNZNdh94Ajrd77qTzFb1XAomJ2ERzflLkX15xOsWjgUzE5CSybLxLGjuajJU2VbdXAomJ2Elkwbl587mdF1fitZdfD/ZLMherHtIC/uPujxBKsqDgWzIWo5Np7gULAq4lAwG6LmTJazTj2Fc6eOT7sUs2HjUDAbgu7u4NFMlsVzp3iqbKsqDgWzIXj+pVfZc/AoS+Z5viOrLg4FsyE4NlX24nM9nmDVxaFgNgTNmSznnTmBM049Je1SzIaVQ8FsEFasbeXKO7/HIxuztLZ3sGJt68BPMqsgnsHLrEQr1rZy+8Pr6DjaBcCBw13c/vA6AK5f0NTfU80qho8UzEq0fPWGnkA4puNoF8tXb0ipIrPh51AwK9HO9o5BtZtVIoeCWYmmNTYMqt2sEjkUzEq0bOl8xtQd/0G1hvo6li2dn1JFZsPPoWBWousXNPGGWZMQIKCpsYE733mhB5mtqvjqI7NBeGnvIa6aP5UvvH9R2qWYJcJHCmYlam3vYHP2gGdFtarmUDAr0bGpLZZ46U2rYg4FsxK1ZLJMmTCG+WdOTLsUs8Q4FMxKEBG0eKpsqwEOBbMSbHh5H9n9RzyeYFXPoWBWguaNXnrTaoNDwawELZksc6aMp8mfXrYq51AwG8CRzm6e2LLbRwlWExwKZgN4dns7B490ORSsJjgUzAbQnMkySnDFHK/HbNXPoWA2gEczWS6c3shp4+rTLsUscYmGgqSrJW2QlJF0Wy+P/6mk5yU9J+l7ks5Jsh6zwdp36Chrt7ezZK6PEqw2JBYKkuqAe4BrgPOBGyWdX9RtLbAwIi4C/gP4RFL1mA3Fk1t209UdHk+wmpHkkcIiIBMRmyPiCPAgcF1hh4j4QUQczN99HJieYD1mg9acyXJK/SgumXl62qWYjYgkQ6EJ2F5wf0e+rS83A9/p7QFJt0haI2nNrl27hrFEs/61ZLK8YdYkTqmvS7sUsxFRFgPNkn4XWAgs7+3xiLg3IhZGxMKpU6eObHFWs1559RA/f3k/S3zqyGpIkovstAIzCu5Pz7cdR9Jbgf8D/EpEHE6wHrNBadnkqS2s9iR5pPAUME/SbEljgBuAlYUdJC0A/hm4NiJeSbAWs0Fr3tjG6ePqOf/sU9MuxWzEJBYKEdEJ3AqsBl4AHoqI9ZLukHRtvttyYALw75KelbSyj82ZjahjU2VfOXcKo0Z5qmyrHYmu0RwRq4BVRW0fLbj91iRf32yoNu06wC9ePeTxBKs5ZTHQbFZuepbedChYjXEomPWiOZNl5qRxzJg0Lu1SzEaUQ8GsSGdXN49vavNVR1aTHApmRZ5r3cu+w50+dWQ1yaFgVqRlYxYJrjjXk+BZ7XEomBVpzmS5YNqpTBo/Ju1SzEacQ8GswMEjnTzz4h6PJ1jNciiYFXhyy26OdoXHE6xmORTMCrRksowZPYo3zJqUdilmqXAomBVozrSx8JzTPVW21SyHglledv9hXnjpVY8nWE1zKJgBK9a28rZP/QiA+1u2smLtCbO8m9WERCfEM6sEK9a2cvvD6+g42gXArv2Huf3hdQBcv6C/xQLNqo+PFKzmLV+9oScQjuk42sXy1RtSqsgsPQ4Fq3k72zsG1W5WzRwKVvPObjyl1/ZpjQ0jXIlZ+hwKVvOuOm/qCW0N9XUsWzo/hWrM0uWBZqtphzu7+NHPs0xvbKCb4KX2Q0xrbGDZ0vkeZLaa5FCwmvaVJ16ktb2DB25exBvnnXjEYFZrfPrIatb+w5185vsZrjx3suc6MstzKFjN+tdHttB24Ah/dvVrkJR2OWZlwaFgNalt/2Hue2QzV19wFhfPaEy7HLOy4VCwmvTZH27i4JFO/tfS89IuxaysOBSs5rS2d/DAY9t416XTmXvGxLTLMSsrDgWrOXf/189B8OG3+ijBrJhDwWpK5pV9fO2ZHfz+5efQ5E8sm53AoWA15ZOrf864MaP5ozfPTbsUs7LkULCa8ez2dr67/hd84I1zmDR+TNrlmJUlh4LVhIjg49/5GZPHj+HmN85OuxyzsuVQsJrQnMny2OY2bn3LXCaM9ewuZn1xKFjV6+4OPvHdDTQ1NnDTZTPTLsesrCX6J5Okq4FPA3XAv0TEXUWPjwX+DbgUaAPeExFbh7uOFWtbWb56AzvbOwacAbPUvt5m5WyzNb9Yzk2LZjB2dF2v2zSznMRCQVIdcA/wNmAH8JSklRHxfEG3m4E9ETFX0g3Ax4H3DGcdxevvtrZ39Ln+bql9vc3K2ybA19e2smj2ZE+JbdYPRUQyG5auAD4WEUvz928HiIg7C/qszvd5TNJo4BfA1OinqIULF8aaNWtKrmPxXd/v+Uux0OhRYvaU8ce1bckeoLP7xJcu7ltqP2+z/LbZ1NhAy21vOaHdrNpJejoiFg7UL8nTR03A9oL7O4DL+uoTEZ2S9gKTgWxhJ0m3ALcAzJw5uHPCfa2z29kdzDtzwnFtG1/ZX1LfUvt5m+W3Ta+7bNa/irgMIyLuBe6F3JHCYJ47rbGh1yOFpsYGPvs7lx7X1tdRRXHfUvt5m+W3Ta+7bNa/JK8+agVmFNyfnm/rtU/+9NFp5Aach82ypfNpqD9+cLGv9XdL7ettVtc2zeyXkjxSeAqYJ2k2uV/+NwA3FfVZCbwXeAx4F/D9/sYThuLYoGIpV6uU2tfbrK5tmtkvJTbQDCDp7cDd5C5J/XxE/K2kO4A1EbFS0inAA8ACYDdwQ0Rs7m+bgx1oNjOz8hhoJiJWAauK2j5acPsQ8NtJ1mBmZqXzJ5rNzKyHQ8HMzHo4FMzMrIdDwczMeiR69VESJO0Ctg3x6VMo+rR0Fai2faq2/YHq26dq2x+ovn3qbX/OiYipAz2x4kLhZEhaU8olWZWk2vap2vYHqm+fqm1/oPr26WT2x6ePzMysh0PBzMx61Foo3Jt2AQmotn2qtv2B6tunatsfqL59GvL+1NSYgpmZ9a/WjhTMzKwfDgUzM+tRE6Eg6bclrZfULWlhQfssSR2Sns1/fS7NOkvV1/7kH7tdUkbSBklL06rxZEj6mKTWgp/L29OuaSgkXZ3/OWQk3ZZ2PcNB0lZJ6/I/l4qbrljS5yW9IumnBW2TJP2XpI3576enWeNg9bFPQ34P1UQoAD8F3gn8uJfHNkXExfmvPxzhuoaq1/2RdD65dSsuAK4GPiup7sSnV4S/L/i5rBq4e3nJ/7vfA1wDnA/cmP/5VIM3538ulXhd//3k3huFbgO+FxHzgO/l71eS+zlxn2CI76GaCIWIeCEiNqRdx3DpZ3+uAx6MiMMRsQXIAItGtjrLWwRkImJzRBwBHiT387EURcSPya3dUug64Iv5218Erh/Rok5SH/s0ZDURCgOYLWmtpB9JemPaxZykJmB7wf0d+bZKdKuk5/KHxhV1OJ9XTT+LQgH8p6SnJd2SdjHD5MyIeCl/+xfAmWkWM4yG9B6qmlCQ9N+SftrLV39/nb0EzIyIBcCfAl+RdOrIVNy/Ie5PxRhg//4JOBe4mNzP6O9SLdYKLYmIS8idFvuQpDelXdBwyi8HXA3X6Q/5PZToymsjKSLeOoTnHAYO528/LWkTcB6Q+gDaUPaH3FrYMwruT8+3lZ1S90/SfcC3Ei4nCRXzsxiMiGjNf39F0tfJnSbrbayukrws6eyIeEnS2cAraRd0siLi5WO3B/seqpojhaGQNPXYQKykOcA8oN81osvcSuAGSWMlzSa3P0+mXNOg5d+Yx/wmuYH1SvMUME/SbEljyF0AsDLlmk6KpPGSJh67DfwalfmzKbYSeG/+9nuBb6RYy7A4mfdQ1Rwp9EfSbwL/CEwFvi3p2YhYCrwJuEPSUaAb+MOIGLYBm6T0tT8RsV7SQ8DzQCfwoYjoSrPWIfqEpIvJHcZvBT6YbjmDFxGdkm4FVgN1wOcjYn3KZZ2sM4GvS4Lc746vRMR30y1pcCR9FbgKmCJpB/BXwF3AQ5JuJjct/7vTq3Dw+tinq4b6HvI0F2Zm1qOmTx+ZmdnxHApmZta/yCnvAAABT0lEQVTDoWBmZj0cCmZm1sOhYGZmPRwKZv2Q9GgC25wl6abh3q7ZcHAomPUjIq5MYLOzAIeClSWHglk/JO3Pf79K0g8l/Yekn0n6svKf4sqvMfCJ/DoDT0qam2+/X9K7irdF7sNSb8zPc/8/R3qfzPrjUDAr3QLgT8itjzAHWFzw2N6IuBD4DHD3ANu5DXgkP8/93ydSqdkQORTMSvdkROyIiG7gWXKngY75asH3K0a6MLPh4lAwK93hgttdHD93WPRyu5P8e0zSKGBMotWZDQOHgtnweE/B98fyt7cCl+ZvXwvU52/vAyaOWGVmg1ATs6SajYDTJT1H7mjixnzbfcA3JP0E+C5wIN/+HNCVb7/f4wpWTjxLqtlJkrQVWBgR2bRrMTtZPn1kZmY9fKRgZmY9fKRgZmY9HApmZtbDoWBmZj0cCmZm1sOhYGZmPf4/mLRCf+clxvAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cx.plot_f(hard_sigmoid, frange=(-15, 15, 1), xlabel=\"input\", ylabel=\"output\", title=\"hard_sigmoid()\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## linear"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function linear in module conx.activations:\n",
      "\n",
      "linear(x)\n",
      "    Linear activation function.\n",
      "    \n",
      "    >>> linear(1) == 1\n",
      "    True\n",
      "    >>> linear(-1) == -1\n",
      "    True\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(linear)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "linear(0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xt8VPWZx/HPkxCSAOEe7nIVUEFFjajVWqwo2lpBrK113dor2utud6vF2m7txdbVVtutXVustHattlYF2UJF7Vrx1mpQCXcEBCEECJeEAEnI5dk/MuAQZsIkmZkzl+/79corZ845mXkOMfN4zu87v2PujoiISDQ5QRcgIiKpTY1CRETapEYhIiJtUqMQEZE2qVGIiEib1ChERKRNahQiYcxsk5lNNbNvmtmvA6xjmpnNDy3nm9kaMysOqh7JbqbPUYi8x8w2AZ9z9+cCrqMU+LK7/z30+BZgoLv/e5B1SXbSGYVICjGzXDM7G+h1uEmEPALcYGb5AZUmWUyNQiQCM7vdzB4OLY80MzezG8zsXTPbZWa3he2bY2azzWyDme02s8fMrG/Y9j+Z2XYzqzazJWY2IWzbb83sfjNbZGYHgIuAy4EXwutx963AXuDcBB+6yDHUKERidwEwHrgY+A8zOzm0/ivADOADwBBa3tB/EfZzfwHGAgOAN4Dft3re64A7gCLgJeBUYG2E118NnB6PAxFpDzUKkdh9191r3X0ZsIz33rRvAm5z963uXg/cDnzUzLoAuPtcd68J23a6mfUKe96n3P1ld2929zqgN1AT4fVrQttEkqpL0AWIpJHtYcsHgR6h5RHAPDNrDtveBAw0s+20nC1cAxQDh/fpD1SHlre0ep29tJxdtFYEVHW4epEO0hmFSOdtAS53995hXwXuXk7LZaXpwFSgFzAy9DMW9vOto4dlwLgIr3MyLWcyIkmlRiHSeb8E7jCzEQBmVmxm00PbioB6YDfQDfhhDM+3iJbxjiPMbCjQF/h7xJ8QSSA1CpHO+xmwAHjGzGpoeTM/J7Ttd8BmoBxYRQxv9O7+BlBtZueErb4OeCg0ziGSVPrAnUgKMrNLgS+6+4zQZyeWARe6+86AS5MspEYhIiJt0qUnERFpkxqFiIi0SY1CRETaFOgH7sxsLnAFsNPdJ4bW3Q58HqgM7fZNd1/U1vP079/fR44cmcBKRUQyz9KlS3e5+3Gnrw/6k9m/Be6jJUIY7l53/3GsTzJy5EhKS0vjWZeISMYzs82x7BfopSd3XwLsCbIGERFpW6qOUXzZzMrMbK6Z9Ym0g5nNMrNSMyutrKyMtIuIiMRBKjaK+4ExwCSgAvhJpJ3cfY67l7h7SXGx7hApIpIoKdco3H2Huze5ezPwADA56JpERLJZyjUKMxsc9vAqYEVQtYiISPDx2EeBKUB/M9sKfAeYYmaTaJl6eRNwY2AFioikqPlvlnP34rVsq6plSO9Cbp42nhlnDE3IawXaKNz9ExFWP5j0QkRE0sj8N8u59cnl1DY0AVBeVcutTy4HSEizSLlLTyIi0ra7F6890iQOq21o4u7FkW613nlqFCIiaWZbVW271ndW0J/MFhGRGDU1O795+Z2o24f0LkzI66pRiIikgXU7arjl8TLe2lLFKYOL2Fh5gLrG5iPbC/NyuXna+IS8thqFiEgKO9TYzP1/28B9z79NUUEeP7t2EleePoSn3tqWHaknERF5T+vI68fPHsbCsu2s3VHD9ElD+I8rTqFfj3ygJd2UqMbQmhqFiEgKiBR5vefZt+lZ0IVff7KEqacMDKw2pZ5ERFJApMgrQPf8LoE2CVCjEBFJCdGirdur65JcybHUKEREAvbcqh2YRd6WqMhre2iMQkQkILv31/Pd/13FgmXbGNwznz0HG6hPUuS1PdQoRESSzN1ZsGwbty9Yyf76Rr42dRxfmDKGRcsrkhZ5bQ81ChGRBAuPvQ7smU/f7l1ZVVHDpBN6c9dHT2PcwCIguZHX9lCjEBFJoNax1+376tm+r54Zk4bwk49NIjcnyuBECtFgtohIAkWLvb6+aW9aNAlQoxARSZjGpmbKkzzTayKoUYiIJMDqin3MvP+VqNtTIfYaKzUKEZE4qm9s4p5n1/GRn79E+d5abjhvBIV5R7/VpkrsNVYazBYRiZM33t3LNx4v4+2d+5l5xlC+fcUp9OnelTOG90nJ2Gus1ChERDogPPI6qFcB4wb2YMnbuxjcs4DffPpsLho/4Mi+qRp7jZUahYhIO7WOvFZU11FRXcf5Y/rxy38+i6KCvIArjC+NUYiItFO0yOum3QczrkmAGoWISLtlQuS1PdQoRERiVFlTz5ceeSPq9nSKvLaHGoWIyHG4O0++sZVL7n2BZ1fu4EMTB1GQ5pHX9gh0MNvM5gJXADvdfWJoXV/gj8BIYBPwMXffG1SNIpLdyqtquW3ecv62tpIzh7dM4nfigKJj7m+dbpHX9jB3D+7FzS4E9gO/C2sUdwF73P1OM5sN9HH3b7T1PCUlJV5aWpr4gkUk44U3gJ6FedQeaiQ3J4dbLhvPJ88bmTbzM8XCzJa6e8nx9gv00pO7LwH2tFo9HXgotPwQMCOpRYlI1jocey2vqsWB6toGGpudr08bx6fPH5VRTaI9UnGMYqC7V4SWtwPB3lVcRLLGXYvXHBN7bXaY+9KmYApKEanYKI7wlutiEa+NmdksMys1s9LKysokVyYimWbVtn1sq6qLuC1TY6+xSsVGscPMBgOEvu+MtJO7z3H3EncvKS4uTmqBIpI56hqa+PHitVx530tEu7KUqbHXWKVio1gA3BBavgF4KsBaRCSDLd28hw//14vc9/x6pk8ayg9mTKQwL/eofTI59hqroOOxjwJTgP5mthX4DnAn8JiZfRbYDHwsuApFJBO0jrJ+9YMnsnp7DQ+9uokhvQp56DOT+cC4lisT3bp2yZrYa6wCjcfGi+KxIhJN6wn8AIyWwc8bzhvBzZedRI/87JwfNdZ4bHb+64hI1og0gZ8D/Xt05bvTJwZTVJpJxTEKEZG4iZZY2r3/UJIrSV9qFCKSsXbW1JGfF/ltLtuTTO2hRiEiGcfdeXzpVi65ZwkNjc10aZV7VZKpfTRGISIZZeveg3xz3gqWrKukZEQf7rz6NFaUVyvJ1AlqFCKStsJjr4N7F3Du6H48vWI7Bnxv+gSuP2cEOTnGiQN6qDF0ghqFiKSl1rHXbVV1PPlGOeMHFfHgDSUM69Mt4Aozh8YoRCQtRbtv9f66BjWJOFOjEJG0FC32Gm1iP+k4XXoSkbRS19DEz/76duRppVHsNRHUKEQkbby+aQ/feLyMjbsOcM6oPizbWk1dQ/OR7Yq9JoYahYikvP31jdz19Bp+9+pmhvUp5OHPnsMFY/tn1X2rg6RGISIppfWb/4dPG8TCsu1sq67l0+eP5OuXjqd7aBK/GWcMVWNIAjUKEUkZrSOv5VW1zFnyDgOK8nn8pvdx1og+AVeYnZR6EpGUES3y2iXH1CQCpEYhIikjWuS1olqR1yDp0pOIBM7d+VPp1qjbFXkNlhqFiARqy56D3Prkcl5av4vR/btTXlVLfaMir6lEjUJEAtHU7Dz0yibuXryW3BzjBzMmct3k4SxYtk2R1xSjRiEiSREeey0uyqcwL4fNe2q5aHwxd1x16pHLS4q8ph41ChFJuNax15019QBcf+5wvj99ImbW1o9LwJR6EpGEixZ7fX5NpZpEGlCjEJGEqmtoojzqTK+R10tqUaMQkYT5+8bdXPbTJVG3K/aaHtQoRCTuauoauG3ecq6d83eaHb44ZQyFeblH7aPYa/pI2cFsM9sE1ABNQKO7lwRbkYjE4v/W7OC2eSvYsa+Oz10win+7dBzdunZh3MAixV7TVMo2ipCL3H1X0EWISGThkddBvQoY0quApe9WMXZAD/77C+/jjOHvzc+k2Gv6SvVGISIpqnXktaK6jorqOqadMpD/uu4M8rvkHucZJF2k8hiFA8+Y2VIzmxV0MSJytGiR1xXb9qlJZJhUPqO4wN3LzWwA8KyZrXH3I/GJUPOYBTB8+PCgahTJSu6uyGsWSdkzCncvD33fCcwDJrfaPsfdS9y9pLi4OIgSRbLS5t0HuO6Bf0Tdrshr5knJRmFm3c2s6PAycCmwItiqRLJbU7Pz6xc3Mu2nS1hRXs01JcMozDv6LUSR18yUqpeeBgLzQh/t7wI84u5PB1uSSPZofd/q688dztMrd7BsSxUXnzSAH1w1kcG9Cjl/TH9FXrOAuXvQNXRaSUmJl5aWBl2GSEZonWY6rHvXXH4481SuPH2I5mfKEGa2NJbPqKXqGYWIBCRamqmoMI/pk3S2kI1ScoxCRIITLbW0Q/etzlpqFCJyxCsbdpGTE/myktJM2UuXnkSEfXUN/GjRGh597V36dc+jpr6JQ7pvtYSoUYhkuedW7eC2+cuprKln1oWj+drUcSxeuV1pJjlCjUIkS7SOvH5hymhee2cvC5Zt46RBRcz55xJOP6E3oAn85GhqFCJZoHXktbyqlm/NX0mOwb9dMo6bPjCGrl00ZCmRqVGIZIFokdf+PfL56sVjA6hI0on+F0IkC0SLvFbW1Ce5EklHahQiGe6dXQfIy438p67Iq8RCl55EMlRjUzNzX36HnzyzDjMnL9doaHpvyh5FXiVWahQiGWh1xT6+8UQZZVurueSUgfxgxkRe3bBbkVfpEDUKkTQXHnsd3KuAU4f25K9rKundLY9fXHcmHzp1EGamyKt0mBqFSBprHXvdVl3Htuo6Skb05oFPnk2f7l0DrlAygQazRdJYtNhrRXW9moTEjRqFSBrTfaslGXTpSSQNVdc28MOFq6NuV+xV4kmNQiTNLF65nW/PX8HuA4e4+OQBvLx+F3UNmulVEiemS09m9i+xrBORxKmsqedLv3+DG/9nKf165DP/i+fz4A1nc+fM0xjauxADhvYu5EczT1W6SeIqpntmm9kb7n5mq3VvuvsZCausHXTPbMk0R8/0WsCU8cUsXL6dg/VN/MvUscy6cHTUT1uLxCou98w2s08A1wGjzGxB2KYiYE/nShSRSI6d6bWO3/9jCyP6FvL4Tedx4oCigCuUbHO8MYpXgAqgP/CTsPU1QFmiihLJZtEirw3NriYhgWizUbj7ZmAzcF5yyhGRaNHWiqq6JFci0iKm1JOZ1QCHBzO6AnnAAXfvmajCRLJNY1Mzc17cSLRRQ0VeJSgxNQp3P3K+a2YGTAfOTVRRItlm5bZqvvFEGSvK93Ha0J6s27GfukZFXiU1tDs24S3mA9MSUI9IVqlraOLuxWu48r6X2V5dz/3/dCYLvvJ+7rxakVdJHbFeepoZ9jAHKAESesHUzC4DfgbkAr929zsT+XoiyRAee+3Xoys5BjtrDnH1mcP49hUn07tby/xMmulVUkmsn8z+SNhyI7CJlstPCWFmucAvgEuArcDrZrbA3Vcl6jVFEq117HXX/kMYcOMHRnPr5ScHW5xIG2Ido/h0ogtpZTKw3t03ApjZH2hpTGoUkrYixV4d+POyCjUKSWmxTuEx2sz+18wqzWynmT1lZqMTWNdQYEvY462hdeE1zTKzUjMrraysTGApIp1XdfCQZnqVtBXrYPYjwGPAYGAI8Cfg0UQVFQt3n+PuJe5eUlxcHGQpIm36y/IKpt6zJOp2xV4l1cXaKLq5+/+4e2Po62GgIIF1lQMnhD0eFlonkjZ21tTxhYeX8oXfv8HAnvn8+6XjKMzLPWofxV4lHcQ6mP0XM5sN/IGWy6ofBxaZWV8Ad4/3vE+vA2PNbBQtDeJaWuacEkk5R0/gV8jXLx1HY7Pzg4WrqW1o4pbLxvP597dM4ndCn25H7XvztPFKN0nKi3X22Hfa2OzuHvfxCjP7EPBTWuKxc939jmj7avZYCUrrJBNAjkGzw9kj+3Dn1acxprhHgBWKRBeX2WPDnOzuR31uwswKWq+LJ3dfBCxK1POLxEOkJFOzQ6/CPP446zxyciygykTiJ9YxildiXCeSVaIllvbVNqhJSMY43v0oBtESSy00szOAw//l9wS6Jbg2kZTW0NRMUUEX9tU1HrNNSSbJJMe79DQN+BQtqaN7wtbXAN9MUE0iKW9FeTW3PF7GvrpGcg2awob6lGSSTHO8+1E8BDxkZle7+xNJqkkkZdU1NPGzv77NnCUb6du9K7+8/qzQxH5KMknminUwe6KZTWi90t2/F+d6RFJG69jrzDOHsrCsgo27DvCxkmHc9qFT6NUtD0CNQTJarI1if9hyAXAFsDr+5YikhmPvW13Lz/9vPX265fHwZ8/hgrH9A65QJHlinRQw/H7ZmNmPgcUJqUgkBUS7b3VBXq6ahGSddt+4KKQbLQPcIhkpWux1e7XuWy3ZJ9YbFy3nvXtm5wADgO8nqiiRoLg7i5ZvxwwiTVqg2Ktko1jHKK4A+gDvB3oDi9x9acKqEgnAzn11fGv+Cp5ZtYNhvQup3F9Pve5bLRLzpafpwP8A/YE84Ddm9pWEVSWSRO7OY69v4eJ7XuCFdZXcevlJ/O3mKfyn7lstAsQ+KWAZcJ67Hwg97g686u6nJbi+mGhSQIlV68jrZ84fyfNrK3lp/S4mj+rLnTNPZbQm8ZMsEe9JAQ0Ij4A08d50HiJpIVLk9fsLV5Ofa/xgxkSumzxc8zOJRBBro/gN8A8zmxd6PAN4MDEliSRGtMhr7+5duf7cEQFUJJIeYv0cxT1m9jfggtCqT7v7mwmrSiQBokVed+6rT3IlIukl1jMK3P0N4I0E1iKSMGVbq8jNMRqbjx2TU+RVpG0xNwqRdFTX0MS9z67jgRc30iM/l7oG51CTIq8i7aFGIRnr7xt3M/uJMjbtPsgnJp/A7MtP5vk1OzXTq0g7qVFIRgiPvQ7qVcCo/t14ZcMehvftxiOfO4f3ndgyP9OMM4aqMYi0kxqFpL3WsdeK6joqquuYMq4/919fQmHX3IArFElvHZ0UUCRlRIu9vr3zgJqESByoUUhac3fKo8Reo8VhRaR91CgkbW2vruPzv4s+N6ViryLxoUYhacfdefS1d7nknhd4aX0l008fQmHe0f8pK/YqEj8azJa0snn3AWY/sZxXN+7m3NF9uXPmaYzs3/2Yyf4UexWJn5RrFGZ2O/B5oDK06pvuvii4iiQo4W/+g3sXMHlkX55euZ28nBx+NPNUrj37BMxaJvFT7FUkcVKuUYTc6+4/DroICU7ryOu2qjrmv7WNCUOKePCGyQzqVRBwhSLZQ2MUkpKiRV6rDjaoSYgkWao2ii+bWZmZzTWzPpF2MLNZZlZqZqWVlZWRdpE0Fi3auq2qLsmViEggjcLMnjOzFRG+pgP3A2OASUAF8JNIz+Huc9y9xN1LiouLk1i9JFLtoSbuWLiKaPddVORVJPkCGaNw96mx7GdmDwB/TnA5kiJe2bCL2U8s5909BzlvTF/efLeKugbN9CoStJQbzDazwe5eEXp4FbAiyHok8fbVNfCjRWt49LV3GdGvG49+/lzOG9NPkVeRFJFyjQK4y8wmAQ5sAm4MthyJt/AG0Kd7Vxqbmthf38SNF47mX6eOOzI/kyKvIqkh5RqFu/9z0DVI4rSOve45cAgDvnbJWL568bhgixORiFI19SQZ6u7Fa46JvTrwx9e3BlOQiByXGoUkzbaqWsqjxFs106tI6kq5S0+SeZqbnUdff5cfLVqDQcToq2KvIqlLjUIS6p1dB5j9RBn/eGcP55/Yjw+OH8CPn1l31OUnxV5FUpsahcRN60n8Sob3YfGqHXTtksNdV5/GNSXDMDP69chX7FUkjZh7tM/Apo+SkhIvLS0Nuoys1jrNdNjEIT158FNnM7Cn5mcSSTVmttTdS463nwazJS6iTeK39+AhNQmRNKdGIXGhSfxEMpfGKKRTDh5q5MeL12kSP5EMpkYhHfby+l3MfrKMLXtqueDEfpRu3qtJ/EQykBqFtFt1bQM/XLiaP5ZuYVT/7jx243lMHtVXk/iJZCg1CmlT6zf/aRMG8ueyCnYfOMQXpozhXy4eS0GeJvETyWRqFBJV68hreVUtc1/exJBeBcz/4vmcOqxXwBWKSDIo9SRRRYu8YqhJiGQRNQqJKlrktUKRV5GsoktPcozmZufhf2yOul2RV5HsokYhR9lQuZ/ZT5Tx+qa9jB/Yg027D1LfqMirSDZToxAAGpuamfPiRn763NsUdMnh7o+exkfPGsZTb21T5FUky6lRZKnw2Gv/onzyc42tVXVcNmEQ35sxgQFFLfMzKfIqImoUWah17LWyph6AT71vBLdfOTHI0kQkBSn1lIWixV6fXbUzgGpEJNWpUWSZA/WNlEed6VX3rRaRY6lRZJEl6yq59N4lUbcr9ioikahRZIGqg4f4+p+W8cm5r5Gfl8NXLz6RwtD8TIcp9ioi0WgwO8P9ZXkF335qJXsPHuJLF43hKx9smcRvdP8eir2KSEwCaRRmdg1wO3AyMNndS8O23Qp8FmgCvurui4OoMR2FR14H9ixgQM+ulG3dx4QhPXnoM2czYch78zMp9ioisQrqjGIFMBP4VfhKMzsFuBaYAAwBnjOzce4eYWY6Cdc68rp9Xx3b99VxxWmD+enHJ9ElV1cZRaRjAnn3cPfV7r42wqbpwB/cvd7d3wHWA5OTW116ihZ5ffPdKjUJEemUVHsHGQpsCXu8NbTuGGY2y8xKzay0srIyKcWlquZmV+RVRBImYY3CzJ4zsxURvqbH4/ndfY67l7h7SXFxcTyeMi2t31nDNb96Nep2RV5FpLMSNkbh7lM78GPlwAlhj4eF1kkrDU3N/OqFDfzXX9fTLT+XfzrnBJ58o5zaBs30KiLxlWrx2AXAI2Z2Dy2D2WOB14ItKfWsKK/m5sfLWF2xjw+fOpjbr5xAcVE+Z4/sp8iriMRdUPHYq4CfA8XAQjN7y92nuftKM3sMWAU0Al9S4uno2Gv3/C4cqG+kf1E+v7z+LC6bOOjIfoq8ikgiBNIo3H0eMC/KtjuAO5JbUepqHXvdX99Ibo7xtaljj2oSIiKJkmqpJ2nlP59ec0zstanZ+cXzGwKqSESyjRpFCnt+7U4qqusiblPsVUSSJdUGswXYe+AQ3//zKp58s5wuOUZjsx+zj2KvIpIsahQpxN1ZuLyC7zy1kuraBr76wRMZ3rcb335q5VGXnxR7FZFkUqMISHiSaUjvQmZdOIqX1+/mmVU7OHVoLx7+3DmcPLgnAF1ycxR7FZHAmPuxlzXSTUlJiZeWlh5/xxTROsl0WK7BNy4/ic+cP0rzM4lIwpnZUncvOd5+OqMIQLQJ/PoX5TPrwjEBVCQiEp3+tzUA0RJLO/fVJ7kSEZHjU6NIsrd31NAl1yJuU5JJRFKRLj0lyaHGZn75wgbu+7/15OW0NIqGpvfGh5RkEpFUpUaRBGVbq7jl8TLWbK/hI6cP4faPnMKLb+9SkklE0oIaRZyFx14H9yrgpEFF/G1dJcVF+TzwyRIuOWUgoAn8RCR9qFHEUevY67bqOrZV13He6L786pMl9CzIC7hCEZH202B2HEWLvb67p1ZNQkTSlhpFHOm+1SKSiXTpKQ72HDjE9/53ZdTtir2KSDrTGUUnuDsLlm1j6j0vsHB5BZdNGEhB3tH/pIq9iki60xlFB22vruNb85fz3OqdnH5Cb+66+jTGDyo6ZrI/xV5FJN2pUcTg6Df/As4/sT9/Wb6dhuZmvvXhk/n0+aPIDX2ITrFXEck0ahTH0TryWl5Vx2OlWzmxuDsPfupsRvTrHnCFIiKJpTGK44gWea1taFKTEJGsoEZxHNGirduqIt/LWkQk0+jSUxSHGpv5xfPriXZbJ0VeRSRbqFFE8NaWKm55fBnrduznrOG9WVmxj7qG5iPbFXkVkWyiRhHm4KFG7nlmHXNffoeBPQuY+6kSPnjSQEVeRSSrBdIozOwa4HbgZGCyu5eG1o8EVgNrQ7v+3d1vSlQd4Q2gX4+uNLuz50AD/3TOcGZffhJFofmZFHkVkWwW1BnFCmAm8KsI2za4+6REF9A69rpr/yEM+PJFY/j6tJMS/fIiImkjkNSTu69297XH3zNxIsVeHZj35rZgChIRSVGpGI8dZWZvmtkLZvb+aDuZ2SwzKzWz0srKyna/SPTYq2Z6FREJl7BLT2b2HDAowqbb3P2pKD9WAQx3991mdhYw38wmuPu+1ju6+xxgDkBJSUm0FGtUQ3oXRpwWXLFXEZGjJaxRuPvUDvxMPVAfWl5qZhuAcUBpnMvj5mnjjxqjAMVeRUQiSal4rJkVA3vcvcnMRgNjgY2JeK3DKSbFXkVE2hZUPPYq4OdAMbDQzN5y92nAhcD3zKwBaAZucvc9iapDsVcRkeMLpFG4+zxgXoT1TwBPJL8iERGJJhVTTyIikkLUKEREpE1qFCIi0iY1ChERaZO5t/uzainHzCqBzZ14iv7ArjiVkwp0PKkv044p044HMu+YIh3PCHcvPt4PZkSj6CwzK3X3kqDriBcdT+rLtGPKtOOBzDumzhyPLj2JiEib1ChERKRNahQt5gRdQJzpeFJfph1Tph0PZN4xdfh4NEYhIiJt0hmFiIi0SY1CRETalLWNwsyuMbOVZtZsZiVh60eaWa2ZvRX6+mWQdbZHtGMKbbvVzNab2VozmxZUjR1lZrebWXnY7+VDQdfUEWZ2Weh3sN7MZgddTzyY2SYzWx76vcT93jHJYGZzzWynma0IW9fXzJ41s7dD3/sEWWN7RDmeDv8NZW2jAFYAM4ElEbZtcPdJoa+bklxXZ0Q8JjM7BbgWmABcBvy3meUmv7xOuzfs97Io6GLaK/Rv/gvgcuAU4BOh300muCj0e0nXzx38lpa/jXCzgb+6+1jgr6HH6eK3HHs80MG/oaxtFO6+2t3XBl1HPLVxTNOBP7h7vbu/A6wHJie3OqHl33y9u29090PAH2j53UjA3H0J0PreN9OBh0LLDwEzklpUJ0Q5ng7L2kZxHKPM7E0ze8HM3h90MXEwFNgS9nhraF26+bKZlYVOq9PmMkCYTPk9tObAM2a21MxmBV1MHA1094rQ8nZgYJDFxEmH/oYyulGY2XNmtiLCV1v/F1cBDHf3M4B/Ax4xs57Jqfj4OnhMaeE4x3Y/MAZdj1mzAAAC1UlEQVSYRMvv6CeBFivhLnD3M2m5pPYlM7sw6ILizVs+R5DunyXo8N9QSt0zO97cfWoHfqYeqA8tLzWzDcA4ICUG6TpyTEA5cELY42GhdSkl1mMzsweAPye4nERIi99De7l7eej7TjObR8sltkhjf+lmh5kNdvcKMxsM7Ay6oM5w9x2Hl9v7N5TRZxQdYWbFhwd6zWw0MBbYGGxVnbYAuNbM8s1sFC3H9FrANbVL6A/1sKtoGbhPN68DY81slJl1pSVgsCDgmjrFzLqbWdHhZeBS0vN3E8kC4IbQ8g3AUwHW0mmd+RvK6DOKtpjZVcDPgWJgoZm95e7TgAuB75lZA9AM3OTucRsUSqRox+TuK83sMWAV0Ah8yd2bgqy1A+4ys0m0nP5vAm4Mtpz2c/dGM/sysBjIBea6+8qAy+qsgcA8M4OW95NH3P3pYEtqPzN7FJgC9DezrcB3gDuBx8zss7TcxuBjwVXYPlGOZ0pH/4Y0hYeIiLRJl55ERKRNahQiItImNQoREWmTGoWIiLRJjUJERNqkRiHSTmb2SgKec6SZXRfv5xWJBzUKkXZy9/cl4GlHAmoUkpLUKETaycz2h75PMbO/mdnjZrbGzH5voU+ehe7RcFfoPg2vmdmJofW/NbOPtn4uWj7c9f7QfQK+luxjEmmLGoVI55wB/Cst95cYDZwftq3a3U8F7gN+epznmQ28GLpPwL0JqVSkg9QoRDrnNXff6u7NwFu0XEI67NGw7+cluzCReFGjEOmc+rDlJo6eP80jLDcS+rszsxyga0KrE4kDNQqRxPl42PdXQ8ubgLNCy1cCeaHlGqAoaZWJtEPWzh4rkgR9zKyMlrOOT4TWPQA8ZWbLgKeBA6H1ZUBTaP1vNU4hqUSzx4okgJltAkrcfVfQtYh0li49iYhIm3RGISIibdIZhYiItEmNQkRE2qRGISIibVKjEBGRNqlRiIhIm/4fKOfoAJ54JxQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cx.plot_f(linear, frange=(-15, 15, 1), xlabel=\"input\", ylabel=\"output\", title=\"linear()\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comparison"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "functions = [hard_sigmoid, sigmoid, tanh, relu, \n",
    "             softsign, softplus, elu, selu, linear]\n",
    "float_range = cx.frange(-2, 2, 0.1)\n",
    "lines = {}\n",
    "symbols = {}\n",
    "for v in float_range:\n",
    "    for f in functions:\n",
    "        if f.__name__ not in lines:\n",
    "            lines[f.__name__] = []\n",
    "            symbols[f.__name__] = \"-\"\n",
    "        lines[f.__name__].append(f(v))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg height=\"432pt\" version=\"1.1\" viewBox=\"0 0 576 432\" width=\"576pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 432  L 576 432  L 576 0  L 0 0  z \" style=\"fill:#ffffff;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 72 378  L 518.4 378  L 518.4 51.84  L 72 51.84  z \" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0  L 0 3.5  \" id=\"m8421d756a3\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"92.290909\" xlink:href=\"#m8421d756a3\" y=\"378\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- −2.0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.59375 35.5  L 73.1875 35.5  L 73.1875 27.203125  L 10.59375 27.203125  z \" id=\"DejaVuSans-2212\"/>\n",
       "       <path d=\"M 19.1875 8.296875  L 53.609375 8.296875  L 53.609375 0  L 7.328125 0  L 7.328125 8.296875  Q 12.9375 14.109375 22.625 23.890625  Q 32.328125 33.6875 34.8125 36.53125  Q 39.546875 41.84375 41.421875 45.53125  Q 43.3125 49.21875 43.3125 52.78125  Q 43.3125 58.59375 39.234375 62.25  Q 35.15625 65.921875 28.609375 65.921875  Q 23.96875 65.921875 18.8125 64.3125  Q 13.671875 62.703125 7.8125 59.421875  L 7.8125 69.390625  Q 13.765625 71.78125 18.9375 73  Q 24.125 74.21875 28.421875 74.21875  Q 39.75 74.21875 46.484375 68.546875  Q 53.21875 62.890625 53.21875 53.421875  Q 53.21875 48.921875 51.53125 44.890625  Q 49.859375 40.875 45.40625 35.40625  Q 44.1875 33.984375 37.640625 27.21875  Q 31.109375 20.453125 19.1875 8.296875  z \" id=\"DejaVuSans-32\"/>\n",
       "       <path d=\"M 10.6875 12.40625  L 21 12.40625  L 21 0  L 10.6875 0  z \" id=\"DejaVuSans-2e\"/>\n",
       "       <path d=\"M 31.78125 66.40625  Q 24.171875 66.40625 20.328125 58.90625  Q 16.5 51.421875 16.5 36.375  Q 16.5 21.390625 20.328125 13.890625  Q 24.171875 6.390625 31.78125 6.390625  Q 39.453125 6.390625 43.28125 13.890625  Q 47.125 21.390625 47.125 36.375  Q 47.125 51.421875 43.28125 58.90625  Q 39.453125 66.40625 31.78125 66.40625  z M 31.78125 74.21875  Q 44.046875 74.21875 50.515625 64.515625  Q 56.984375 54.828125 56.984375 36.375  Q 56.984375 17.96875 50.515625 8.265625  Q 44.046875 -1.421875 31.78125 -1.421875  Q 19.53125 -1.421875 13.0625 8.265625  Q 6.59375 17.96875 6.59375 36.375  Q 6.59375 54.828125 13.0625 64.515625  Q 19.53125 74.21875 31.78125 74.21875  z \" id=\"DejaVuSans-30\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(80.149503 392.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"144.318881\" xlink:href=\"#m8421d756a3\" y=\"378\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- −1.5 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875  L 28.515625 8.296875  L 28.515625 63.921875  L 10.984375 60.40625  L 10.984375 69.390625  L 28.421875 72.90625  L 38.28125 72.90625  L 38.28125 8.296875  L 54.390625 8.296875  L 54.390625 0  L 12.40625 0  z \" id=\"DejaVuSans-31\"/>\n",
       "       <path d=\"M 10.796875 72.90625  L 49.515625 72.90625  L 49.515625 64.59375  L 19.828125 64.59375  L 19.828125 46.734375  Q 21.96875 47.46875 24.109375 47.828125  Q 26.265625 48.1875 28.421875 48.1875  Q 40.625 48.1875 47.75 41.5  Q 54.890625 34.8125 54.890625 23.390625  Q 54.890625 11.625 47.5625 5.09375  Q 40.234375 -1.421875 26.90625 -1.421875  Q 22.3125 -1.421875 17.546875 -0.640625  Q 12.796875 0.140625 7.71875 1.703125  L 7.71875 11.625  Q 12.109375 9.234375 16.796875 8.0625  Q 21.484375 6.890625 26.703125 6.890625  Q 35.15625 6.890625 40.078125 11.328125  Q 45.015625 15.765625 45.015625 23.390625  Q 45.015625 31 40.078125 35.4375  Q 35.15625 39.890625 26.703125 39.890625  Q 22.75 39.890625 18.8125 39.015625  Q 14.890625 38.140625 10.796875 36.28125  z \" id=\"DejaVuSans-35\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(132.177475 392.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"196.346853\" xlink:href=\"#m8421d756a3\" y=\"378\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- −1.0 -->\n",
       "      <g transform=\"translate(184.205447 392.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"248.374825\" xlink:href=\"#m8421d756a3\" y=\"378\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- −0.5 -->\n",
       "      <g transform=\"translate(236.233419 392.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"300.402797\" xlink:href=\"#m8421d756a3\" y=\"378\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 0.0 -->\n",
       "      <g transform=\"translate(292.451235 392.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"352.430769\" xlink:href=\"#m8421d756a3\" y=\"378\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(344.479207 392.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_7\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"404.458741\" xlink:href=\"#m8421d756a3\" y=\"378\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(396.507179 392.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_8\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"456.486713\" xlink:href=\"#m8421d756a3\" y=\"378\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(448.535151 392.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_9\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"508.514685\" xlink:href=\"#m8421d756a3\" y=\"378\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 2.0 -->\n",
       "      <g transform=\"translate(500.563123 392.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_10\">\n",
       "     <!-- input -->\n",
       "     <defs>\n",
       "      <path d=\"M 9.421875 54.6875  L 18.40625 54.6875  L 18.40625 0  L 9.421875 0  z M 9.421875 75.984375  L 18.40625 75.984375  L 18.40625 64.59375  L 9.421875 64.59375  z \" id=\"DejaVuSans-69\"/>\n",
       "      <path d=\"M 54.890625 33.015625  L 54.890625 0  L 45.90625 0  L 45.90625 32.71875  Q 45.90625 40.484375 42.875 44.328125  Q 39.84375 48.1875 33.796875 48.1875  Q 26.515625 48.1875 22.3125 43.546875  Q 18.109375 38.921875 18.109375 30.90625  L 18.109375 0  L 9.078125 0  L 9.078125 54.6875  L 18.109375 54.6875  L 18.109375 46.1875  Q 21.34375 51.125 25.703125 53.5625  Q 30.078125 56 35.796875 56  Q 45.21875 56 50.046875 50.171875  Q 54.890625 44.34375 54.890625 33.015625  z \" id=\"DejaVuSans-6e\"/>\n",
       "      <path d=\"M 18.109375 8.203125  L 18.109375 -20.796875  L 9.078125 -20.796875  L 9.078125 54.6875  L 18.109375 54.6875  L 18.109375 46.390625  Q 20.953125 51.265625 25.265625 53.625  Q 29.59375 56 35.59375 56  Q 45.5625 56 51.78125 48.09375  Q 58.015625 40.1875 58.015625 27.296875  Q 58.015625 14.40625 51.78125 6.484375  Q 45.5625 -1.421875 35.59375 -1.421875  Q 29.59375 -1.421875 25.265625 0.953125  Q 20.953125 3.328125 18.109375 8.203125  z M 48.6875 27.296875  Q 48.6875 37.203125 44.609375 42.84375  Q 40.53125 48.484375 33.40625 48.484375  Q 26.265625 48.484375 22.1875 42.84375  Q 18.109375 37.203125 18.109375 27.296875  Q 18.109375 17.390625 22.1875 11.75  Q 26.265625 6.109375 33.40625 6.109375  Q 40.53125 6.109375 44.609375 11.75  Q 48.6875 17.390625 48.6875 27.296875  z \" id=\"DejaVuSans-70\"/>\n",
       "      <path d=\"M 8.5 21.578125  L 8.5 54.6875  L 17.484375 54.6875  L 17.484375 21.921875  Q 17.484375 14.15625 20.5 10.265625  Q 23.53125 6.390625 29.59375 6.390625  Q 36.859375 6.390625 41.078125 11.03125  Q 45.3125 15.671875 45.3125 23.6875  L 45.3125 54.6875  L 54.296875 54.6875  L 54.296875 0  L 45.3125 0  L 45.3125 8.40625  Q 42.046875 3.421875 37.71875 1  Q 33.40625 -1.421875 27.6875 -1.421875  Q 18.265625 -1.421875 13.375 4.4375  Q 8.5 10.296875 8.5 21.578125  z M 31.109375 56  z \" id=\"DejaVuSans-75\"/>\n",
       "      <path d=\"M 18.3125 70.21875  L 18.3125 54.6875  L 36.8125 54.6875  L 36.8125 47.703125  L 18.3125 47.703125  L 18.3125 18.015625  Q 18.3125 11.328125 20.140625 9.421875  Q 21.96875 7.515625 27.59375 7.515625  L 36.8125 7.515625  L 36.8125 0  L 27.59375 0  Q 17.1875 0 13.234375 3.875  Q 9.28125 7.765625 9.28125 18.015625  L 9.28125 47.703125  L 2.6875 47.703125  L 2.6875 54.6875  L 9.28125 54.6875  L 9.28125 70.21875  z \" id=\"DejaVuSans-74\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(282.339063 406.276563)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"27.783203\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "      <use x=\"91.162109\" xlink:href=\"#DejaVuSans-70\"/>\n",
       "      <use x=\"154.638672\" xlink:href=\"#DejaVuSans-75\"/>\n",
       "      <use x=\"218.017578\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0  L -3.5 0  \" id=\"m3b1b642bf8\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"72\" xlink:href=\"#m3b1b642bf8\" y=\"363.174545\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- −2.0 -->\n",
       "      <g transform=\"translate(40.717187 366.973764)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"72\" xlink:href=\"#m3b1b642bf8\" y=\"326.472305\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- −1.5 -->\n",
       "      <g transform=\"translate(40.717187 330.271523)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"72\" xlink:href=\"#m3b1b642bf8\" y=\"289.770064\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- −1.0 -->\n",
       "      <g transform=\"translate(40.717187 293.569282)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"72\" xlink:href=\"#m3b1b642bf8\" y=\"253.067823\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_14\">\n",
       "      <!-- −0.5 -->\n",
       "      <g transform=\"translate(40.717187 256.867042)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"72\" xlink:href=\"#m3b1b642bf8\" y=\"216.365582\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_15\">\n",
       "      <!-- 0.0 -->\n",
       "      <g transform=\"translate(49.096875 220.164801)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"72\" xlink:href=\"#m3b1b642bf8\" y=\"179.663341\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_16\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(49.096875 183.46256)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"72\" xlink:href=\"#m3b1b642bf8\" y=\"142.9611\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_17\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(49.096875 146.760319)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_8\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"72\" xlink:href=\"#m3b1b642bf8\" y=\"106.258859\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_18\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(49.096875 110.058078)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_9\">\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"72\" xlink:href=\"#m3b1b642bf8\" y=\"69.556618\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_19\">\n",
       "      <!-- 2.0 -->\n",
       "      <g transform=\"translate(49.096875 73.355837)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_20\">\n",
       "     <!-- output -->\n",
       "     <defs>\n",
       "      <path d=\"M 30.609375 48.390625  Q 23.390625 48.390625 19.1875 42.75  Q 14.984375 37.109375 14.984375 27.296875  Q 14.984375 17.484375 19.15625 11.84375  Q 23.34375 6.203125 30.609375 6.203125  Q 37.796875 6.203125 41.984375 11.859375  Q 46.1875 17.53125 46.1875 27.296875  Q 46.1875 37.015625 41.984375 42.703125  Q 37.796875 48.390625 30.609375 48.390625  z M 30.609375 56  Q 42.328125 56 49.015625 48.375  Q 55.71875 40.765625 55.71875 27.296875  Q 55.71875 13.875 49.015625 6.21875  Q 42.328125 -1.421875 30.609375 -1.421875  Q 18.84375 -1.421875 12.171875 6.21875  Q 5.515625 13.875 5.515625 27.296875  Q 5.515625 40.765625 12.171875 48.375  Q 18.84375 56 30.609375 56  z \" id=\"DejaVuSans-6f\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(34.6375 231.411406)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"61.181641\" xlink:href=\"#DejaVuSans-75\"/>\n",
       "      <use x=\"124.560547\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"163.769531\" xlink:href=\"#DejaVuSans-70\"/>\n",
       "      <use x=\"227.246094\" xlink:href=\"#DejaVuSans-75\"/>\n",
       "      <use x=\"290.625\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_19\">\n",
       "    <path clip-path=\"url(#p386362bc70)\" d=\"M 92.290909 209.025134  L 102.696503 207.557044  L 113.102098 206.088953  L 123.507692 204.620865  L 133.913287 203.152777  L 144.318881 201.684686  L 154.724476 200.216596  L 165.13007 198.748506  L 175.535664 197.280417  L 185.941259 195.812329  L 196.346853 194.344237  L 206.752448 192.876148  L 217.158042 191.40806  L 227.563636 189.939967  L 237.969231 188.471879  L 248.374825 187.003789  L 258.78042 185.535701  L 269.186014 184.06761  L 279.591608 182.59952  L 289.997203 181.131432  L 300.402797 179.663341  L 310.808392 178.195253  L 321.213986 176.72716  L 331.61958 175.259072  L 342.025175 173.790984  L 352.430769 172.322891  L 362.836364 170.854803  L 373.241958 169.386715  L 383.647552 167.918622  L 394.053147 166.450534  L 404.458741 164.982446  L 414.864336 163.514353  L 425.26993 162.046265  L 435.675524 160.578177  L 446.081119 159.110088  L 456.486713 157.641996  L 466.892308 156.173903  L 477.297902 154.705815  L 487.703497 153.237727  L 498.109091 151.769638  \" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_20\">\n",
       "    <path clip-path=\"url(#p386362bc70)\" d=\"M 92.290909 207.615553  L 102.696503 206.815037  L 113.102098 205.953078  L 123.507692 205.02714  L 133.913287 204.034979  L 144.318881 202.974731  L 154.724476 201.844993  L 165.13007 200.644909  L 175.535664 199.374263  L 185.941259 198.033555  L 196.346853 196.624076  L 206.752448 195.147981  L 217.158042 193.60832  L 227.563636 192.009077  L 237.969231 190.355165  L 248.374825 188.652404  L 258.78042 186.907457  L 269.186014 185.127755  L 279.591608 183.321379  L 289.997203 181.496927  L 300.402797 179.663341  L 310.808392 177.829758  L 321.213986 176.005306  L 331.61958 174.198928  L 342.025175 172.419225  L 352.430769 170.674276  L 362.836364 168.971517  L 373.241958 167.317607  L 383.647552 165.718362  L 394.053147 164.178704  L 404.458741 162.702604  L 414.864336 161.293127  L 425.26993 159.952415  L 435.675524 158.681769  L 446.081119 157.481687  L 456.486713 156.351954  L 466.892308 155.291704  L 477.297902 154.299538  L 487.703497 153.373609  L 498.109091 152.511641  \" style=\"fill:none;stroke:#ff7f0e;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_21\">\n",
       "    <path clip-path=\"url(#p386362bc70)\" d=\"M 92.290909 287.129527  L 102.696503 286.5577  L 113.102098 285.865387  L 123.507692 285.028796  L 133.913287 284.020183  L 144.318881 282.807517  L 154.724476 281.354362  L 165.13007 279.619926  L 175.535664 277.559573  L 185.941259 275.125802  L 196.346853 272.270008  L 206.752448 268.945056  L 217.158042 265.108864  L 227.563636 260.728887  L 237.969231 255.787433  L 248.374825 250.287053  L 258.78042 244.255542  L 269.186014 237.749233  L 279.591608 230.853812  L 289.997203 223.681659  L 300.402797 216.365582  L 310.808392 209.049505  L 321.213986 201.877352  L 331.61958 194.981931  L 342.025175 188.475622  L 352.430769 182.444111  L 362.836364 176.943731  L 373.241958 172.002277  L 383.647552 167.6223  L 394.053147 163.786108  L 404.458741 160.461156  L 414.864336 157.605362  L 425.26993 155.171591  L 435.675524 153.111238  L 446.081119 151.376802  L 456.486713 149.923647  L 466.892308 148.710981  L 477.297902 147.702368  L 487.703497 146.865777  L 498.109091 146.173464  \" style=\"fill:none;stroke:#2ca02c;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_22\">\n",
       "    <path clip-path=\"url(#p386362bc70)\" d=\"M 92.290909 216.365582  L 102.696503 216.365582  L 113.102098 216.365582  L 123.507692 216.365582  L 133.913287 216.365582  L 144.318881 216.365582  L 154.724476 216.365582  L 165.13007 216.365582  L 175.535664 216.365582  L 185.941259 216.365582  L 196.346853 216.365582  L 206.752448 216.365582  L 217.158042 216.365582  L 227.563636 216.365582  L 237.969231 216.365582  L 248.374825 216.365582  L 258.78042 216.365582  L 269.186014 216.365582  L 279.591608 216.365582  L 289.997203 216.365582  L 300.402797 216.365582  L 310.808392 209.025134  L 321.213986 201.684685  L 331.61958 194.344237  L 342.025175 187.003789  L 352.430769 179.663341  L 362.836364 172.322891  L 373.241958 164.982446  L 383.647552 157.641996  L 394.053147 150.30155  L 404.458741 142.9611  L 414.864336 135.62065  L 425.26993 128.2802  L 435.675524 120.939759  L 446.081119 113.599309  L 456.486713 106.258859  L 466.892308 98.918409  L 477.297902 91.577959  L 487.703497 84.237518  L 498.109091 76.897068  \" style=\"fill:none;stroke:#d62728;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_23\">\n",
       "    <path clip-path=\"url(#p386362bc70)\" d=\"M 92.290909 265.301905  L 102.696503 264.458173  L 113.102098 263.554177  L 123.507692 262.583217  L 133.913287 261.537572  L 144.318881 260.408273  L 154.724476 259.184862  L 165.13007 257.855071  L 175.535664 256.404391  L 185.941259 254.815551  L 196.346853 253.067823  L 206.752448 251.136125  L 217.158042 248.989798  L 227.563636 246.590955  L 237.969231 243.892263  L 248.374825 240.833743  L 258.78042 237.338292  L 269.186014 233.305079  L 279.591608 228.599662  L 289.997203 223.038717  L 300.402797 216.365582  L 310.808392 209.692447  L 321.213986 204.131502  L 331.61958 199.426085  L 342.025175 195.392872  L 352.430769 191.897421  L 362.836364 188.838901  L 373.241958 186.140209  L 383.647552 183.741365  L 394.053147 181.595039  L 404.458741 179.663341  L 414.864336 177.915613  L 425.26993 176.326773  L 435.675524 174.876093  L 446.081119 173.546302  L 456.486713 172.322891  L 466.892308 171.193592  L 477.297902 170.147947  L 487.703497 169.176987  L 498.109091 168.272991  \" style=\"fill:none;stroke:#9467bd;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_24\">\n",
       "    <path clip-path=\"url(#p386362bc70)\" d=\"M 92.290909 207.048494  L 102.696503 206.133974  L 113.102098 205.136336  L 123.507692 204.049339  L 133.913287 202.866441  L 144.318881 201.580941  L 154.724476 200.185958  L 165.13007 198.674484  L 175.535664 197.03947  L 185.941259 195.273881  L 196.346853 193.370772  L 206.752448 191.323355  L 217.158042 189.125132  L 227.563636 186.769917  L 237.969231 184.252003  L 248.374825 181.566206  L 258.78042 178.707962  L 269.186014 175.673421  L 279.591608 172.459504  L 289.997203 169.063979  L 300.402797 165.485472  L 310.808392 161.723529  L 321.213986 157.778617  L 331.61958 153.65208  L 342.025175 149.346171  L 352.430769 144.863966  L 362.836364 140.209314  L 373.241958 135.386785  L 383.647552 130.401548  L 394.053147 125.259328  L 404.458741 119.966293  L 414.864336 114.528953  L 425.26993 108.954091  L 435.675524 103.248662  L 446.081119 97.419677  L 456.486713 91.474222  L 466.892308 85.419273  L 477.297902 79.261715  L 487.703497 73.008278  L 498.109091 66.665455  \" style=\"fill:none;stroke:#8c564b;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_25\">\n",
       "    <path clip-path=\"url(#p386362bc70)\" d=\"M 92.290909 279.835849  L 102.696503 278.791057  L 113.102098 277.636385  L 123.507692 276.360274  L 133.913287 274.949952  L 144.318881 273.391309  L 154.724476 271.668739  L 165.13007 269.765008  L 175.535664 267.66106  L 185.941259 265.335835  L 196.346853 262.766063  L 206.752448 259.926029  L 217.158042 256.787304  L 227.563636 253.318476  L 237.969231 249.484832  L 248.374825 245.247994  L 258.78042 240.565569  L 269.186014 235.390687  L 279.591608 229.671556  L 289.997203 223.350941  L 300.402797 216.365582  L 310.808392 209.025134  L 321.213986 201.684685  L 331.61958 194.344237  L 342.025175 187.003789  L 352.430769 179.663341  L 362.836364 172.322891  L 373.241958 164.982446  L 383.647552 157.641996  L 394.053147 150.30155  L 404.458741 142.9611  L 414.864336 135.62065  L 425.26993 128.2802  L 435.675524 120.939759  L 446.081119 113.599309  L 456.486713 106.258859  L 466.892308 98.918409  L 477.297902 91.577959  L 487.703497 84.237518  L 498.109091 76.897068  \" style=\"fill:none;stroke:#e377c2;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_26\">\n",
       "    <path clip-path=\"url(#p386362bc70)\" d=\"M 92.290909 327.952617  L 102.696503 326.115774  L 113.102098 324.085738  L 123.507692 321.842207  L 133.913287 319.362728  L 144.318881 316.622475  L 154.724476 313.594033  L 165.13007 310.247073  L 175.535664 306.548125  L 185.941259 302.460147  L 196.346853 297.94224  L 206.752448 292.94918  L 217.158042 287.430986  L 227.563636 281.332442  L 237.969231 274.592512  L 248.374825 267.143735  L 258.78042 258.911562  L 269.186014 249.813607  L 279.591608 239.758806  L 289.997203 228.646537  L 300.402797 216.365582  L 310.808392 208.652965  L 321.213986 200.940349  L 331.61958 193.227732  L 342.025175 185.515115  L 352.430769 177.8025  L 362.836364 170.089883  L 373.241958 162.37727  L 383.647552 154.664648  L 394.053147 146.952035  L 404.458741 139.239418  L 414.864336 131.526801  L 425.26993 123.814183  L 435.675524 116.101575  L 446.081119 108.388958  L 456.486713 100.67634  L 466.892308 92.963714  L 477.297902 85.251097  L 487.703497 77.538489  L 498.109091 69.825871  \" style=\"fill:none;stroke:#7f7f7f;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_27\">\n",
       "    <path clip-path=\"url(#p386362bc70)\" d=\"M 92.290909 363.174545  L 102.696503 355.834097  L 113.102098 348.493649  L 123.507692 341.153201  L 133.913287 333.812753  L 144.318881 326.472305  L 154.724476 319.131856  L 165.13007 311.791408  L 175.535664 304.45096  L 185.941259 297.110512  L 196.346853 289.770064  L 206.752448 282.429616  L 217.158042 275.089167  L 227.563636 267.748719  L 237.969231 260.408271  L 248.374825 253.067823  L 258.78042 245.727375  L 269.186014 238.386926  L 279.591608 231.046478  L 289.997203 223.70603  L 300.402797 216.365582  L 310.808392 209.025134  L 321.213986 201.684686  L 331.61958 194.344237  L 342.025175 187.003789  L 352.430769 179.663341  L 362.836364 172.322893  L 373.241958 164.982445  L 383.647552 157.641997  L 394.053147 150.301548  L 404.458741 142.9611  L 414.864336 135.620652  L 425.26993 128.280204  L 435.675524 120.939756  L 446.081119 113.599308  L 456.486713 106.258859  L 466.892308 98.918411  L 477.297902 91.577963  L 487.703497 84.237515  L 498.109091 76.897067  \" style=\"fill:none;stroke:#bcbd22;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 72 378  L 72 51.84  \" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 518.4 378  L 518.4 51.84  \" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 72 378  L 518.4 378  \" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 72 51.84  L 518.4 51.84  \" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"text_21\">\n",
       "    <!-- Activation Functions -->\n",
       "    <defs>\n",
       "     <path d=\"M 34.1875 63.1875  L 20.796875 26.90625  L 47.609375 26.90625  z M 28.609375 72.90625  L 39.796875 72.90625  L 67.578125 0  L 57.328125 0  L 50.6875 18.703125  L 17.828125 18.703125  L 11.1875 0  L 0.78125 0  z \" id=\"DejaVuSans-41\"/>\n",
       "     <path d=\"M 48.78125 52.59375  L 48.78125 44.1875  Q 44.96875 46.296875 41.140625 47.34375  Q 37.3125 48.390625 33.40625 48.390625  Q 24.65625 48.390625 19.8125 42.84375  Q 14.984375 37.3125 14.984375 27.296875  Q 14.984375 17.28125 19.8125 11.734375  Q 24.65625 6.203125 33.40625 6.203125  Q 37.3125 6.203125 41.140625 7.25  Q 44.96875 8.296875 48.78125 10.40625  L 48.78125 2.09375  Q 45.015625 0.34375 40.984375 -0.53125  Q 36.96875 -1.421875 32.421875 -1.421875  Q 20.0625 -1.421875 12.78125 6.34375  Q 5.515625 14.109375 5.515625 27.296875  Q 5.515625 40.671875 12.859375 48.328125  Q 20.21875 56 33.015625 56  Q 37.15625 56 41.109375 55.140625  Q 45.0625 54.296875 48.78125 52.59375  z \" id=\"DejaVuSans-63\"/>\n",
       "     <path d=\"M 2.984375 54.6875  L 12.5 54.6875  L 29.59375 8.796875  L 46.6875 54.6875  L 56.203125 54.6875  L 35.6875 0  L 23.484375 0  z \" id=\"DejaVuSans-76\"/>\n",
       "     <path d=\"M 34.28125 27.484375  Q 23.390625 27.484375 19.1875 25  Q 14.984375 22.515625 14.984375 16.5  Q 14.984375 11.71875 18.140625 8.90625  Q 21.296875 6.109375 26.703125 6.109375  Q 34.1875 6.109375 38.703125 11.40625  Q 43.21875 16.703125 43.21875 25.484375  L 43.21875 27.484375  z M 52.203125 31.203125  L 52.203125 0  L 43.21875 0  L 43.21875 8.296875  Q 40.140625 3.328125 35.546875 0.953125  Q 30.953125 -1.421875 24.3125 -1.421875  Q 15.921875 -1.421875 10.953125 3.296875  Q 6 8.015625 6 15.921875  Q 6 25.140625 12.171875 29.828125  Q 18.359375 34.515625 30.609375 34.515625  L 43.21875 34.515625  L 43.21875 35.40625  Q 43.21875 41.609375 39.140625 45  Q 35.0625 48.390625 27.6875 48.390625  Q 23 48.390625 18.546875 47.265625  Q 14.109375 46.140625 10.015625 43.890625  L 10.015625 52.203125  Q 14.9375 54.109375 19.578125 55.046875  Q 24.21875 56 28.609375 56  Q 40.484375 56 46.34375 49.84375  Q 52.203125 43.703125 52.203125 31.203125  z \" id=\"DejaVuSans-61\"/>\n",
       "     <path id=\"DejaVuSans-20\"/>\n",
       "     <path d=\"M 9.8125 72.90625  L 51.703125 72.90625  L 51.703125 64.59375  L 19.671875 64.59375  L 19.671875 43.109375  L 48.578125 43.109375  L 48.578125 34.8125  L 19.671875 34.8125  L 19.671875 0  L 9.8125 0  z \" id=\"DejaVuSans-46\"/>\n",
       "     <path d=\"M 44.28125 53.078125  L 44.28125 44.578125  Q 40.484375 46.53125 36.375 47.5  Q 32.28125 48.484375 27.875 48.484375  Q 21.1875 48.484375 17.84375 46.4375  Q 14.5 44.390625 14.5 40.28125  Q 14.5 37.15625 16.890625 35.375  Q 19.28125 33.59375 26.515625 31.984375  L 29.59375 31.296875  Q 39.15625 29.25 43.1875 25.515625  Q 47.21875 21.78125 47.21875 15.09375  Q 47.21875 7.46875 41.1875 3.015625  Q 35.15625 -1.421875 24.609375 -1.421875  Q 20.21875 -1.421875 15.453125 -0.5625  Q 10.6875 0.296875 5.421875 2  L 5.421875 11.28125  Q 10.40625 8.6875 15.234375 7.390625  Q 20.0625 6.109375 24.8125 6.109375  Q 31.15625 6.109375 34.5625 8.28125  Q 37.984375 10.453125 37.984375 14.40625  Q 37.984375 18.0625 35.515625 20.015625  Q 33.0625 21.96875 24.703125 23.78125  L 21.578125 24.515625  Q 13.234375 26.265625 9.515625 29.90625  Q 5.8125 33.546875 5.8125 39.890625  Q 5.8125 47.609375 11.28125 51.796875  Q 16.75 56 26.8125 56  Q 31.78125 56 36.171875 55.265625  Q 40.578125 54.546875 44.28125 53.078125  z \" id=\"DejaVuSans-73\"/>\n",
       "    </defs>\n",
       "    <g transform=\"translate(234.181875 45.84)scale(0.12 -0.12)\">\n",
       "     <use xlink:href=\"#DejaVuSans-41\"/>\n",
       "     <use x=\"68.392578\" xlink:href=\"#DejaVuSans-63\"/>\n",
       "     <use x=\"123.373047\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "     <use x=\"162.582031\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "     <use x=\"190.365234\" xlink:href=\"#DejaVuSans-76\"/>\n",
       "     <use x=\"249.544922\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "     <use x=\"310.824219\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "     <use x=\"350.033203\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "     <use x=\"377.816406\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "     <use x=\"438.998047\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "     <use x=\"502.376953\" xlink:href=\"#DejaVuSans-20\"/>\n",
       "     <use x=\"534.164062\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "     <use x=\"591.605469\" xlink:href=\"#DejaVuSans-75\"/>\n",
       "     <use x=\"654.984375\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "     <use x=\"718.363281\" xlink:href=\"#DejaVuSans-63\"/>\n",
       "     <use x=\"773.34375\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "     <use x=\"812.552734\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "     <use x=\"840.335938\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "     <use x=\"901.517578\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "     <use x=\"964.896484\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 79 192.22125  L 178.245313 192.22125  Q 180.245313 192.22125 180.245313 190.22125  L 180.245313 58.84  Q 180.245313 56.84 178.245313 56.84  L 79 56.84  Q 77 56.84 77 58.84  L 77 190.22125  Q 77 192.22125 79 192.22125  z \" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_28\">\n",
       "     <path d=\"M 81 64.938437  L 101 64.938437  \" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_29\"/>\n",
       "    <g id=\"text_22\">\n",
       "     <!-- hard_sigmoid -->\n",
       "     <defs>\n",
       "      <path d=\"M 54.890625 33.015625  L 54.890625 0  L 45.90625 0  L 45.90625 32.71875  Q 45.90625 40.484375 42.875 44.328125  Q 39.84375 48.1875 33.796875 48.1875  Q 26.515625 48.1875 22.3125 43.546875  Q 18.109375 38.921875 18.109375 30.90625  L 18.109375 0  L 9.078125 0  L 9.078125 75.984375  L 18.109375 75.984375  L 18.109375 46.1875  Q 21.34375 51.125 25.703125 53.5625  Q 30.078125 56 35.796875 56  Q 45.21875 56 50.046875 50.171875  Q 54.890625 44.34375 54.890625 33.015625  z \" id=\"DejaVuSans-68\"/>\n",
       "      <path d=\"M 41.109375 46.296875  Q 39.59375 47.171875 37.8125 47.578125  Q 36.03125 48 33.890625 48  Q 26.265625 48 22.1875 43.046875  Q 18.109375 38.09375 18.109375 28.8125  L 18.109375 0  L 9.078125 0  L 9.078125 54.6875  L 18.109375 54.6875  L 18.109375 46.1875  Q 20.953125 51.171875 25.484375 53.578125  Q 30.03125 56 36.53125 56  Q 37.453125 56 38.578125 55.875  Q 39.703125 55.765625 41.0625 55.515625  z \" id=\"DejaVuSans-72\"/>\n",
       "      <path d=\"M 45.40625 46.390625  L 45.40625 75.984375  L 54.390625 75.984375  L 54.390625 0  L 45.40625 0  L 45.40625 8.203125  Q 42.578125 3.328125 38.25 0.953125  Q 33.9375 -1.421875 27.875 -1.421875  Q 17.96875 -1.421875 11.734375 6.484375  Q 5.515625 14.40625 5.515625 27.296875  Q 5.515625 40.1875 11.734375 48.09375  Q 17.96875 56 27.875 56  Q 33.9375 56 38.25 53.625  Q 42.578125 51.265625 45.40625 46.390625  z M 14.796875 27.296875  Q 14.796875 17.390625 18.875 11.75  Q 22.953125 6.109375 30.078125 6.109375  Q 37.203125 6.109375 41.296875 11.75  Q 45.40625 17.390625 45.40625 27.296875  Q 45.40625 37.203125 41.296875 42.84375  Q 37.203125 48.484375 30.078125 48.484375  Q 22.953125 48.484375 18.875 42.84375  Q 14.796875 37.203125 14.796875 27.296875  z \" id=\"DejaVuSans-64\"/>\n",
       "      <path d=\"M 50.984375 -16.609375  L 50.984375 -23.578125  L -0.984375 -23.578125  L -0.984375 -16.609375  z \" id=\"DejaVuSans-5f\"/>\n",
       "      <path d=\"M 45.40625 27.984375  Q 45.40625 37.75 41.375 43.109375  Q 37.359375 48.484375 30.078125 48.484375  Q 22.859375 48.484375 18.828125 43.109375  Q 14.796875 37.75 14.796875 27.984375  Q 14.796875 18.265625 18.828125 12.890625  Q 22.859375 7.515625 30.078125 7.515625  Q 37.359375 7.515625 41.375 12.890625  Q 45.40625 18.265625 45.40625 27.984375  z M 54.390625 6.78125  Q 54.390625 -7.171875 48.1875 -13.984375  Q 42 -20.796875 29.203125 -20.796875  Q 24.46875 -20.796875 20.265625 -20.09375  Q 16.0625 -19.390625 12.109375 -17.921875  L 12.109375 -9.1875  Q 16.0625 -11.328125 19.921875 -12.34375  Q 23.78125 -13.375 27.78125 -13.375  Q 36.625 -13.375 41.015625 -8.765625  Q 45.40625 -4.15625 45.40625 5.171875  L 45.40625 9.625  Q 42.625 4.78125 38.28125 2.390625  Q 33.9375 0 27.875 0  Q 17.828125 0 11.671875 7.65625  Q 5.515625 15.328125 5.515625 27.984375  Q 5.515625 40.671875 11.671875 48.328125  Q 17.828125 56 27.875 56  Q 33.9375 56 38.28125 53.609375  Q 42.625 51.21875 45.40625 46.390625  L 45.40625 54.6875  L 54.390625 54.6875  z \" id=\"DejaVuSans-67\"/>\n",
       "      <path d=\"M 52 44.1875  Q 55.375 50.25 60.0625 53.125  Q 64.75 56 71.09375 56  Q 79.640625 56 84.28125 50.015625  Q 88.921875 44.046875 88.921875 33.015625  L 88.921875 0  L 79.890625 0  L 79.890625 32.71875  Q 79.890625 40.578125 77.09375 44.375  Q 74.3125 48.1875 68.609375 48.1875  Q 61.625 48.1875 57.5625 43.546875  Q 53.515625 38.921875 53.515625 30.90625  L 53.515625 0  L 44.484375 0  L 44.484375 32.71875  Q 44.484375 40.625 41.703125 44.40625  Q 38.921875 48.1875 33.109375 48.1875  Q 26.21875 48.1875 22.15625 43.53125  Q 18.109375 38.875 18.109375 30.90625  L 18.109375 0  L 9.078125 0  L 9.078125 54.6875  L 18.109375 54.6875  L 18.109375 46.1875  Q 21.1875 51.21875 25.484375 53.609375  Q 29.78125 56 35.6875 56  Q 41.65625 56 45.828125 52.96875  Q 50 49.953125 52 44.1875  z \" id=\"DejaVuSans-6d\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(109 68.438437)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-68\"/>\n",
       "      <use x=\"63.378906\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"124.658203\" xlink:href=\"#DejaVuSans-72\"/>\n",
       "      <use x=\"165.755859\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "      <use x=\"229.232422\" xlink:href=\"#DejaVuSans-5f\"/>\n",
       "      <use x=\"279.232422\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"331.332031\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"359.115234\" xlink:href=\"#DejaVuSans-67\"/>\n",
       "      <use x=\"422.591797\" xlink:href=\"#DejaVuSans-6d\"/>\n",
       "      <use x=\"520.003906\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"581.185547\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"608.96875\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_30\">\n",
       "     <path d=\"M 81 79.894687  L 101 79.894687  \" style=\"fill:none;stroke:#ff7f0e;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_31\"/>\n",
       "    <g id=\"text_23\">\n",
       "     <!-- sigmoid -->\n",
       "     <g transform=\"translate(109 83.394687)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"52.099609\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"79.882812\" xlink:href=\"#DejaVuSans-67\"/>\n",
       "      <use x=\"143.359375\" xlink:href=\"#DejaVuSans-6d\"/>\n",
       "      <use x=\"240.771484\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"301.953125\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"329.736328\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_32\">\n",
       "     <path d=\"M 81 94.572812  L 101 94.572812  \" style=\"fill:none;stroke:#2ca02c;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_33\"/>\n",
       "    <g id=\"text_24\">\n",
       "     <!-- tanh -->\n",
       "     <g transform=\"translate(109 98.072812)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"100.488281\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "      <use x=\"163.867188\" xlink:href=\"#DejaVuSans-68\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_34\">\n",
       "     <path d=\"M 81 109.250937  L 101 109.250937  \" style=\"fill:none;stroke:#d62728;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_35\"/>\n",
       "    <g id=\"text_25\">\n",
       "     <!-- relu -->\n",
       "     <defs>\n",
       "      <path d=\"M 56.203125 29.59375  L 56.203125 25.203125  L 14.890625 25.203125  Q 15.484375 15.921875 20.484375 11.0625  Q 25.484375 6.203125 34.421875 6.203125  Q 39.59375 6.203125 44.453125 7.46875  Q 49.3125 8.734375 54.109375 11.28125  L 54.109375 2.78125  Q 49.265625 0.734375 44.1875 -0.34375  Q 39.109375 -1.421875 33.890625 -1.421875  Q 20.796875 -1.421875 13.15625 6.1875  Q 5.515625 13.8125 5.515625 26.8125  Q 5.515625 40.234375 12.765625 48.109375  Q 20.015625 56 32.328125 56  Q 43.359375 56 49.78125 48.890625  Q 56.203125 41.796875 56.203125 29.59375  z M 47.21875 32.234375  Q 47.125 39.59375 43.09375 43.984375  Q 39.0625 48.390625 32.421875 48.390625  Q 24.90625 48.390625 20.390625 44.140625  Q 15.875 39.890625 15.1875 32.171875  z \" id=\"DejaVuSans-65\"/>\n",
       "      <path d=\"M 9.421875 75.984375  L 18.40625 75.984375  L 18.40625 0  L 9.421875 0  z \" id=\"DejaVuSans-6c\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(109 112.750937)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-72\"/>\n",
       "      <use x=\"41.082031\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"102.605469\" xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"130.388672\" xlink:href=\"#DejaVuSans-75\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_36\">\n",
       "     <path d=\"M 81 123.929062  L 101 123.929062  \" style=\"fill:none;stroke:#9467bd;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_37\"/>\n",
       "    <g id=\"text_26\">\n",
       "     <!-- softsign -->\n",
       "     <defs>\n",
       "      <path d=\"M 37.109375 75.984375  L 37.109375 68.5  L 28.515625 68.5  Q 23.6875 68.5 21.796875 66.546875  Q 19.921875 64.59375 19.921875 59.515625  L 19.921875 54.6875  L 34.71875 54.6875  L 34.71875 47.703125  L 19.921875 47.703125  L 19.921875 0  L 10.890625 0  L 10.890625 47.703125  L 2.296875 47.703125  L 2.296875 54.6875  L 10.890625 54.6875  L 10.890625 58.5  Q 10.890625 67.625 15.140625 71.796875  Q 19.390625 75.984375 28.609375 75.984375  z \" id=\"DejaVuSans-66\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(109 127.429062)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"52.099609\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"113.28125\" xlink:href=\"#DejaVuSans-66\"/>\n",
       "      <use x=\"148.470703\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"187.679688\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"239.779297\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"267.5625\" xlink:href=\"#DejaVuSans-67\"/>\n",
       "      <use x=\"331.039062\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_38\">\n",
       "     <path d=\"M 81 138.607187  L 101 138.607187  \" style=\"fill:none;stroke:#8c564b;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_39\"/>\n",
       "    <g id=\"text_27\">\n",
       "     <!-- softplus -->\n",
       "     <g transform=\"translate(109 142.107187)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"52.099609\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"113.28125\" xlink:href=\"#DejaVuSans-66\"/>\n",
       "      <use x=\"148.470703\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"187.679688\" xlink:href=\"#DejaVuSans-70\"/>\n",
       "      <use x=\"251.15625\" xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"278.939453\" xlink:href=\"#DejaVuSans-75\"/>\n",
       "      <use x=\"342.318359\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_40\">\n",
       "     <path d=\"M 81 153.285312  L 101 153.285312  \" style=\"fill:none;stroke:#e377c2;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_41\"/>\n",
       "    <g id=\"text_28\">\n",
       "     <!-- elu -->\n",
       "     <g transform=\"translate(109 156.785312)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"61.523438\" xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"89.306641\" xlink:href=\"#DejaVuSans-75\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_42\">\n",
       "     <path d=\"M 81 167.963437  L 101 167.963437  \" style=\"fill:none;stroke:#7f7f7f;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_43\"/>\n",
       "    <g id=\"text_29\">\n",
       "     <!-- selu -->\n",
       "     <g transform=\"translate(109 171.463437)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"52.099609\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"113.623047\" xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"141.40625\" xlink:href=\"#DejaVuSans-75\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_44\">\n",
       "     <path d=\"M 81 182.641562  L 101 182.641562  \" style=\"fill:none;stroke:#bcbd22;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_45\"/>\n",
       "    <g id=\"text_30\">\n",
       "     <!-- linear -->\n",
       "     <g transform=\"translate(109 186.141562)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"27.783203\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"55.566406\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "      <use x=\"118.945312\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"180.46875\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"241.748047\" xlink:href=\"#DejaVuSans-72\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p386362bc70\">\n",
       "   <rect height=\"326.16\" width=\"446.4\" x=\"72\" y=\"51.84\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cx.plot(list(lines.items()), xs=float_range, symbols=symbols, height=6.0, width=8.0, \n",
    "        title=\"Activation Functions\", xlabel=\"input\", ylabel=\"output\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See also the advanced activation functions:\n",
    "\n",
    "https://keras.io/layers/advanced-activations/"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}