cem/cem_test.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Cross-entropy (CE) Method Test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## CE method test 1\n",
    "The distribution in CE method directly fits the f-value, in this approach, the f-value has to be the directly fitted by the cem, it cannot depend on other parameters. But it works! How to prove the convergence?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('true min: ', -2.921757706470594)\n",
      "[-3.05239919 -3.05239919 -3.05239919 -3.05239919 -3.05239919 -3.05239919\n",
      " -3.05239919 -3.05239919 -3.05239919 -3.05239919]\n"
     ]
    }
   ],
   "source": [
    "# cem test 1\n",
    "# cem directly fits the f-value, in this approach, the f has to be the direct\n",
    "# values fitted by the cem, it cannot depend on other parameters. But it works!\n",
    "import numpy as np\n",
    "N=100 # number of samples\n",
    "ratio=0.1 # ratio of selection\n",
    "num_itr=50\n",
    "samples = np.random.normal(0,1,N)  # f-value: from a normal distribution\n",
    "print('true min: ', np.min(samples))\n",
    "for i in range(num_itr):\n",
    "    # sample and select top 10% (max [-a:]or min [:a])\n",
    "    selected_samples = samples[np.array(samples).argsort()[:int(N*ratio)]]\n",
    "#     selected_samples = samples[np.array(samples).argsort()[-int(N*ratio):]]\n",
    "    m=np.mean(selected_samples)\n",
    "    std=np.std(selected_samples)\n",
    "    # fit the gaussian\n",
    "    samples=np.random.normal(m, std, N)\n",
    "print(selected_samples)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Results from above: \n",
    "* After CE iterations, the values close to the ture minimal of original samples or the f-distribution are finally sampled, which testifies directly fitting original f-value could work in some situations.\n",
    "* And the standard deviation in the fitting Gaussian distribution of CE method decreases very fast actually. \n",
    "* The final sampled values are actually still not the true optimal value of the original distribution, just close to it. CE method actually cannot be accurate, because it always depends on the number of samples used for evaluation. The larger the number, the more accurate it is.　This method could be accurate to the infinite large limit number of samples, under the law of large numbers. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## CE method test 2\n",
    "The distribution of CE method fits the x of f(x). In this approach, the x can be parameters of \n",
    "the function f, therefore f can be dependent on other variables z like f(x,z). It works \n",
    "as long as there exists minimum of the function f."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('true min: ', 0.0003483104651236677)\n",
      "[2.95679397e-112 7.84355027e-112 1.87860981e-111 2.02189501e-111\n",
      " 2.30254618e-111 2.77905031e-111 2.84213630e-111 3.37676217e-111\n",
      " 3.53559618e-111 3.84893811e-111]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd8XGed7/HPT6Nm2eqSbVnFPS5xi+3YjkkCxIlTCHE2kMUsEO8SNgTCLmUvu8mF14YLu3fhsnspd2khCQQIKYSEBEhwOiZxXOTeLblKbpKtYsmy6jz3jzkygyXZsqSZMyN936+XrJnnPDPz09F4vjrPc4o55xAREQmX4HcBIiISexQOIiLShcJBRES6UDiIiEgXCgcREelC4SAiIl0oHEREpAuFg4iIdKFwEBGRLhL9LqCv8vLy3Lhx4/wuQ0QkbmzYsOGkcy6/N33jNhzGjRtHaWmp32WIiMQNMzvU274aVhIRkS4UDiIi0oXCQUREulA4iIhIFwoHERHpQuEgIiJdKBxERKSLIRUOre1BfvDmPlbtrfa7FBGRS/bqzhM8tGof0bi885AKh6SA8dCqffx+6zG/SxERuWTPbTrCz9ccwswi/lpDKhzMjBmFmWw7Uu93KSIil2zbkXpmFmZG5bUuGg5m9qiZVZnZ9rC2HDN7xczKvO/ZXruZ2XfNrNzMtprZ3LDHrPD6l5nZirD2eWa2zXvMdy3CkTirKJO9JxpobuuI5MuIiAyouqZWDtc0MbMwKyqv15sth58CN53Xdj/wmnNuMvCadx/gZmCy93UP8AMIhQnwILAQWAA82BkoXp97wh53/msNqJmFmbQHHbuPN0TyZUREBtT2I6cBYmfLwTm3Cqg5r3kZ8Jh3+zHg9rD2n7mQNUCWmRUANwKvOOdqnHO1wCvATd6yDOfcOy40w/KzsOeKiJlFodTdVlkXyZcRERlQW4+EPrNiJhx6MMo5dwzA+z7Say8EKsL6VXptF2qv7KY9YsZkppIzPFnzDiISV7YfqackJ43MtKSovN5AT0h3N1/g+tDe/ZOb3WNmpWZWWl3dt91RzYyZhZlsrVQ4iEj82FpZz8yi6Gw1QN/D4YQ3JIT3vcprrwSKw/oVAUcv0l7UTXu3nHMPOefmO+fm5+f36noV3ZpZmElZVaMmpUUkLtSeaaWy9mzUhpSg7+HwAtC5x9EK4Pmw9ru8vZYWAfXesNNKYKmZZXsT0UuBld6yBjNb5O2ldFfYc0XMzKJMOoKOncdOR/qlRET6rXMYfFYshYOZPQG8A0wxs0ozuxv4OnCDmZUBN3j3AV4E9gPlwI+BTwM452qArwHrva+vem0AnwIe9h6zD3hpYH60nnWm73bNO4hIHOgMh8ujGA4XvUyoc+7DPSxa0k1fB9zXw/M8CjzaTXspMONidQykgsxU8kYka95BROLCtsp6xuWmkTksOpPRMMSOkO7UeaS0thxEJB5sO1LPjChuNcAQDQcIjd3tPdHA2VZNSotI7DrV2MKRurPMiuKeSjCEw2FGYSZBBzuPaetBRGJX53yDthyiZNa5I6UVDiISu7YrHKJrVEYK+ekpmpQWkZi2tbKe8XnDyUiN3mQ0DOFwMDNmF2WyWedYEpEY5Zxjc0Uds6M83wBDOBwA5hRnsb/6DPVNbX6XIiLSxbH6ZqoaWphTHJ3TdIcb4uEQOmv4Fm09iEgM2lIR+myaU5J9kZ4Db0iHw6ziTMz+/AsQEYklmyvqSA4kMK0gPeqvPaTDISM1iYn5I9iscBCRGLSpoo5pYzJISQxE/bWHdDhAaN5hc0UdoTN/iIjEhvaOINsq67nCh/kGUDgwuziLU97pcEVEYkVZVSNn2zp8mYwGhcO5VN6koSURiSGdw90KB59MGZ1OSmKCJqVFJKZsPlxHVloSY3PTfHn9IR8OSYEEZhZmalJaRGJK6OC3LELXQYu+IR8OENps236knraOoN+liIjQ2NLO3qoG34aUQOEAhCalW9qD7D7W4HcpIiJsq6zHOZhTonDwVWc6b66o9bkSEZE/T0bPLlI4+Kooexh5I5K1x5KIxITNFbWMzU0jZ3iybzUoHAidofWKkmw2HVY4iIi/nHNsPFzn28FvnRQOnnljszlw8gynGlv8LkVEhrDK2rNUN7Qwb1yOr3UoHDzzxobOerhRWw8i4qPSQzUAzPPhTKzhFA6emYWZJAXs3C9GRMQPGw7VMiIlkSmjo38m1nAKB09qUoAZhZlsPKQ9lkTEPxsO1XFFSRaBBH8OfuukcAgzrySbLZX1tLbrYDgRib6G5jb2HD/NXJ+HlEDh8Bfmjc2mtT3IjqP1fpciIkPQ5oo6gg7mj1M4xJTOSekNGloSER9sOFSLmX9nYg2ncAgzMiOV4pxhCgcR8cWGQ7VMGZVOemqS36UoHM43rySb0kO1ujKciERVR9Cx6XBdTAwpgcKhi3ljs6luaNGV4UQkqvaeaKCxpf3c8Lbf+hUOZvZ5M9thZtvN7AkzSzWz8Wa21szKzOwpM0v2+qZ498u95ePCnucBr32Pmd3Yvx+pf+aNDR2VqKElEYmmUu8zZ16Jv0dGd+pzOJhZIfCPwHzn3AwgACwHvgF8yzk3GagF7vYecjdQ65ybBHzL64eZTfcedzlwE/B9Mwv0ta7+mjI6neHJAYWDiETVxkO15KenUJwzzO9SgP4PKyUCw8wsEUgDjgHXAc94yx8DbvduL/Pu4y1fYqFLHC0DnnTOtTjnDgDlwIJ+1tVngYTQSfjWH9SR0iISPesP1jCvJNu3K7+dr8/h4Jw7AvwncJhQKNQDG4A651y7160SKPRuFwIV3mPbvf654e3dPOYvmNk9ZlZqZqXV1dV9Lf2iFozPYc+JBuqaWiP2GiIinY7UnaWy9iwLxsfGkBL0b1gpm9Bf/eOBMcBw4OZuunbu9tNdHLoLtHdtdO4h59x859z8/Pz8Sy+6lxaMz8E5KD2ooSURibz1B0IjFYMiHIDrgQPOuWrnXBvwLLAYyPKGmQCKgKPe7UqgGMBbngnUhLd38xhfzCnOIjmQwDoNLYlIFKw9UEN6aiLTCjL8LuWc/oTDYWCRmaV5cwdLgJ3AG8AHvT4rgOe92y949/GWv+5CBxO8ACz39mYaD0wG1vWjrn5LTQowpziLtQcUDiISeesOnOLKcTm+n2wvXH/mHNYSmljeCGzznush4F+AL5hZOaE5hUe8hzwC5HrtXwDu955nB/A0oWD5A3Cfc66jr3UNlAXjc9h+pJ4zLe0X7ywi0kcnG1vYV30mpoaUILS3UZ855x4EHjyveT/d7G3knGsG7uzhef4d+Pf+1DLQFozP4b/fKGfj4VqumRy5+Q0RGdpicb4BdIR0j+aOzSaQYKzT0JKIRNDaAzUMSwowY0ym36X8BYVDD0akJDJjTAZr9yscRCRy1h6oYe7YLJITY+vjOLaqiTELJ+SyuaKO5jbfp0BEZBCqb2pj9/HTLByf63cpXSgcLmDBuBxaO4JsqajzuxQRGYRKD9XgXOzNN4DC4YKuHJeDGZp3EJGIWHeghuRAQkxc3Od8CocLyExLYsqodB3vICIRsfZADbOLM0lN8u1coz1SOFzEogm5lB6qobU96HcpIjKINDS3se1IfUzON4DC4aIWT8yluS3IZs07iMgAWn+who6gY/EkhUNcWjghlwSDt8tP+l2KiAwib5efIjkxgbklsXHlt/MpHC4ic1gSMwozeWffKb9LEZFBZPW+U8wfmx2T8w2gcOiVxRPz2FRRS1OrzrMkIv1Xc6aVXcdO865JeX6X0iOFQy8snphLW4djva7vICIDoHMk4qqJsTnfAAqHXpk/LpukgLF6n+YdRKT/Vu87yYiURGYVxtb5lMIpHHohLTmRK0qyWV2ueQcR6b/V+06xcHwOiYHY/QiO3cpizOKJuWw/Wk99U5vfpYhIHDtad5YDJ8/E9JASKBx6bfHEPJyDNQe09SAifdc537B4YuxORoPCodfmFGcxLCnAah3vICL98Pa+k+QMT2bq6HS/S7kghUMvJScmcOX4HN7W8Q4i0kfOOd7Zd4qrJuSSEEPXi+6OwuESXD0pl/KqRo7Vn/W7FBGJQ/uqGzlW3xzTxzd0UjhcgmsvC11L+k97NbQkIpfuj95nx7WXKRwGlSmj0hmVkcIfy6r9LkVE4tCqvdVMyB9OUXaa36VclMLhEpgZ10zO562yk3QEnd/liEgcaW7rYO2BU1w7Od/vUnpF4XCJrr0sn/qzbWyt1Cm8RaT31h+sobktyLsvUzgMSldPysMMVmneQUQuwaq91SQHElg4IfauF90dhcMlyhmezMzCTFZp3kFELsGqvSeZPy6btOREv0vpFYVDH1w7OZ/NFXXUn9WpNETk4o7XN7PnRMO5PR7jgcKhD669LJ+OoNPR0iLSK50jDfEyGQ0Khz65oiSLESmJrCpTOIjIxf2p7CT56SlMK4jtU2aEUzj0QVIggcUTc1m1txrntEuriPSsI+h4q6yaaybnYRbbp8wI169wMLMsM3vGzHab2S4zu8rMcszsFTMr875ne33NzL5rZuVmttXM5oY9zwqvf5mZrejvDxUN75kykiN1ZymvavS7FBGJYVsq66htaoubXVg79XfL4TvAH5xzU4HZwC7gfuA159xk4DXvPsDNwGTv6x7gBwBmlgM8CCwEFgAPdgZKLLtu6kgAXttd5XMlIhLLXt9VRSDBeM9lI/0u5ZL0ORzMLAO4FngEwDnX6pyrA5YBj3ndHgNu924vA37mQtYAWWZWANwIvOKcq3HO1QKvADf1ta5oGZ2ZyuVjMnh9l8JBRHr22u4q5o/NJjMtye9SLkl/thwmANXAT8xsk5k9bGbDgVHOuWMA3vfOuCwEKsIeX+m19dQe866bOpLSQzXUNbX6XYqIxKCjdWfZdez0uZGGeNKfcEgE5gI/cM5dAZzhz0NI3eluJsZdoL3rE5jdY2alZlZaXe3/QWjXTR1J0MEf9/pfi4jEnte9Yecl04ZWOFQClc65td79ZwiFxQlvuAjve1VY/+KwxxcBRy/Q3oVz7iHn3Hzn3Pz8fP8nd2YXZZE7PJnXNLQkIt14fXcVJTlpTMwf4Xcpl6zP4eCcOw5UmNkUr2kJsBN4Aejc42gF8Lx3+wXgLm+vpUVAvTfstBJYambZ3kT0Uq8t5iUkGO+dOpI391TR3hH0uxwRiSFnWzt4u/wk100dGVe7sHbq70k+/gF43MySgf3A3xEKnKfN7G7gMHCn1/dF4BagHGjy+uKcqzGzrwHrvX5fdc7V9LOuqFkydSTPbKhkw6FaFk7I9bscEYkRq/edpKU9GJdDStDPcHDObQbmd7NoSTd9HXBfD8/zKPBof2rxy9WT80gKGK/vrlI4iMg5r+2uYnhygAXj4+MsrOfTEdL9lJ6axMLxuecmnkREnHO8sbuKaybnk5IY8LucPlE4DIDrpo6krKqRgyfP+F2KiMSAHUdPc6y+mevidEgJFA4D4obpowBYueO4z5WISCxYueM4CRaak4xXCocBUJyTxozCDIWDiAChcFgwPofcESl+l9JnCocBcuP00Ww8XMeJ081+lyIiPtpf3cjeE43cePlov0vpF4XDALlxRuiN8PLOEz5XIiJ+Wrkj9BmwVOEgAJNHjmBC3nBe1tCSyJC2csdxZhVlUpg1zO9S+kXhMEDMjKWXj+adfaeob9K1pUWGouP1zWyuqIv7ISVQOAyoGy8fRXvQ8dpuDS2JDEUv7wyNHNx4+SifK+k/hcMAml2UxeiMVO21JDJErdxxnIn5w5k0Mn6uFd0ThcMASkgwll4+ij/uraaptd3vckQkiuqaWlmzv2ZQDCmBwmHA3TRjNM1tQd7YrWs8iAwlK3ccpyPouHlGgd+lDAiFwwBbOD6X/PQUfrul20tSiMgg9dstxxibGzogdjBQOAywQIJxy4zRvLGnisYWDS2JDAUnG1tYve8kt84qiMtrN3RH4RAB7589hpb2IK/qgDiRIeGl7ccJutD//cFC4RABc0uyKchM1dCSyBDx2y1HmTRyBFNGxf9eSp0UDhGQkGC8b2YBq8qqdUCcyCB34nQz6w/WDKohJVA4RMz7Z4+hrcOxcqeOeRAZzH6/9RjOwa2zBs+QEigcImZWUSYlOWkaWhIZ5H679SjTCjKYNHKE36UMKIVDhJgZ75tVwOp9pzjV2OJ3OSISAZW1TWw6XMetswbHsQ3hFA4RdNvsMXQEHb/beszvUkQkAp7fHBoZeP8gG1IChUNETSvIYFpBBs9urPS7FBEZYM45fr2xkgXjcijJTfO7nAGncIiwD8wtZEtlPeVVDX6XIiIDaEtlPfurz3DH3EK/S4kIhUOE3TZnDAkGz2484ncpIjKAnt1YSXJiArcMwvkGUDhE3Mj0VK69LJ/nNh0hGHR+lyMiA6C1PcgLW46ydPooMlKT/C4nIhQOUXDH3CKO1TezZv8pv0sRkQHwxp4q6pra+MDcIr9LiRiFQxQsnT6K9JREfq2hJZFB4dmNleSNSOGayXl+lxIxCocoSE0K8L5ZBby0/ZguAiQS52rPtPL67ipunzOGxMDg/QgdvD9ZjLljbhFNrR28uE2n0xCJZ89vPkJbh+OvBuleSp36HQ5mFjCzTWb2O+/+eDNba2ZlZvaUmSV77Sne/XJv+biw53jAa99jZjf2t6ZYdOW4bCbkDefJdYf9LkVE+sg5x5PrK5hZmMnlYzL9LieiBmLL4bPArrD73wC+5ZybDNQCd3vtdwO1zrlJwLe8fpjZdGA5cDlwE/B9MwsMQF0xxcxYvqCY0kO1lJ3QMQ8i8WhzRR27jzfw4QUlfpcScf0KBzMrAt4HPOzdN+A64Bmvy2PA7d7tZd59vOVLvP7LgCedcy3OuQNAObCgP3XFqg/MLSIpYDyxrsLvUkSkD55Yd5i05AC3zRl8p8s4X3+3HL4N/DMQ9O7nAnXOuc5Z10qgc2CuEKgA8JbXe/3PtXfzmEEld0QKS6eP5tlNlTS3dfhdjohcgobmNn675RjvnzWGESmJfpcTcX0OBzO7Fahyzm0Ib+6mq7vIsgs95vzXvMfMSs2stLq6+pLqjRXLFxRT19TGyh2amBaJJy9sOcrZtg6WLyj2u5So6M+Ww7uA28zsIPAkoeGkbwNZZtYZq0VA5wUNKoFiAG95JlAT3t7NY/6Cc+4h59x859z8/Pz8fpTun3dNzKM4ZxhPamhJJK48ua6CqaPTmVOc5XcpUdHncHDOPeCcK3LOjSM0ofy6c+4jwBvAB71uK4DnvdsvePfxlr/unHNe+3Jvb6bxwGRgXV/rinUJCcbyK0t4Z/8pDpw843c5ItIL24/Us+1IPR9eUDKoLgV6IZE4zuFfgC+YWTmhOYVHvPZHgFyv/QvA/QDOuR3A08BO4A/Afc65QT0gf+e8IhITjMfXHPK7FBHphV+sOURqUgK3zxmU06HdGpBZFefcm8Cb3u39dLO3kXOuGbizh8f/O/DvA1FLPBiZkcrNMwt4qrSCz99wGcOHwOSWSLyqPdPKc5uOcMfcIjLTBudJ9rqjI6R98reLx9LQ3M5zm3S+JZFY9lRpBS3tQVYsHut3KVGlcPDJ3JJsZhZm8tjqg4SmXkQk1rR3BPn5O4e4akIuU0dn+F1OVCkcfGJmrFg8jrKqRlbv06m8RWLRq7uqOFJ3lhWLx/ldStQpHHx066wCcoYn89PVB/0uRUS68djqgxRmDeP6aSP9LiXqFA4+Sk0K8DcLSnh11wkqapr8LkdEwuw+fpp39p/iY1eNHdSn5u7J0PuJY8xHFpUQMOMnbx/0uxQRCfPoWwdISUzgQ/OHxhHR51M4+Kwgcxi3zR7Dk+sPU9fU6nc5IgKcON3Mc5uO8KEri8kenux3Ob5QOMSAe949gabWDn6hg+JEYsKjbx+gI+j4xNUT/C7FNwqHGDB1dAbvmZLPT94+qLO1ivjsdHMbv1xzmFtmFlCSm+Z3Ob5ROMSIe989kVNnWnlmQ6XfpYgMaU+sPUxDSzv3vnui36X4SuEQIxaOz2F2cRY//tN+OoI6KE7EDy3tHTzy1gGunpTHjMLBfRnQi1E4xAgz41PvnsChU028tP2Y3+WIDEm/2XSEqoYWPvnuoTvX0EnhEENumD6aCfnD+e/Xywlq60Ekqto7gnz/zX3MKMzg6kl5fpfjO4VDDAkkGJ9dMpndxxt0pTiRKHtu0xEOnWric0suGzLXbLgQhUOMuXXWGCbkD+c7r5Vp60EkSto6gvy/18uZUZjBkiF4qozuKBxiTPjWwx+09SASFc9tOsLhGm01hFM4xKBzWw+vautBJNJCWw1lzCzM1FZDGIVDDOrcethzooGXtmvrQSSSntt4hIqas3zu+snaagijcIhRt84aw6SRI/ivV/bQ3hH0uxyRQam5rYNvv7qXWUWZXDdVWw3hFA4xKpBg/PONU9hffYanSiv8LkdkUHps9UGO1jdz/81TtdVwHoVDDLth+ijmj83mW6+Ucaal3e9yRAaVuqZWvvdGOe+Zks/iiTqu4XwKhxhmZjxwyzRONrbw8J8O+F2OyKDy/Tf30dDSzv03T/W7lJikcIhx88Zmc/OM0Ty0ah/VDS1+lyMyKFTWNvHTtw/ywblFTB2d4Xc5MUnhEAe+eOMUmtuDfPvVvX6XIjIo/OfKPZjBF5Ze5ncpMUvhEAcm5I/gY4vG8sS6w+w4Wu93OSJxbf3BGn6z+Sh/f80ECjKH+V1OzFI4xInP33AZ2WnJPPj8DpzTgXEifdERdPzr8zsYk5nKp987tK/XcDEKhziROSyJf7lpKqWHavnN5iN+lyMSl3659hC7jp3my7dOJy050e9yYprCIY58cF4Rs4uz+N8v7qahuc3vckTiyqnGFr65cg+LJ+Zy84zRfpcT8xQOcSQhwfjqbZdzsrGF77xa5nc5InHlmyv30NTawf+67XId8NYLCoc4M7s4i+VXlvDo2wfYWlnndzkicWHN/lM8ub6Cv3vXOCaPSve7nLjQ53Aws2Ize8PMdpnZDjP7rNeeY2avmFmZ9z3bazcz+66ZlZvZVjObG/ZcK7z+ZWa2ov8/1uB2/81TyRuRwj8/s5U2nXdJ5IKa2zq4/9dbKclJ4ws3TPG7nLjRny2HduCfnHPTgEXAfWY2HbgfeM05Nxl4zbsPcDMw2fu6B/gBhMIEeBBYCCwAHuwMFOle5rAk/u32Gew+3sCP/rjP73JEYtq3Xt3LwVNNfP2OmQxLDvhdTtzoczg454455zZ6txuAXUAhsAx4zOv2GHC7d3sZ8DMXsgbIMrMC4EbgFedcjXOuFngFuKmvdQ0VSy8fzftmFfDd18opr2rwuxyRmLStsp4fr9rP8iuLWazrQl+SAZlzMLNxwBXAWmCUc+4YhAIE6DwPbiEQfnrRSq+tp3a5iK+8/3LSUgJ88ZmtOq23yHla2jv44jNbyBuRwgO3TPO7nLjT73AwsxHAr4HPOedOX6hrN23uAu3dvdY9ZlZqZqXV1dWXXuwgk5+ewleXzWDT4Tq+94aGl0TC/efKPew+3sB/3DGTzGFJfpcTd/oVDmaWRCgYHnfOPes1n/CGi/C+V3ntlUBx2MOLgKMXaO/COfeQc26+c25+fn5+f0ofNG6bPYbb54zhu6+XsfFwrd/liMSEt8pO8uM/HeCji0pYMm2U3+XEpf7srWTAI8Au59z/DVv0AtC5x9EK4Pmw9ru8vZYWAfXesNNKYKmZZXsT0Uu9Numlr94+g9EZqXzuyc006roPMsTVnmnln361mYn5w/nSLdP9Lidu9WfL4V3Ax4DrzGyz93UL8HXgBjMrA27w7gO8COwHyoEfA58GcM7VAF8D1ntfX/XapJcyUpP49vI5VNY28ZUXdvhdjohvnHM88Ow2as608p3lV2jvpH7o88lFnHNv0f18AcCSbvo74L4enutR4NG+1iJw5bgcPvPeSXz39XIWjMvhr68svviDRAaZR98+yB92HOeBm6cyozDT73Limo6QHkQ+e/1lXD0pjy8/v53tR3Rqbxla1h+s4T9e3MUN00dxz7UT/C4n7ikcBpFAgvGd5XPIG57Mvb/YQF1Tq98liURFVUMz9z2+kaLsYfzXX8/WuZMGgMJhkMkdkcL3PjKXE6eb+dxTm+kI6toPMri1tgf5zC83cbq5jR98dB4ZqdptdSAoHAahK0qy+cptl/Pmnmr+7fc7/S5HJGKcc3zpuW2sO1DDNz4wi2kFuh70QNHVLgapjywcy76qMzz69gHG5w3nrqvG+V2SyID7/pv7+NWGSv5xyWSWzdGJFQaSwmEQ+9L7pnG4JrR7a3F2Gu+dOvLiDxKJE7/bepRvrtzDsjlj+Pz1k/0uZ9DRsNIg1jlBPa0gg/t+uZFNOoJaBol39p3iC09vYf7YbL7xgVmagI4AhcMgNzwlkUf/9kryRqTwtz9Zz+7jFzr9lUjs21xRxyceW8/YnDQeums+qUk60C0SFA5DwKiMVB7/xEKGJQX46MPrOHDyjN8lifTJ7uOnWfHoOnJHpPCLTywkZ3iy3yUNWgqHIaI4J41ffGIBQef46MNrOXyqye+SRC5JeVUDH3tkHalJCTz+iYWMykj1u6RBTeEwhEwamc7PPr6AM63t3Pmj1ZRXNfpdkkiv7Dhaz1//aA3OwS/uXkhxTprfJQ16CochZkZhJk/es4iOIHzoR++w86jmICS2bTxcy4cfWkNqYgK/uvcqJo9K97ukIUHhMARNHZ3B059cRHJiAssfeoe1+0/5XZJIt/64t5qPPbyW7OHJPH3vVYzPG+53SUOGwmGImpA/gqc/eRV56Sl87JF1/GbTEb9LEvkLv1x7mI//dD3FOWk8/cmrKMrWUFI0KRyGsOKcNJ791GKuKMnic09t5juvlhE6s7qIf4JBx3+8uIv/+dw2rpmcxzOfWqzJZx8oHIa4rLRkfn73Qu6YW8i3Xt3Lpx/fSENzm99lyRBVe6aVjz+2nh+t2s9HF5Xw8F3zGZGiEzn4QeEgJCcm8F93zuZLt0zj5Z0nuO2/39bBchJ1WyrquPX/vcXq8lN87fYZfG3ZDBID+ojyi9a8AGBm/P21E3ji7xdxpqWd27/3Nr9Yc0jDTBJxwaDj4T/t584fvgPAr+49NJXsAAAMlUlEQVS9io8tGqtTYvhM4SB/YcH4HH73j1dz5bgcvvyb7fztT9Zz4nSz32XJIFVZ28TfPLyGf/v9Lq69LI/f/cPVzC7O8rssASxe/zKcP3++Ky0t9buMQSsYdPxi7SH+94u7SEkM8OX3TeOD84r015wMiI6g44l1h/n6S7sB+Nf3T+dOvb8izsw2OOfm96qvwkEuZH91I198ZisbDtVy5bhs/u32mUwZrYOQpO+2H6nnS7/ZzpaKOhZPzOUbH5ilI56jROEgAyoYdPxqQwX/8dJuGpvb+eiisfzDdZPIHZHid2kSR6pON/OtV8t4av1hcoan8OX3TWPZnDHaWogihYNERM2ZVr65cg9Pl1YwLCnAJ6+dwN3XjCctWbsaSs8amtt4aNV+Hv7TAdqDQT6ycCyfv+EyMofpWs/RpnCQiCqvauT//GE3L+88QXZaEh9/13juumocmWn6zy5/VnOmlZ+8fYCfrj5IQ3M7t84q4Is3TmFsrk6B4ReFg0TFxsO1fP+Ncl7dVcWIlEQ+dGUxH1s0lnE6/82QVl7VwM/fOcTTpZU0t3dw0+Wjue+9k5hRmOl3aUOewkGiatex0/zwj/v4/dZjtAcd116Wz98sKOG9U/NJSdRVuoaC5rYOXt11gsfXHOad/adIDiRw25wx3PvuCUwaqR0YYoXCQXxRdbqZJ9ZV8Mt1hzhxuoXMYUncOquAZXMKmTc2m0CCJh4Hk/aOIOsO1PD85qO8uO0YDS3tFGYN4yOLSvjQ/GLtsBCDFA7iq/aOIH8qP8lzG4/w8s7jNLcFyR2ezPXTRnH99FEsmpBDeqrmJ+JR/dk23tl3kld2VvHa7hPUNbWRlhzgphmjueOKIq6amKs/AmLYpYSDdjORAZcYSOC9U0by3ikjaWxp543dVby88wS/33aMp0orSEwwrijJYvHEPK4cl8OckiydXC1GnW5uY9PhOkoP1vBW+Um2VNQRdJCRmsiSaaNYOn0U756Srz3WBqGY2XIws5uA7wAB4GHn3Ncv1F9bDvGnpb2DDYdqeavsJG+Vn2TbkXqcgwSDKaMzmFmYwYzCTKYXZDB5VLp2dYyyuqZW9p5oZMfRenYcPc32I/XsOdFw7nc0qyiLaybncfWkPOaOzSZJJ8WLO3E3rGRmAWAvcANQCawHPuyc29nTYxQO8a+h86/SQ7VsOlzLjqOnqTnTem75yPQUJo0cwdjc4ZTkpDE2N42CzFQKMoeRn56i4YtL1N4RpLqxhWP1zRytO8vhmiYOn2ri4KkzlFed4WRjy7m+eSNSmFGYwRXF2cwfl82c4iyGa+su7sXjsNICoNw5tx/AzJ4ElgE9hoPEv/TUJK69LJ9rL8sHwDnH8dPN7Dx6mvKqRsqqGtlX3cjLO45zKiw0AAIJRu7wZPJGpJCXnkJ2WhLZaclkDksiPTWRjNTQ97SURIYnBxiWHGBYUoCUpACpiQkkJyaQFEggOZBAQoyHTDDoaO0I0tYRpKU99NXc1sHZ1g7OtnXQ1NrBmZZ2GprbaGhu53RzO/VNrdQ2tVHb1MrJxlaqG1qoOdNC8Ly/BfNGJFOSk8Z1U/OZPDKdSSNHMH1Mhi6uIzETDoVARdj9SmChT7WIT8yMgsxhFGQOY8m0UX+xrKG5jYqasxyrP8ux+maO1zdT3dDCycbQ18GTZ6hrauV0c/slv24gwUhMMJICCQQSjECCkWBGIAESLHQ7IQEMI8FCdRqAlynm1R7WROdnsHPu3G1cqN05R9CBwxEMQtA5gs7R4d1u7wjSEXS0e18d53+i90LmsCSyvMAszBrG7KJM8tNTGJ2ZypjMYYzOTKUkJ01bA9KjWHlndPenW5f/EWZ2D3APQElJSaRrkhiSnprE9DFJTB+TccF+7R1BzrR0cNr7K/psWztnWkJ/Xbe0d9Dc1kFzW5DW9iCtHaHv7cEg7R3eh3FHkI7OD+qg8z64Qx/anR/0Qce561y4c/+EPuzDWVh6dL7BzbyAOXc7dD8UQKGQ6gyoxMCfQyspkECKt8WTmhggJSmB1KQAacmdX4mkpyaSnpLEiNREDblJv8VKOFQCxWH3i4Cj53dyzj0EPAShOYfolCbxJDGQQGZagk7lIdJPsbK7wXpgspmNN7NkYDnwgs81iYgMWTGx5eCcazezzwArCe3K+qhzbofPZYmIDFkxEQ4AzrkXgRf9rkNERGJnWElERGKIwkFERLpQOIiISBcKBxER6ULhICIiXcTEiff6wsyqgUN9fHgecHIAyxkoquvSqK5Lo7ouzWCsa6xzLr83HeM2HPrDzEp7e2bCaFJdl0Z1XRrVdWmGel0aVhIRkS4UDiIi0sVQDYeH/C6gB6rr0qiuS6O6Ls2QrmtIzjmIiMiFDdUtBxERuYBBGw5mdqeZ7TCzoJnNP2/ZA2ZWbmZ7zOzGHh4/3szWmlmZmT3lnUp8oGt8ysw2e18HzWxzD/0Omtk2r1/EL5xtZl8xsyNhtd3SQ7+bvHVYbmb3R6Gub5rZbjPbambPmVlWD/2isr4u9vObWYr3Oy733kvjIlVL2GsWm9kbZrbLe/9/tps+7zGz+rDf779Gui7vdS/4e7GQ73rra6uZzY1CTVPC1sNmMzttZp87r09U1peZPWpmVWa2Pawtx8xe8T6HXjGz7B4eu8LrU2ZmKwakINd5hatB9gVMA6YAbwLzw9qnA1uAFGA8sA8IdPP4p4Hl3u0fAp+KcL3/BfxrD8sOAnlRXHdfAf7HRfoEvHU3AUj21un0CNe1FEj0bn8D+IZf66s3Pz/waeCH3u3lwFNR+N0VAHO92+nA3m7qeg/wu2i9n3r7ewFuAV4idKG8RcDaKNcXAI4TOhYg6usLuBaYC2wPa/s/wP3e7fu7e88DOcB+73u2dzu7v/UM2i0H59wu59yebhYtA550zrU45w4A5cCC8A4WuiDwdcAzXtNjwO2RqtV7vb8GnojUa0TAAqDcObffOdcKPElo3UaMc+5l51znRaLXELpioF968/MvI/TegdB7aYl1Xmw6Qpxzx5xzG73bDcAuQtdojwfLgJ+5kDVAlpkVRPH1lwD7nHN9Pbi2X5xzq4Ca85rD30M9fQ7dCLzinKtxztUCrwA39beeQRsOF1AIVITdr6Trf55coC7sg6i7PgPpGuCEc66sh+UOeNnMNnjX0Y6Gz3ib9o/2sCnbm/UYSR8n9Fdmd6Kxvnrz85/r472X6gm9t6LCG8a6AljbzeKrzGyLmb1kZpdHqaSL/V78fk8tp+c/0PxYXwCjnHPHIBT8wMhu+kRkvcXMxX76wsxeBUZ3s+hLzrnne3pYN23n77LVmz690ssaP8yFtxre5Zw7amYjgVfMbLf3V0afXagu4AfA1wj9zF8jNOT18fOfopvH9nvXt96sLzP7EtAOPN7D0wz4+uqu1G7aIvY+ulRmNgL4NfA559zp8xZvJDR00ujNJ/0GmByFsi72e/FzfSUDtwEPdLPYr/XVWxFZb3EdDs656/vwsEqgOOx+EXD0vD4nCW3SJnp/8XXXZ0BqNLNE4A5g3gWe46j3vcrMniM0pNGvD7verjsz+zHwu24W9WY9Dnhd3mTbrcAS5w24dvMcA76+utGbn7+zT6X3e86k67DBgDOzJELB8Lhz7tnzl4eHhXPuRTP7vpnlOecieh6hXvxeIvKe6qWbgY3OuRPnL/BrfXlOmFmBc+6YN8RW1U2fSkLzIp2KCM219stQHFZ6AVju7UkyntBfAOvCO3gfOm8AH/SaVgA9bYn01/XAbudcZXcLzWy4maV33iY0Kbu9u74D5bxx3r/q4fXWA5MttFdXMqFN8hciXNdNwL8AtznnmnroE6311Zuf/wVC7x0IvZde7ynQBoo3p/EIsMs593976DO6c+7DzBYQ+hw4FeG6evN7eQG4y9traRFQ3zmkEgU9br37sb7ChL+HevocWgksNbNsbwh4qdfWP5Gegffri9CHWiXQApwAVoYt+xKhPU32ADeHtb8IjPFuTyAUGuXAr4CUCNX5U+De89rGAC+G1bHF+9pBaHgl0uvu58A2YKv35iw4vy7v/i2E9obZF6W6ygmNrW72vn54fl3RXF/d/fzAVwmFF0Cq994p995LE6Kwjq4mNKSwNWw93QLc2/k+Az7jrZsthCb2F0ehrm5/L+fVZcD3vPW5jbC9DCNcWxqhD/vMsLaory9C4XQMaPM+u+4mNEf1GlDmfc/x+s4HHg577Me991k58HcDUY+OkBYRkS6G4rCSiIhchMJBRES6UDiIiEgXCgcREelC4SAiIl0oHEREpAuFg4iIdKFwEBGRLv4/Uw5YHte+pP0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# cem test 2\n",
    "# cem fits the x of f(x), in this approach, the x can be parameters of \n",
    "# the function f, therefore f can be dependent on other variables z like f(x,z). It works \n",
    "# as long as there are minimum of the function f.\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "N=100\n",
    "ratio=0.1\n",
    "num_itr=50\n",
    "\n",
    "f=lambda x: 3*x**2+x**4  # function of f(x)\n",
    "samples = np.random.normal(0,1,N) # x-value of f(x)\n",
    "print('true min: ', np.min(f(samples)))\n",
    "for i in range(num_itr):\n",
    "    # sample and select top 10% (max [-a:]or min [:a])\n",
    "    selected_samples = samples[np.array(f(samples)).argsort()[:int(N*ratio)]]\n",
    "    m=np.mean(selected_samples)\n",
    "    std=np.std(selected_samples)  # std decrease fastly\n",
    "#     print(std)\n",
    "    # fit the gaussian\n",
    "    samples=np.random.normal(m, std, N)\n",
    "print(f(selected_samples))\n",
    "\n",
    "'''display the function f(x) '''\n",
    "x=np.linspace(-10,10,N)\n",
    "y=f(x)\n",
    "plt.plot(x,y)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Additional results from above:\n",
    "* CE method can also fit on the parameters of a function, like x of f(x), which actually indicates the fitting can be done on any parameterized variables of the original variable (as criteria value) for CE, and on any re-parameterized variables for the previous parameterized variables, ie:\n",
    "\n",
    "    f->f(x)->f(g(z))->...\n",
    "    \n",
    "    where the Gaussian distribution can be fitted on f or x or z, ... respectively."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Notes for applying CE method in practice:\n",
    "* Although from above analysis, CE method can fit on any parameterized variables of the criteria function, there is actually a preference of CE method for which variables to fit on:\n",
    "\n",
    "    First, the main idea of CE method is **to transform the original optimization problem**, which is on the criteria function, **to an associated stochastic problem in another space**, which is the parameterized variable space, **for applying the sampling-based iterative method to get the optimum in original criteria function**.\n",
    "    \n",
    "    And the reason of applying this transformation is that **if directly applying the sampling-based interative evaluation on original criteria value function (like in CE method test 1), the probability of drawing a higg-valued smaple is presumably very low**, therefore finding the optimal samples from naive sampling is intractable. This is for the case that the criteria function is highly complicated and non-convex.\n",
    "    \n",
    "    Therefore, the sampling space transformation is to make the probability of drawing high-valued (criteria value) samples larger than direcly drawing from the criteria function. How to guarantee that, w.r.t. the transformation function?\n",
    "    \n",
    "    **So a linear transformation will probabily not help with changing the map of distribution/function values, therefore not help with making the probability of sampling large criteria values,** like g(x)=w\\*x+b.\n",
    "    \n",
    "    CE method also has disadvantages in **sensitivity to initialization of the Gaussian distribution**. A non-proper initialization could make CE method converge to local optimum easily. For example, in algorithm QT-Opt, if CE method isn't initialized properly, the several optimum reached by different runnings of CE method for the same case could make the Q-network applying CE not converge.\n",
    "    \n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Noisy CE method\n",
    "Simply adding an off-set on the standard deviation term in distribution of CE method to prevent it from decreasing too fast. It is basically a boosting of exploration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('true min: ', 2.816180325996992e-05)\n",
      "[2.81618033e-05 7.84159573e-03 2.21277088e-02 2.29479332e-02\n",
      " 3.39956756e-02 3.63122442e-02 4.04509899e-02 4.77346549e-02\n",
      " 5.10375036e-02 5.72493898e-02]\n",
      "[8.36304082e-05 8.38654406e-05 9.10359685e-05 2.15292065e-04\n",
      " 2.16907529e-04 2.23119129e-04 4.80424000e-04 1.82039641e-03\n",
      " 1.88579998e-03 2.10844679e-03]\n",
      "[9.31848595e-07 4.40038174e-06 5.67261022e-06 3.35705137e-05\n",
      " 4.29568315e-05 1.38643382e-04 2.80497025e-04 3.54977146e-04\n",
      " 3.84487082e-04 4.02956090e-04]\n",
      "[1.42217153e-08 5.75305431e-07 4.32870325e-06 2.62155296e-05\n",
      " 4.57140915e-05 1.70235951e-04 2.30567775e-04 2.79622631e-04\n",
      " 4.70014324e-04 5.75670854e-04]\n",
      "[1.42689591e-05 1.85447077e-05 3.74520285e-05 1.30825893e-04\n",
      " 1.87130405e-04 1.98527602e-04 1.99311729e-04 2.17112893e-04\n",
      " 2.50367761e-04 2.83030564e-04]\n",
      "[4.23016691e-06 1.61900257e-05 1.80599703e-05 2.59262006e-05\n",
      " 3.75359212e-05 4.83828089e-05 8.25788968e-05 2.68068747e-04\n",
      " 3.62467303e-04 3.63692132e-04]\n",
      "[1.34604182e-07 4.48231706e-07 5.27240097e-06 1.91937834e-05\n",
      " 2.27815076e-05 2.99961737e-05 4.80293500e-04 6.29068708e-04\n",
      " 6.87174197e-04 8.40109214e-04]\n",
      "[9.76329819e-05 1.34382069e-04 1.68772689e-04 2.83917935e-04\n",
      " 2.91877558e-04 3.03378848e-04 3.29107004e-04 3.82476395e-04\n",
      " 5.82130264e-04 6.78730896e-04]\n",
      "[2.02377523e-05 4.62723451e-05 9.29452462e-05 1.10043506e-04\n",
      " 1.16813789e-04 2.23976183e-04 3.34638777e-04 3.56431547e-04\n",
      " 5.61330508e-04 6.16714331e-04]\n",
      "[1.03780648e-06 1.86880951e-06 3.16140748e-06 1.62233697e-05\n",
      " 2.54224060e-05 3.02126081e-05 4.08869987e-05 5.08387450e-05\n",
      " 7.62840474e-05 1.16979192e-04]\n",
      "[5.29817565e-07 4.17499804e-05 9.29434990e-05 1.18867084e-04\n",
      " 1.56702056e-04 1.87102422e-04 2.33401423e-04 3.27341291e-04\n",
      " 4.31323514e-04 6.76943444e-04]\n",
      "[3.33641931e-05 7.85116187e-05 8.34445806e-05 1.29575902e-04\n",
      " 1.79789735e-04 1.99310233e-04 2.00733610e-04 2.65730034e-04\n",
      " 2.75491933e-04 3.89174851e-04]\n",
      "[1.60714168e-05 2.45533582e-05 6.84251388e-05 1.92949290e-04\n",
      " 2.09326873e-04 2.12052512e-04 3.14561973e-04 4.89153778e-04\n",
      " 5.45650559e-04 5.66545401e-04]\n",
      "[8.79764204e-07 9.37911342e-06 1.01062978e-05 3.61004119e-05\n",
      " 5.61415484e-05 6.36421310e-05 6.61550516e-05 1.54045754e-04\n",
      " 1.96151393e-04 1.97341552e-04]\n",
      "[0.00010686 0.00018792 0.00019301 0.00033549 0.000463   0.00051223\n",
      " 0.0010384  0.00137516 0.00139257 0.00140892]\n",
      "[7.58255300e-06 1.22748469e-05 2.10171417e-05 8.08693202e-05\n",
      " 1.07974736e-04 1.74184460e-04 2.00698170e-04 2.51146517e-04\n",
      " 2.74080155e-04 5.95198729e-04]\n",
      "[1.44192105e-05 2.56322607e-05 1.07439849e-04 1.29202762e-04\n",
      " 1.46671463e-04 1.74820723e-04 2.89859621e-04 4.71956996e-04\n",
      " 6.12834134e-04 6.67551145e-04]\n",
      "[1.99392621e-05 2.85423581e-05 3.76239733e-05 5.17202491e-05\n",
      " 7.32880593e-05 2.00822740e-04 5.29315874e-04 6.65974151e-04\n",
      " 7.93378725e-04 8.55057031e-04]\n",
      "[1.82565242e-06 1.24468510e-05 2.09280047e-05 5.72889439e-05\n",
      " 5.77254965e-05 1.23543184e-04 1.28470941e-04 1.34497525e-04\n",
      " 1.48976837e-04 1.67831573e-04]\n",
      "[1.22026110e-05 1.00137527e-04 1.18625365e-04 1.84330022e-04\n",
      " 3.28028530e-04 4.39822487e-04 6.27838345e-04 8.26646499e-04\n",
      " 1.00129017e-03 1.32373273e-03]\n",
      "[1.70540782e-05 4.88945805e-05 1.62308734e-04 1.87248175e-04\n",
      " 2.21895519e-04 2.89714872e-04 5.57218016e-04 5.57299908e-04\n",
      " 7.35587838e-04 1.32467393e-03]\n",
      "[1.39185099e-05 1.78352654e-05 6.01636879e-05 6.96822760e-05\n",
      " 2.00177683e-04 2.77963394e-04 5.68500847e-04 6.11957349e-04\n",
      " 8.95446482e-04 1.03190617e-03]\n",
      "[1.08089893e-06 2.93590412e-06 1.48779478e-05 4.58155203e-05\n",
      " 5.33952369e-05 7.40724040e-05 1.15647937e-04 1.75838727e-04\n",
      " 2.55425906e-04 3.84133305e-04]\n",
      "[2.59726115e-06 3.28538589e-06 3.20000818e-05 3.88226402e-05\n",
      " 3.91454712e-05 4.61003436e-05 1.07647923e-04 1.21624425e-04\n",
      " 1.41735610e-04 5.05452098e-04]\n",
      "[5.40213671e-06 8.14634918e-06 1.36311091e-05 2.20181951e-05\n",
      " 2.59723010e-05 9.84303303e-05 5.13123916e-04 5.44141636e-04\n",
      " 5.56513439e-04 5.91817262e-04]\n",
      "[1.41455244e-05 2.50581283e-05 1.00637544e-04 2.27647394e-04\n",
      " 4.07210545e-04 4.26146357e-04 5.47068430e-04 5.77059454e-04\n",
      " 5.79906969e-04 5.93881889e-04]\n",
      "[3.42389982e-08 2.92081471e-06 2.88648217e-05 3.69432463e-05\n",
      " 5.30057338e-05 1.35312682e-04 2.33111158e-04 4.60653477e-04\n",
      " 4.73540508e-04 5.44524483e-04]\n",
      "[5.23746463e-06 5.43598679e-06 5.26934745e-05 8.86805961e-05\n",
      " 4.34286470e-04 8.15672173e-04 8.29143073e-04 8.53777308e-04\n",
      " 1.00284367e-03 1.10148312e-03]\n",
      "[1.82143359e-05 3.42539644e-05 6.35654936e-05 8.14762600e-05\n",
      " 1.67566865e-04 2.31178402e-04 2.47132054e-04 2.71950493e-04\n",
      " 3.29458628e-04 3.65341247e-04]\n",
      "[2.21592030e-06 1.85552692e-05 9.60469997e-05 1.71511674e-04\n",
      " 1.82906864e-04 2.50148756e-04 2.64353791e-04 2.68594035e-04\n",
      " 2.70632785e-04 2.95969438e-04]\n",
      "[5.67217219e-06 4.17916580e-05 1.98557927e-04 2.81615044e-04\n",
      " 2.84162224e-04 6.60152514e-04 6.89191373e-04 6.99064412e-04\n",
      " 1.06383680e-03 1.22984735e-03]\n",
      "[1.18541791e-05 6.23852749e-05 8.71349552e-05 1.08268543e-04\n",
      " 2.05693294e-04 2.13485427e-04 2.96625239e-04 3.87945395e-04\n",
      " 5.47609082e-04 5.72450284e-04]\n",
      "[4.05001777e-06 4.48987773e-06 9.98652785e-06 6.78016928e-05\n",
      " 1.57124809e-04 1.61646706e-04 1.80458628e-04 2.58289487e-04\n",
      " 3.23599500e-04 5.52791078e-04]\n",
      "[9.49420397e-06 1.27378008e-05 2.23564523e-05 2.97190803e-05\n",
      " 4.78205303e-05 1.70588353e-04 1.79177847e-04 3.37851006e-04\n",
      " 4.90708365e-04 5.44138913e-04]\n",
      "[2.88016549e-06 1.15122986e-05 3.16456952e-05 4.49607906e-05\n",
      " 1.08897336e-04 1.44819500e-04 1.62212658e-04 1.71828809e-04\n",
      " 2.08851021e-04 2.13002343e-04]\n",
      "[1.94137322e-07 4.30385127e-07 1.01023750e-06 3.79303400e-06\n",
      " 4.11372499e-06 1.36567903e-05 1.98812664e-05 2.47900334e-05\n",
      " 4.73898748e-05 6.35876580e-05]\n",
      "[3.94781557e-06 1.56150762e-05 2.17343412e-05 9.67003630e-05\n",
      " 9.69149937e-05 1.61188157e-04 1.99873954e-04 2.09876325e-04\n",
      " 3.34517945e-04 3.65821647e-04]\n",
      "[3.49961697e-06 6.31033245e-06 9.95640739e-06 2.53745766e-05\n",
      " 1.40758394e-04 1.89004137e-04 2.09689278e-04 2.59378244e-04\n",
      " 4.35527416e-04 5.90150917e-04]\n",
      "[2.14739993e-06 2.22748651e-06 4.65224000e-06 1.27432063e-05\n",
      " 1.31159364e-05 1.42197252e-04 2.23348830e-04 2.97272030e-04\n",
      " 3.99566611e-04 6.06400623e-04]\n",
      "[7.32542995e-06 1.97871106e-05 3.84806742e-05 4.81128924e-05\n",
      " 7.32387271e-05 1.00751703e-04 1.48002245e-04 2.81946249e-04\n",
      " 3.01909568e-04 3.56979238e-04]\n",
      "[2.02805638e-06 8.31589407e-06 1.42569645e-05 3.17552416e-05\n",
      " 4.25639674e-05 5.59483644e-05 6.23844765e-05 1.32680426e-04\n",
      " 1.58107652e-04 1.78238799e-04]\n",
      "[0.00010568 0.00011728 0.00018002 0.00020274 0.00025227 0.0003804\n",
      " 0.00048542 0.00060927 0.00067321 0.00067909]\n",
      "[3.31757768e-05 4.27411000e-05 6.49965062e-05 9.24294462e-05\n",
      " 1.10672224e-04 1.44890985e-04 3.45211273e-04 6.02236736e-04\n",
      " 6.64191818e-04 7.49029974e-04]\n",
      "[4.55151950e-09 6.65332599e-08 4.35690785e-05 5.41489659e-05\n",
      " 9.71136190e-05 1.34247377e-04 1.50476763e-04 2.11028263e-04\n",
      " 2.39546018e-04 2.91389507e-04]\n",
      "[1.00519333e-06 1.45680024e-05 2.11037010e-05 2.54397638e-05\n",
      " 5.57230423e-05 3.26699370e-04 3.79674424e-04 4.58018814e-04\n",
      " 5.09888794e-04 6.69342493e-04]\n",
      "[5.65845115e-06 6.27732894e-05 7.63334883e-05 9.35972984e-05\n",
      " 1.49346993e-04 1.68902334e-04 2.08142848e-04 2.55454645e-04\n",
      " 3.11164664e-04 3.40561978e-04]\n",
      "[2.88534551e-05 3.34166979e-05 3.34542990e-05 6.05197815e-05\n",
      " 3.03908185e-04 3.19484905e-04 3.83825354e-04 4.15510714e-04\n",
      " 5.59103818e-04 6.12332934e-04]\n",
      "[1.95447389e-07 3.55593538e-07 1.75262533e-05 2.08494846e-05\n",
      " 5.80986183e-05 6.99997656e-05 9.93880959e-05 1.60812850e-04\n",
      " 2.64020148e-04 2.85657521e-04]\n",
      "[2.83088182e-06 1.69267762e-05 2.64760817e-05 6.07819897e-05\n",
      " 6.81757563e-05 8.79849582e-05 1.18185003e-04 1.20542190e-04\n",
      " 1.97931717e-04 2.34867213e-04]\n",
      "[4.01864949e-05 5.98428379e-05 1.81383475e-04 2.41719780e-04\n",
      " 4.52470801e-04 4.79586263e-04 4.87414237e-04 4.90343657e-04\n",
      " 6.44714746e-04 6.46735217e-04]\n",
      "[4.01864949e-05 5.98428379e-05 1.81383475e-04 2.41719780e-04\n",
      " 4.52470801e-04 4.79586263e-04 4.87414237e-04 4.90343657e-04\n",
      " 6.44714746e-04 6.46735217e-04]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd8XGed7/HPT6Nm2eqSbVnFPS5xi+3YjkkCxIlTCHE2kMUsEO8SNgTCLmUvu8mF14YLu3fhsnspd2khCQQIKYSEBEhwOiZxXOTeLblKbpKtYsmy6jz3jzkygyXZsqSZMyN936+XrJnnPDPz09F4vjrPc4o55xAREQmX4HcBIiISexQOIiLShcJBRES6UDiIiEgXCgcREelC4SAiIl0oHEREpAuFg4iIdKFwEBGRLhL9LqCv8vLy3Lhx4/wuQ0QkbmzYsOGkcy6/N33jNhzGjRtHaWmp32WIiMQNMzvU274aVhIRkS4UDiIi0oXCQUREulA4iIhIFwoHERHpQuEgIiJdKBxERKSLIRUOre1BfvDmPlbtrfa7FBGRS/bqzhM8tGof0bi885AKh6SA8dCqffx+6zG/SxERuWTPbTrCz9ccwswi/lpDKhzMjBmFmWw7Uu93KSIil2zbkXpmFmZG5bUuGg5m9qiZVZnZ9rC2HDN7xczKvO/ZXruZ2XfNrNzMtprZ3LDHrPD6l5nZirD2eWa2zXvMdy3CkTirKJO9JxpobuuI5MuIiAyouqZWDtc0MbMwKyqv15sth58CN53Xdj/wmnNuMvCadx/gZmCy93UP8AMIhQnwILAQWAA82BkoXp97wh53/msNqJmFmbQHHbuPN0TyZUREBtT2I6cBYmfLwTm3Cqg5r3kZ8Jh3+zHg9rD2n7mQNUCWmRUANwKvOOdqnHO1wCvATd6yDOfcOy40w/KzsOeKiJlFodTdVlkXyZcRERlQW4+EPrNiJhx6MMo5dwzA+z7Say8EKsL6VXptF2qv7KY9YsZkppIzPFnzDiISV7YfqackJ43MtKSovN5AT0h3N1/g+tDe/ZOb3WNmpWZWWl3dt91RzYyZhZlsrVQ4iEj82FpZz8yi6Gw1QN/D4YQ3JIT3vcprrwSKw/oVAUcv0l7UTXu3nHMPOefmO+fm5+f36noV3ZpZmElZVaMmpUUkLtSeaaWy9mzUhpSg7+HwAtC5x9EK4Pmw9ru8vZYWAfXesNNKYKmZZXsT0UuBld6yBjNb5O2ldFfYc0XMzKJMOoKOncdOR/qlRET6rXMYfFYshYOZPQG8A0wxs0ozuxv4OnCDmZUBN3j3AV4E9gPlwI+BTwM452qArwHrva+vem0AnwIe9h6zD3hpYH60nnWm73bNO4hIHOgMh8ujGA4XvUyoc+7DPSxa0k1fB9zXw/M8CjzaTXspMONidQykgsxU8kYka95BROLCtsp6xuWmkTksOpPRMMSOkO7UeaS0thxEJB5sO1LPjChuNcAQDQcIjd3tPdHA2VZNSotI7DrV2MKRurPMiuKeSjCEw2FGYSZBBzuPaetBRGJX53yDthyiZNa5I6UVDiISu7YrHKJrVEYK+ekpmpQWkZi2tbKe8XnDyUiN3mQ0DOFwMDNmF2WyWedYEpEY5Zxjc0Uds6M83wBDOBwA5hRnsb/6DPVNbX6XIiLSxbH6ZqoaWphTHJ3TdIcb4uEQOmv4Fm09iEgM2lIR+myaU5J9kZ4Db0iHw6ziTMz+/AsQEYklmyvqSA4kMK0gPeqvPaTDISM1iYn5I9iscBCRGLSpoo5pYzJISQxE/bWHdDhAaN5hc0UdoTN/iIjEhvaOINsq67nCh/kGUDgwuziLU97pcEVEYkVZVSNn2zp8mYwGhcO5VN6koSURiSGdw90KB59MGZ1OSmKCJqVFJKZsPlxHVloSY3PTfHn9IR8OSYEEZhZmalJaRGJK6OC3LELXQYu+IR8OENps236knraOoN+liIjQ2NLO3qoG34aUQOEAhCalW9qD7D7W4HcpIiJsq6zHOZhTonDwVWc6b66o9bkSEZE/T0bPLlI4+Kooexh5I5K1x5KIxITNFbWMzU0jZ3iybzUoHAidofWKkmw2HVY4iIi/nHNsPFzn28FvnRQOnnljszlw8gynGlv8LkVEhrDK2rNUN7Qwb1yOr3UoHDzzxobOerhRWw8i4qPSQzUAzPPhTKzhFA6emYWZJAXs3C9GRMQPGw7VMiIlkSmjo38m1nAKB09qUoAZhZlsPKQ9lkTEPxsO1XFFSRaBBH8OfuukcAgzrySbLZX1tLbrYDgRib6G5jb2HD/NXJ+HlEDh8Bfmjc2mtT3IjqP1fpciIkPQ5oo6gg7mj1M4xJTOSekNGloSER9sOFSLmX9nYg2ncAgzMiOV4pxhCgcR8cWGQ7VMGZVOemqS36UoHM43rySb0kO1ujKciERVR9Cx6XBdTAwpgcKhi3ljs6luaNGV4UQkqvaeaKCxpf3c8Lbf+hUOZvZ5M9thZtvN7AkzSzWz8Wa21szKzOwpM0v2+qZ498u95ePCnucBr32Pmd3Yvx+pf+aNDR2VqKElEYmmUu8zZ16Jv0dGd+pzOJhZIfCPwHzn3AwgACwHvgF8yzk3GagF7vYecjdQ65ybBHzL64eZTfcedzlwE/B9Mwv0ta7+mjI6neHJAYWDiETVxkO15KenUJwzzO9SgP4PKyUCw8wsEUgDjgHXAc94yx8DbvduL/Pu4y1fYqFLHC0DnnTOtTjnDgDlwIJ+1tVngYTQSfjWH9SR0iISPesP1jCvJNu3K7+dr8/h4Jw7AvwncJhQKNQDG4A651y7160SKPRuFwIV3mPbvf654e3dPOYvmNk9ZlZqZqXV1dV9Lf2iFozPYc+JBuqaWiP2GiIinY7UnaWy9iwLxsfGkBL0b1gpm9Bf/eOBMcBw4OZuunbu9tNdHLoLtHdtdO4h59x859z8/Pz8Sy+6lxaMz8E5KD2ooSURibz1B0IjFYMiHIDrgQPOuWrnXBvwLLAYyPKGmQCKgKPe7UqgGMBbngnUhLd38xhfzCnOIjmQwDoNLYlIFKw9UEN6aiLTCjL8LuWc/oTDYWCRmaV5cwdLgJ3AG8AHvT4rgOe92y949/GWv+5CBxO8ACz39mYaD0wG1vWjrn5LTQowpziLtQcUDiISeesOnOLKcTm+n2wvXH/mHNYSmljeCGzznush4F+AL5hZOaE5hUe8hzwC5HrtXwDu955nB/A0oWD5A3Cfc66jr3UNlAXjc9h+pJ4zLe0X7ywi0kcnG1vYV30mpoaUILS3UZ855x4EHjyveT/d7G3knGsG7uzhef4d+Pf+1DLQFozP4b/fKGfj4VqumRy5+Q0RGdpicb4BdIR0j+aOzSaQYKzT0JKIRNDaAzUMSwowY0ym36X8BYVDD0akJDJjTAZr9yscRCRy1h6oYe7YLJITY+vjOLaqiTELJ+SyuaKO5jbfp0BEZBCqb2pj9/HTLByf63cpXSgcLmDBuBxaO4JsqajzuxQRGYRKD9XgXOzNN4DC4YKuHJeDGZp3EJGIWHeghuRAQkxc3Od8CocLyExLYsqodB3vICIRsfZADbOLM0lN8u1coz1SOFzEogm5lB6qobU96HcpIjKINDS3se1IfUzON4DC4aIWT8yluS3IZs07iMgAWn+who6gY/EkhUNcWjghlwSDt8tP+l2KiAwib5efIjkxgbklsXHlt/MpHC4ic1gSMwozeWffKb9LEZFBZPW+U8wfmx2T8w2gcOiVxRPz2FRRS1OrzrMkIv1Xc6aVXcdO865JeX6X0iOFQy8snphLW4djva7vICIDoHMk4qqJsTnfAAqHXpk/LpukgLF6n+YdRKT/Vu87yYiURGYVxtb5lMIpHHohLTmRK0qyWV2ueQcR6b/V+06xcHwOiYHY/QiO3cpizOKJuWw/Wk99U5vfpYhIHDtad5YDJ8/E9JASKBx6bfHEPJyDNQe09SAifdc537B4YuxORoPCodfmFGcxLCnAah3vICL98Pa+k+QMT2bq6HS/S7kghUMvJScmcOX4HN7W8Q4i0kfOOd7Zd4qrJuSSEEPXi+6OwuESXD0pl/KqRo7Vn/W7FBGJQ/uqGzlW3xzTxzd0UjhcgmsvC11L+k97NbQkIpfuj95nx7WXKRwGlSmj0hmVkcIfy6r9LkVE4tCqvdVMyB9OUXaa36VclMLhEpgZ10zO562yk3QEnd/liEgcaW7rYO2BU1w7Od/vUnpF4XCJrr0sn/qzbWyt1Cm8RaT31h+sobktyLsvUzgMSldPysMMVmneQUQuwaq91SQHElg4IfauF90dhcMlyhmezMzCTFZp3kFELsGqvSeZPy6btOREv0vpFYVDH1w7OZ/NFXXUn9WpNETk4o7XN7PnRMO5PR7jgcKhD669LJ+OoNPR0iLSK50jDfEyGQ0Khz65oiSLESmJrCpTOIjIxf2p7CT56SlMK4jtU2aEUzj0QVIggcUTc1m1txrntEuriPSsI+h4q6yaaybnYRbbp8wI169wMLMsM3vGzHab2S4zu8rMcszsFTMr875ne33NzL5rZuVmttXM5oY9zwqvf5mZrejvDxUN75kykiN1ZymvavS7FBGJYVsq66htaoubXVg79XfL4TvAH5xzU4HZwC7gfuA159xk4DXvPsDNwGTv6x7gBwBmlgM8CCwEFgAPdgZKLLtu6kgAXttd5XMlIhLLXt9VRSDBeM9lI/0u5ZL0ORzMLAO4FngEwDnX6pyrA5YBj3ndHgNu924vA37mQtYAWWZWANwIvOKcq3HO1QKvADf1ta5oGZ2ZyuVjMnh9l8JBRHr22u4q5o/NJjMtye9SLkl/thwmANXAT8xsk5k9bGbDgVHOuWMA3vfOuCwEKsIeX+m19dQe866bOpLSQzXUNbX6XYqIxKCjdWfZdez0uZGGeNKfcEgE5gI/cM5dAZzhz0NI3eluJsZdoL3rE5jdY2alZlZaXe3/QWjXTR1J0MEf9/pfi4jEnte9Yecl04ZWOFQClc65td79ZwiFxQlvuAjve1VY/+KwxxcBRy/Q3oVz7iHn3Hzn3Pz8fP8nd2YXZZE7PJnXNLQkIt14fXcVJTlpTMwf4Xcpl6zP4eCcOw5UmNkUr2kJsBN4Aejc42gF8Lx3+wXgLm+vpUVAvTfstBJYambZ3kT0Uq8t5iUkGO+dOpI391TR3hH0uxwRiSFnWzt4u/wk100dGVe7sHbq70k+/gF43MySgf3A3xEKnKfN7G7gMHCn1/dF4BagHGjy+uKcqzGzrwHrvX5fdc7V9LOuqFkydSTPbKhkw6FaFk7I9bscEYkRq/edpKU9GJdDStDPcHDObQbmd7NoSTd9HXBfD8/zKPBof2rxy9WT80gKGK/vrlI4iMg5r+2uYnhygAXj4+MsrOfTEdL9lJ6axMLxuecmnkREnHO8sbuKaybnk5IY8LucPlE4DIDrpo6krKqRgyfP+F2KiMSAHUdPc6y+mevidEgJFA4D4obpowBYueO4z5WISCxYueM4CRaak4xXCocBUJyTxozCDIWDiAChcFgwPofcESl+l9JnCocBcuP00Ww8XMeJ081+lyIiPtpf3cjeE43cePlov0vpF4XDALlxRuiN8PLOEz5XIiJ+Wrkj9BmwVOEgAJNHjmBC3nBe1tCSyJC2csdxZhVlUpg1zO9S+kXhMEDMjKWXj+adfaeob9K1pUWGouP1zWyuqIv7ISVQOAyoGy8fRXvQ8dpuDS2JDEUv7wyNHNx4+SifK+k/hcMAml2UxeiMVO21JDJErdxxnIn5w5k0Mn6uFd0ThcMASkgwll4+ij/uraaptd3vckQkiuqaWlmzv2ZQDCmBwmHA3TRjNM1tQd7YrWs8iAwlK3ccpyPouHlGgd+lDAiFwwBbOD6X/PQUfrul20tSiMgg9dstxxibGzogdjBQOAywQIJxy4zRvLGnisYWDS2JDAUnG1tYve8kt84qiMtrN3RH4RAB7589hpb2IK/qgDiRIeGl7ccJutD//cFC4RABc0uyKchM1dCSyBDx2y1HmTRyBFNGxf9eSp0UDhGQkGC8b2YBq8qqdUCcyCB34nQz6w/WDKohJVA4RMz7Z4+hrcOxcqeOeRAZzH6/9RjOwa2zBs+QEigcImZWUSYlOWkaWhIZ5H679SjTCjKYNHKE36UMKIVDhJgZ75tVwOp9pzjV2OJ3OSISAZW1TWw6XMetswbHsQ3hFA4RdNvsMXQEHb/beszvUkQkAp7fHBoZeP8gG1IChUNETSvIYFpBBs9urPS7FBEZYM45fr2xkgXjcijJTfO7nAGncIiwD8wtZEtlPeVVDX6XIiIDaEtlPfurz3DH3EK/S4kIhUOE3TZnDAkGz2484ncpIjKAnt1YSXJiArcMwvkGUDhE3Mj0VK69LJ/nNh0hGHR+lyMiA6C1PcgLW46ydPooMlKT/C4nIhQOUXDH3CKO1TezZv8pv0sRkQHwxp4q6pra+MDcIr9LiRiFQxQsnT6K9JREfq2hJZFB4dmNleSNSOGayXl+lxIxCocoSE0K8L5ZBby0/ZguAiQS52rPtPL67ipunzOGxMDg/QgdvD9ZjLljbhFNrR28uE2n0xCJZ89vPkJbh+OvBuleSp36HQ5mFjCzTWb2O+/+eDNba2ZlZvaUmSV77Sne/XJv+biw53jAa99jZjf2t6ZYdOW4bCbkDefJdYf9LkVE+sg5x5PrK5hZmMnlYzL9LieiBmLL4bPArrD73wC+5ZybDNQCd3vtdwO1zrlJwLe8fpjZdGA5cDlwE/B9MwsMQF0xxcxYvqCY0kO1lJ3QMQ8i8WhzRR27jzfw4QUlfpcScf0KBzMrAt4HPOzdN+A64Bmvy2PA7d7tZd59vOVLvP7LgCedcy3OuQNAObCgP3XFqg/MLSIpYDyxrsLvUkSkD55Yd5i05AC3zRl8p8s4X3+3HL4N/DMQ9O7nAnXOuc5Z10qgc2CuEKgA8JbXe/3PtXfzmEEld0QKS6eP5tlNlTS3dfhdjohcgobmNn675RjvnzWGESmJfpcTcX0OBzO7Fahyzm0Ib+6mq7vIsgs95vzXvMfMSs2stLq6+pLqjRXLFxRT19TGyh2amBaJJy9sOcrZtg6WLyj2u5So6M+Ww7uA28zsIPAkoeGkbwNZZtYZq0VA5wUNKoFiAG95JlAT3t7NY/6Cc+4h59x859z8/Pz8fpTun3dNzKM4ZxhPamhJJK48ua6CqaPTmVOc5XcpUdHncHDOPeCcK3LOjSM0ofy6c+4jwBvAB71uK4DnvdsvePfxlr/unHNe+3Jvb6bxwGRgXV/rinUJCcbyK0t4Z/8pDpw843c5ItIL24/Us+1IPR9eUDKoLgV6IZE4zuFfgC+YWTmhOYVHvPZHgFyv/QvA/QDOuR3A08BO4A/Afc65QT0gf+e8IhITjMfXHPK7FBHphV+sOURqUgK3zxmU06HdGpBZFefcm8Cb3u39dLO3kXOuGbizh8f/O/DvA1FLPBiZkcrNMwt4qrSCz99wGcOHwOSWSLyqPdPKc5uOcMfcIjLTBudJ9rqjI6R98reLx9LQ3M5zm3S+JZFY9lRpBS3tQVYsHut3KVGlcPDJ3JJsZhZm8tjqg4SmXkQk1rR3BPn5O4e4akIuU0dn+F1OVCkcfGJmrFg8jrKqRlbv06m8RWLRq7uqOFJ3lhWLx/ldStQpHHx066wCcoYn89PVB/0uRUS68djqgxRmDeP6aSP9LiXqFA4+Sk0K8DcLSnh11wkqapr8LkdEwuw+fpp39p/iY1eNHdSn5u7J0PuJY8xHFpUQMOMnbx/0uxQRCfPoWwdISUzgQ/OHxhHR51M4+Kwgcxi3zR7Dk+sPU9fU6nc5IgKcON3Mc5uO8KEri8kenux3Ob5QOMSAe949gabWDn6hg+JEYsKjbx+gI+j4xNUT/C7FNwqHGDB1dAbvmZLPT94+qLO1ivjsdHMbv1xzmFtmFlCSm+Z3Ob5ROMSIe989kVNnWnlmQ6XfpYgMaU+sPUxDSzv3vnui36X4SuEQIxaOz2F2cRY//tN+OoI6KE7EDy3tHTzy1gGunpTHjMLBfRnQi1E4xAgz41PvnsChU028tP2Y3+WIDEm/2XSEqoYWPvnuoTvX0EnhEENumD6aCfnD+e/Xywlq60Ekqto7gnz/zX3MKMzg6kl5fpfjO4VDDAkkGJ9dMpndxxt0pTiRKHtu0xEOnWric0suGzLXbLgQhUOMuXXWGCbkD+c7r5Vp60EkSto6gvy/18uZUZjBkiF4qozuKBxiTPjWwx+09SASFc9tOsLhGm01hFM4xKBzWw+vautBJNJCWw1lzCzM1FZDGIVDDOrcethzooGXtmvrQSSSntt4hIqas3zu+snaagijcIhRt84aw6SRI/ivV/bQ3hH0uxyRQam5rYNvv7qXWUWZXDdVWw3hFA4xKpBg/PONU9hffYanSiv8LkdkUHps9UGO1jdz/81TtdVwHoVDDLth+ijmj83mW6+Ucaal3e9yRAaVuqZWvvdGOe+Zks/iiTqu4XwKhxhmZjxwyzRONrbw8J8O+F2OyKDy/Tf30dDSzv03T/W7lJikcIhx88Zmc/OM0Ty0ah/VDS1+lyMyKFTWNvHTtw/ywblFTB2d4Xc5MUnhEAe+eOMUmtuDfPvVvX6XIjIo/OfKPZjBF5Ze5ncpMUvhEAcm5I/gY4vG8sS6w+w4Wu93OSJxbf3BGn6z+Sh/f80ECjKH+V1OzFI4xInP33AZ2WnJPPj8DpzTgXEifdERdPzr8zsYk5nKp987tK/XcDEKhziROSyJf7lpKqWHavnN5iN+lyMSl3659hC7jp3my7dOJy050e9yYprCIY58cF4Rs4uz+N8v7qahuc3vckTiyqnGFr65cg+LJ+Zy84zRfpcT8xQOcSQhwfjqbZdzsrGF77xa5nc5InHlmyv30NTawf+67XId8NYLCoc4M7s4i+VXlvDo2wfYWlnndzkicWHN/lM8ub6Cv3vXOCaPSve7nLjQ53Aws2Ize8PMdpnZDjP7rNeeY2avmFmZ9z3bazcz+66ZlZvZVjObG/ZcK7z+ZWa2ov8/1uB2/81TyRuRwj8/s5U2nXdJ5IKa2zq4/9dbKclJ4ws3TPG7nLjRny2HduCfnHPTgEXAfWY2HbgfeM05Nxl4zbsPcDMw2fu6B/gBhMIEeBBYCCwAHuwMFOle5rAk/u32Gew+3sCP/rjP73JEYtq3Xt3LwVNNfP2OmQxLDvhdTtzoczg454455zZ6txuAXUAhsAx4zOv2GHC7d3sZ8DMXsgbIMrMC4EbgFedcjXOuFngFuKmvdQ0VSy8fzftmFfDd18opr2rwuxyRmLStsp4fr9rP8iuLWazrQl+SAZlzMLNxwBXAWmCUc+4YhAIE6DwPbiEQfnrRSq+tp3a5iK+8/3LSUgJ88ZmtOq23yHla2jv44jNbyBuRwgO3TPO7nLjT73AwsxHAr4HPOedOX6hrN23uAu3dvdY9ZlZqZqXV1dWXXuwgk5+ewleXzWDT4Tq+94aGl0TC/efKPew+3sB/3DGTzGFJfpcTd/oVDmaWRCgYHnfOPes1n/CGi/C+V3ntlUBx2MOLgKMXaO/COfeQc26+c25+fn5+f0ofNG6bPYbb54zhu6+XsfFwrd/liMSEt8pO8uM/HeCji0pYMm2U3+XEpf7srWTAI8Au59z/DVv0AtC5x9EK4Pmw9ru8vZYWAfXesNNKYKmZZXsT0Uu9Numlr94+g9EZqXzuyc006roPMsTVnmnln361mYn5w/nSLdP9Lidu9WfL4V3Ax4DrzGyz93UL8HXgBjMrA27w7gO8COwHyoEfA58GcM7VAF8D1ntfX/XapJcyUpP49vI5VNY28ZUXdvhdjohvnHM88Ow2as608p3lV2jvpH7o88lFnHNv0f18AcCSbvo74L4enutR4NG+1iJw5bgcPvPeSXz39XIWjMvhr68svviDRAaZR98+yB92HOeBm6cyozDT73Limo6QHkQ+e/1lXD0pjy8/v53tR3Rqbxla1h+s4T9e3MUN00dxz7UT/C4n7ikcBpFAgvGd5XPIG57Mvb/YQF1Tq98liURFVUMz9z2+kaLsYfzXX8/WuZMGgMJhkMkdkcL3PjKXE6eb+dxTm+kI6toPMri1tgf5zC83cbq5jR98dB4ZqdptdSAoHAahK0qy+cptl/Pmnmr+7fc7/S5HJGKcc3zpuW2sO1DDNz4wi2kFuh70QNHVLgapjywcy76qMzz69gHG5w3nrqvG+V2SyID7/pv7+NWGSv5xyWSWzdGJFQaSwmEQ+9L7pnG4JrR7a3F2Gu+dOvLiDxKJE7/bepRvrtzDsjlj+Pz1k/0uZ9DRsNIg1jlBPa0gg/t+uZFNOoJaBol39p3iC09vYf7YbL7xgVmagI4AhcMgNzwlkUf/9kryRqTwtz9Zz+7jFzr9lUjs21xRxyceW8/YnDQeums+qUk60C0SFA5DwKiMVB7/xEKGJQX46MPrOHDyjN8lifTJ7uOnWfHoOnJHpPCLTywkZ3iy3yUNWgqHIaI4J41ffGIBQef46MNrOXyqye+SRC5JeVUDH3tkHalJCTz+iYWMykj1u6RBTeEwhEwamc7PPr6AM63t3Pmj1ZRXNfpdkkiv7Dhaz1//aA3OwS/uXkhxTprfJQ16CochZkZhJk/es4iOIHzoR++w86jmICS2bTxcy4cfWkNqYgK/uvcqJo9K97ukIUHhMARNHZ3B059cRHJiAssfeoe1+0/5XZJIt/64t5qPPbyW7OHJPH3vVYzPG+53SUOGwmGImpA/gqc/eRV56Sl87JF1/GbTEb9LEvkLv1x7mI//dD3FOWk8/cmrKMrWUFI0KRyGsOKcNJ791GKuKMnic09t5juvlhE6s7qIf4JBx3+8uIv/+dw2rpmcxzOfWqzJZx8oHIa4rLRkfn73Qu6YW8i3Xt3Lpx/fSENzm99lyRBVe6aVjz+2nh+t2s9HF5Xw8F3zGZGiEzn4QeEgJCcm8F93zuZLt0zj5Z0nuO2/39bBchJ1WyrquPX/vcXq8lN87fYZfG3ZDBID+ojyi9a8AGBm/P21E3ji7xdxpqWd27/3Nr9Yc0jDTBJxwaDj4T/t584fvgPAr+49NJXsAAAMlUlEQVS9io8tGqtTYvhM4SB/YcH4HH73j1dz5bgcvvyb7fztT9Zz4nSz32XJIFVZ28TfPLyGf/v9Lq69LI/f/cPVzC7O8rssASxe/zKcP3++Ky0t9buMQSsYdPxi7SH+94u7SEkM8OX3TeOD84r015wMiI6g44l1h/n6S7sB+Nf3T+dOvb8izsw2OOfm96qvwkEuZH91I198ZisbDtVy5bhs/u32mUwZrYOQpO+2H6nnS7/ZzpaKOhZPzOUbH5ilI56jROEgAyoYdPxqQwX/8dJuGpvb+eiisfzDdZPIHZHid2kSR6pON/OtV8t4av1hcoan8OX3TWPZnDHaWogihYNERM2ZVr65cg9Pl1YwLCnAJ6+dwN3XjCctWbsaSs8amtt4aNV+Hv7TAdqDQT6ycCyfv+EyMofpWs/RpnCQiCqvauT//GE3L+88QXZaEh9/13juumocmWn6zy5/VnOmlZ+8fYCfrj5IQ3M7t84q4Is3TmFsrk6B4ReFg0TFxsO1fP+Ncl7dVcWIlEQ+dGUxH1s0lnE6/82QVl7VwM/fOcTTpZU0t3dw0+Wjue+9k5hRmOl3aUOewkGiatex0/zwj/v4/dZjtAcd116Wz98sKOG9U/NJSdRVuoaC5rYOXt11gsfXHOad/adIDiRw25wx3PvuCUwaqR0YYoXCQXxRdbqZJ9ZV8Mt1hzhxuoXMYUncOquAZXMKmTc2m0CCJh4Hk/aOIOsO1PD85qO8uO0YDS3tFGYN4yOLSvjQ/GLtsBCDFA7iq/aOIH8qP8lzG4/w8s7jNLcFyR2ezPXTRnH99FEsmpBDeqrmJ+JR/dk23tl3kld2VvHa7hPUNbWRlhzgphmjueOKIq6amKs/AmLYpYSDdjORAZcYSOC9U0by3ikjaWxp543dVby88wS/33aMp0orSEwwrijJYvHEPK4cl8OckiydXC1GnW5uY9PhOkoP1vBW+Um2VNQRdJCRmsiSaaNYOn0U756Srz3WBqGY2XIws5uA7wAB4GHn3Ncv1F9bDvGnpb2DDYdqeavsJG+Vn2TbkXqcgwSDKaMzmFmYwYzCTKYXZDB5VLp2dYyyuqZW9p5oZMfRenYcPc32I/XsOdFw7nc0qyiLaybncfWkPOaOzSZJJ8WLO3E3rGRmAWAvcANQCawHPuyc29nTYxQO8a+h86/SQ7VsOlzLjqOnqTnTem75yPQUJo0cwdjc4ZTkpDE2N42CzFQKMoeRn56i4YtL1N4RpLqxhWP1zRytO8vhmiYOn2ri4KkzlFed4WRjy7m+eSNSmFGYwRXF2cwfl82c4iyGa+su7sXjsNICoNw5tx/AzJ4ElgE9hoPEv/TUJK69LJ9rL8sHwDnH8dPN7Dx6mvKqRsqqGtlX3cjLO45zKiw0AAIJRu7wZPJGpJCXnkJ2WhLZaclkDksiPTWRjNTQ97SURIYnBxiWHGBYUoCUpACpiQkkJyaQFEggOZBAQoyHTDDoaO0I0tYRpKU99NXc1sHZ1g7OtnXQ1NrBmZZ2GprbaGhu53RzO/VNrdQ2tVHb1MrJxlaqG1qoOdNC8Ly/BfNGJFOSk8Z1U/OZPDKdSSNHMH1Mhi6uIzETDoVARdj9SmChT7WIT8yMgsxhFGQOY8m0UX+xrKG5jYqasxyrP8ux+maO1zdT3dDCycbQ18GTZ6hrauV0c/slv24gwUhMMJICCQQSjECCkWBGIAESLHQ7IQEMI8FCdRqAlynm1R7WROdnsHPu3G1cqN05R9CBwxEMQtA5gs7R4d1u7wjSEXS0e18d53+i90LmsCSyvMAszBrG7KJM8tNTGJ2ZypjMYYzOTKUkJ01bA9KjWHlndPenW5f/EWZ2D3APQElJSaRrkhiSnprE9DFJTB+TccF+7R1BzrR0cNr7K/psWztnWkJ/Xbe0d9Dc1kFzW5DW9iCtHaHv7cEg7R3eh3FHkI7OD+qg8z64Qx/anR/0Qce561y4c/+EPuzDWVh6dL7BzbyAOXc7dD8UQKGQ6gyoxMCfQyspkECKt8WTmhggJSmB1KQAacmdX4mkpyaSnpLEiNREDblJv8VKOFQCxWH3i4Cj53dyzj0EPAShOYfolCbxJDGQQGZagk7lIdJPsbK7wXpgspmNN7NkYDnwgs81iYgMWTGx5eCcazezzwArCe3K+qhzbofPZYmIDFkxEQ4AzrkXgRf9rkNERGJnWElERGKIwkFERLpQOIiISBcKBxER6ULhICIiXcTEiff6wsyqgUN9fHgecHIAyxkoquvSqK5Lo7ouzWCsa6xzLr83HeM2HPrDzEp7e2bCaFJdl0Z1XRrVdWmGel0aVhIRkS4UDiIi0sVQDYeH/C6gB6rr0qiuS6O6Ls2QrmtIzjmIiMiFDdUtBxERuYBBGw5mdqeZ7TCzoJnNP2/ZA2ZWbmZ7zOzGHh4/3szWmlmZmT3lnUp8oGt8ysw2e18HzWxzD/0Omtk2r1/EL5xtZl8xsyNhtd3SQ7+bvHVYbmb3R6Gub5rZbjPbambPmVlWD/2isr4u9vObWYr3Oy733kvjIlVL2GsWm9kbZrbLe/9/tps+7zGz+rDf779Gui7vdS/4e7GQ73rra6uZzY1CTVPC1sNmMzttZp87r09U1peZPWpmVWa2Pawtx8xe8T6HXjGz7B4eu8LrU2ZmKwakINd5hatB9gVMA6YAbwLzw9qnA1uAFGA8sA8IdPP4p4Hl3u0fAp+KcL3/BfxrD8sOAnlRXHdfAf7HRfoEvHU3AUj21un0CNe1FEj0bn8D+IZf66s3Pz/waeCH3u3lwFNR+N0VAHO92+nA3m7qeg/wu2i9n3r7ewFuAV4idKG8RcDaKNcXAI4TOhYg6usLuBaYC2wPa/s/wP3e7fu7e88DOcB+73u2dzu7v/UM2i0H59wu59yebhYtA550zrU45w4A5cCC8A4WuiDwdcAzXtNjwO2RqtV7vb8GnojUa0TAAqDcObffOdcKPElo3UaMc+5l51znRaLXELpioF968/MvI/TegdB7aYl1Xmw6Qpxzx5xzG73bDcAuQtdojwfLgJ+5kDVAlpkVRPH1lwD7nHN9Pbi2X5xzq4Ca85rD30M9fQ7dCLzinKtxztUCrwA39beeQRsOF1AIVITdr6Trf55coC7sg6i7PgPpGuCEc66sh+UOeNnMNnjX0Y6Gz3ib9o/2sCnbm/UYSR8n9Fdmd6Kxvnrz85/r472X6gm9t6LCG8a6AljbzeKrzGyLmb1kZpdHqaSL/V78fk8tp+c/0PxYXwCjnHPHIBT8wMhu+kRkvcXMxX76wsxeBUZ3s+hLzrnne3pYN23n77LVmz690ssaP8yFtxre5Zw7amYjgVfMbLf3V0afXagu4AfA1wj9zF8jNOT18fOfopvH9nvXt96sLzP7EtAOPN7D0wz4+uqu1G7aIvY+ulRmNgL4NfA559zp8xZvJDR00ujNJ/0GmByFsi72e/FzfSUDtwEPdLPYr/XVWxFZb3EdDs656/vwsEqgOOx+EXD0vD4nCW3SJnp/8XXXZ0BqNLNE4A5g3gWe46j3vcrMniM0pNGvD7verjsz+zHwu24W9WY9Dnhd3mTbrcAS5w24dvMcA76+utGbn7+zT6X3e86k67DBgDOzJELB8Lhz7tnzl4eHhXPuRTP7vpnlOecieh6hXvxeIvKe6qWbgY3OuRPnL/BrfXlOmFmBc+6YN8RW1U2fSkLzIp2KCM219stQHFZ6AVju7UkyntBfAOvCO3gfOm8AH/SaVgA9bYn01/XAbudcZXcLzWy4maV33iY0Kbu9u74D5bxx3r/q4fXWA5MttFdXMqFN8hciXNdNwL8AtznnmnroE6311Zuf/wVC7x0IvZde7ynQBoo3p/EIsMs593976DO6c+7DzBYQ+hw4FeG6evN7eQG4y9traRFQ3zmkEgU9br37sb7ChL+HevocWgksNbNsbwh4qdfWP5Gegffri9CHWiXQApwAVoYt+xKhPU32ADeHtb8IjPFuTyAUGuXAr4CUCNX5U+De89rGAC+G1bHF+9pBaHgl0uvu58A2YKv35iw4vy7v/i2E9obZF6W6ygmNrW72vn54fl3RXF/d/fzAVwmFF0Cq994p995LE6Kwjq4mNKSwNWw93QLc2/k+Az7jrZsthCb2F0ehrm5/L+fVZcD3vPW5jbC9DCNcWxqhD/vMsLaory9C4XQMaPM+u+4mNEf1GlDmfc/x+s4HHg577Me991k58HcDUY+OkBYRkS6G4rCSiIhchMJBRES6UDiIiEgXCgcREelC4SAiIl0oHEREpAuFg4iIdKFwEBGRLv4/Uw5YHte+pP0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# constant noise on std\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "N=100\n",
    "ratio=0.1\n",
    "num_itr=50\n",
    "std_noise = 0.1\n",
    "\n",
    "f=lambda x: 3*x**2+x**4  # function of f(x)\n",
    "samples = np.random.normal(0,1,N) # x-value of f(x)\n",
    "print('true min: ', np.min(f(samples)))\n",
    "for i in range(num_itr):\n",
    "    # sample and select top 10% (max [-a:]or min [:a])\n",
    "    selected_samples = samples[np.array(f(samples)).argsort()[:int(N*ratio)]]\n",
    "    m=np.mean(selected_samples)\n",
    "    std=np.std(selected_samples)+std_noise  # constant noise on std\n",
    "#     print(std)\n",
    "    # fit the gaussian\n",
    "    samples=np.random.normal(m, std, N)\n",
    "    print(f(selected_samples))\n",
    "\n",
    "'''display the function f(x) '''\n",
    "x=np.linspace(-10,10,N)\n",
    "y=f(x)\n",
    "plt.plot(x,y)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('true min: ', 8.01845127993259e-06)\n",
      "[8.01845128e-06 9.45632783e-04 1.67819028e-03 2.49221560e-03\n",
      " 1.56548240e-02 1.71088181e-02 2.79620333e-02 2.85924118e-02\n",
      " 3.05718850e-02 3.14230746e-02]\n",
      "[0.00017539 0.00029455 0.0003059  0.00033824 0.00042543 0.00051511\n",
      " 0.0007302  0.00075642 0.00134831 0.00145118]\n",
      "[3.20710112e-08 1.26547507e-06 4.36988787e-05 6.12360916e-05\n",
      " 1.08006425e-04 1.13460867e-04 1.25196559e-04 2.21454615e-04\n",
      " 2.60699643e-04 2.76565422e-04]\n",
      "[6.37219662e-06 4.74167406e-05 7.08663910e-05 7.99693668e-05\n",
      " 8.33320613e-05 1.21946995e-04 1.84803568e-04 2.35983084e-04\n",
      " 4.34559069e-04 7.46879280e-04]\n",
      "[6.42741066e-07 2.08489128e-06 3.62493926e-06 3.94665402e-06\n",
      " 5.68138660e-06 3.52810647e-05 4.69836759e-05 2.25659713e-04\n",
      " 2.28236320e-04 2.33406380e-04]\n",
      "[1.58328376e-05 1.76142951e-05 6.05144202e-05 9.78262501e-05\n",
      " 1.32215957e-04 1.48474725e-04 1.49854704e-04 2.71913713e-04\n",
      " 4.82556771e-04 5.20955422e-04]\n",
      "[3.62779744e-06 3.74979579e-05 4.50799816e-05 1.02483351e-04\n",
      " 1.98320083e-04 3.98142039e-04 4.31822832e-04 6.44710633e-04\n",
      " 7.18398388e-04 8.30195558e-04]\n",
      "[8.12236167e-07 7.52739184e-06 3.21856290e-05 4.36166581e-05\n",
      " 6.08738955e-05 7.79514186e-05 8.03425231e-05 8.58578336e-05\n",
      " 9.72707766e-05 1.00940285e-04]\n",
      "[3.06074888e-08 1.05213751e-06 2.44577249e-05 5.44808606e-05\n",
      " 8.41429399e-05 1.23883173e-04 1.25427319e-04 1.64489525e-04\n",
      " 2.12361260e-04 2.65697528e-04]\n",
      "[2.19850900e-06 3.74084608e-05 1.19849016e-04 1.40138231e-04\n",
      " 1.73544627e-04 1.73555911e-04 2.00324701e-04 2.18444151e-04\n",
      " 2.53284162e-04 4.86276144e-04]\n",
      "[3.80340881e-06 5.37161753e-06 2.63391165e-05 5.51943354e-05\n",
      " 1.10135978e-04 1.62605116e-04 3.51298158e-04 4.64546731e-04\n",
      " 6.08470912e-04 6.78296428e-04]\n",
      "[2.30975848e-05 2.82387295e-05 3.01832481e-05 3.11692760e-05\n",
      " 5.02756227e-05 7.16274202e-05 8.57167342e-05 8.67878724e-05\n",
      " 1.04636268e-04 2.59133025e-04]\n",
      "[1.05134831e-06 3.60682942e-05 8.58054158e-05 9.86978032e-05\n",
      " 1.20315872e-04 1.40059332e-04 1.59673107e-04 2.22034453e-04\n",
      " 2.79566772e-04 3.23730651e-04]\n",
      "[1.09939133e-05 1.49870485e-05 4.18521003e-05 5.22214907e-05\n",
      " 1.49779367e-04 1.58264466e-04 2.16546990e-04 2.25139636e-04\n",
      " 2.47045500e-04 4.83399546e-04]\n",
      "[3.63190426e-07 2.72907124e-05 3.51586416e-05 3.81585784e-05\n",
      " 6.47853573e-05 2.84883972e-04 5.01873587e-04 5.72114731e-04\n",
      " 6.08391390e-04 6.59225353e-04]\n",
      "[1.89439450e-05 2.33307893e-05 4.64126496e-05 8.83380772e-05\n",
      " 1.05303717e-04 1.30819369e-04 1.31440733e-04 1.96445291e-04\n",
      " 2.68129493e-04 3.89088933e-04]\n",
      "[3.28831820e-07 4.17615732e-05 7.05051127e-05 7.60247526e-05\n",
      " 2.15566854e-04 2.87363207e-04 4.21935683e-04 4.47708862e-04\n",
      " 4.56869221e-04 6.98267460e-04]\n",
      "[1.05025352e-06 1.49628282e-06 1.38115903e-05 7.60742974e-05\n",
      " 1.13921161e-04 1.77981876e-04 2.39399532e-04 2.87760226e-04\n",
      " 3.60070000e-04 4.85574583e-04]\n",
      "[1.39174192e-06 1.40352086e-06 2.74388918e-05 2.76174538e-05\n",
      " 3.19253807e-05 3.51243235e-05 8.52085439e-05 9.29849258e-05\n",
      " 9.94092308e-05 1.14376415e-04]\n",
      "[8.13092890e-06 4.62418591e-05 6.13624976e-05 6.67830833e-05\n",
      " 1.55812906e-04 1.66861420e-04 1.83075641e-04 2.31117858e-04\n",
      " 2.58622679e-04 3.21243658e-04]\n",
      "[7.84087444e-07 5.99015120e-06 2.92835863e-05 3.11765926e-05\n",
      " 6.52581999e-05 7.43845083e-05 8.08200611e-05 9.18069966e-05\n",
      " 9.37078455e-05 1.68608702e-04]\n",
      "[1.23515460e-07 1.05022654e-05 3.11461238e-05 3.48683402e-05\n",
      " 3.49358034e-05 4.01229790e-05 8.12894397e-05 1.36148421e-04\n",
      " 2.43787698e-04 2.87465792e-04]\n",
      "[4.61226050e-09 2.23682645e-06 4.70276045e-05 5.60133856e-05\n",
      " 5.75793852e-05 7.11258400e-05 7.39817175e-05 8.24109600e-05\n",
      " 1.22078416e-04 1.53885036e-04]\n",
      "[7.68454022e-06 1.33869456e-05 3.96089341e-05 7.01241693e-05\n",
      " 1.18414129e-04 1.28376215e-04 1.42721466e-04 1.64921138e-04\n",
      " 1.87695902e-04 1.91698686e-04]\n",
      "[2.94200367e-07 1.59555089e-05 2.26578362e-05 2.71529362e-05\n",
      " 4.18009294e-05 5.19249253e-05 7.09482639e-05 7.93100077e-05\n",
      " 1.20387371e-04 1.81682406e-04]\n",
      "[7.80488347e-07 9.62389243e-07 2.17572622e-05 3.66940736e-05\n",
      " 4.15902863e-05 4.30568865e-05 5.65337590e-05 5.96884999e-05\n",
      " 7.16103703e-05 1.14335132e-04]\n",
      "[7.55589031e-07 1.54896848e-06 2.85346694e-06 7.89799929e-06\n",
      " 1.41636510e-05 2.62226158e-05 2.69871856e-05 4.93801327e-05\n",
      " 6.02870657e-05 1.52061938e-04]\n",
      "[6.96080905e-07 3.11741609e-05 4.05749717e-05 4.11012861e-05\n",
      " 5.71539754e-05 9.07447850e-05 1.02743201e-04 1.04878559e-04\n",
      " 1.23286521e-04 1.23408395e-04]\n",
      "[1.56575115e-06 3.39375116e-06 3.59552564e-06 7.92675137e-06\n",
      " 1.38277907e-05 2.40765866e-05 2.64294868e-05 2.73027733e-05\n",
      " 3.10036846e-05 5.11993471e-05]\n",
      "[5.54712406e-09 2.13797003e-07 1.67710334e-06 5.11059447e-06\n",
      " 1.00493446e-05 1.43191515e-05 2.77875915e-05 1.02883915e-04\n",
      " 1.20266812e-04 1.68254082e-04]\n",
      "[9.91335370e-07 3.61176094e-06 1.41893652e-05 2.69245883e-05\n",
      " 3.51038090e-05 6.45959395e-05 8.82935467e-05 1.23462559e-04\n",
      " 1.56904725e-04 1.69228332e-04]\n",
      "[2.53953535e-07 4.21791165e-07 3.65693184e-06 2.38007659e-05\n",
      " 2.82091748e-05 5.84941418e-05 6.72893790e-05 1.27575107e-04\n",
      " 1.39168666e-04 1.64316925e-04]\n",
      "[7.94739644e-07 1.82288755e-05 2.32190925e-05 4.78196377e-05\n",
      " 4.98529582e-05 1.01712261e-04 1.07899556e-04 1.13001979e-04\n",
      " 1.30027767e-04 1.41491647e-04]\n",
      "[5.61138396e-07 6.06490388e-07 4.58871920e-06 5.93774765e-06\n",
      " 9.83205529e-06 1.32924546e-05 2.72726357e-05 4.90490228e-05\n",
      " 8.48376639e-05 1.10603794e-04]\n",
      "[2.41481091e-06 4.07325104e-06 3.71845126e-05 5.89313790e-05\n",
      " 6.44871609e-05 8.94015657e-05 9.70810407e-05 1.09414356e-04\n",
      " 1.10277174e-04 1.25157725e-04]\n",
      "[4.86695887e-07 3.00711627e-06 9.21626849e-06 1.60164471e-05\n",
      " 2.16527644e-05 2.77401020e-05 3.21855288e-05 3.44699551e-05\n",
      " 4.09365937e-05 4.69853311e-05]\n",
      "[1.42635368e-06 1.46773997e-06 3.32596982e-06 3.99054181e-06\n",
      " 8.51253340e-06 1.79438368e-05 2.78285131e-05 3.24910768e-05\n",
      " 5.08137195e-05 6.16677006e-05]\n",
      "[3.29387110e-08 3.05374745e-07 4.38091095e-06 1.24727777e-05\n",
      " 1.38351000e-05 1.66638666e-05 1.77012574e-05 2.12243472e-05\n",
      " 5.20250652e-05 5.40500115e-05]\n",
      "[2.05737302e-09 4.39250980e-08 1.94926110e-07 1.40408796e-06\n",
      " 9.25120820e-06 2.32828885e-05 3.17743314e-05 3.81491054e-05\n",
      " 4.36295413e-05 4.51759382e-05]\n",
      "[4.69439518e-08 7.47340280e-08 1.04008640e-06 1.30040992e-06\n",
      " 1.85265447e-06 2.81581023e-06 4.53050251e-06 7.10538247e-06\n",
      " 1.12291698e-05 1.17364500e-05]\n",
      "[4.28351696e-08 2.47934461e-07 8.74218772e-07 2.67421181e-06\n",
      " 4.26441546e-06 5.38625345e-06 6.11274846e-06 9.65116171e-06\n",
      " 1.12616844e-05 1.57505566e-05]\n",
      "[1.15371066e-09 1.25905657e-06 1.59070585e-06 2.64109136e-06\n",
      " 4.85217865e-06 8.25464012e-06 9.50112051e-06 1.37014114e-05\n",
      " 1.92089538e-05 1.98861226e-05]\n",
      "[1.21967106e-07 1.60045604e-07 9.49943008e-07 1.11306589e-06\n",
      " 2.83892064e-06 3.35434077e-06 4.16069082e-06 4.22110623e-06\n",
      " 5.60196534e-06 6.54906267e-06]\n",
      "[9.39142376e-07 1.10221919e-06 1.81983111e-06 3.35931983e-06\n",
      " 6.02169966e-06 6.70940476e-06 9.51512387e-06 1.26108748e-05\n",
      " 1.26434273e-05 1.57040433e-05]\n",
      "[1.52209639e-09 1.77158044e-09 7.83792285e-09 2.71478261e-06\n",
      " 3.40916076e-06 3.97327402e-06 5.19974376e-06 1.00807132e-05\n",
      " 1.15095393e-05 1.45350289e-05]\n",
      "[2.44671726e-08 7.31930322e-08 9.69567731e-07 1.80715698e-06\n",
      " 2.44654743e-06 3.00709302e-06 1.13026805e-05 1.38338244e-05\n",
      " 1.41185349e-05 1.41577752e-05]\n",
      "[3.52744085e-07 6.63094811e-07 2.70819686e-06 3.29741508e-06\n",
      " 4.15956826e-06 5.06745879e-06 5.18508528e-06 6.59895679e-06\n",
      " 6.68401517e-06 9.69889524e-06]\n",
      "[7.09354885e-09 1.16537691e-08 1.21711538e-08 2.73540334e-07\n",
      " 4.27455456e-07 5.23903672e-07 9.15230941e-07 1.22463905e-06\n",
      " 1.29589570e-06 1.38070871e-06]\n",
      "[4.30891535e-08 2.76212226e-07 3.30117028e-07 4.51895512e-07\n",
      " 6.27035697e-07 8.91247046e-07 9.37944888e-07 1.29424873e-06\n",
      " 1.55621063e-06 1.76853698e-06]\n",
      "[2.02304499e-08 1.54100223e-07 1.92272272e-07 6.32779474e-07\n",
      " 1.03165613e-06 1.44403179e-06 1.94307115e-06 1.94878003e-06\n",
      " 1.96802866e-06 2.40250065e-06]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd8XGed7/HPT6Nm2eqSbVnFPS5xi+3YjkkCxIlTCHE2kMUsEO8SNgTCLmUvu8mF14YLu3fhsnspd2khCQQIKYSEBEhwOiZxXOTeLblKbpKtYsmy6jz3jzkygyXZsqSZMyN936+XrJnnPDPz09F4vjrPc4o55xAREQmX4HcBIiISexQOIiLShcJBRES6UDiIiEgXCgcREelC4SAiIl0oHEREpAuFg4iIdKFwEBGRLhL9LqCv8vLy3Lhx4/wuQ0QkbmzYsOGkcy6/N33jNhzGjRtHaWmp32WIiMQNMzvU274aVhIRkS4UDiIi0oXCQUREulA4iIhIFwoHERHpQuEgIiJdKBxERKSLIRUOre1BfvDmPlbtrfa7FBGRS/bqzhM8tGof0bi885AKh6SA8dCqffx+6zG/SxERuWTPbTrCz9ccwswi/lpDKhzMjBmFmWw7Uu93KSIil2zbkXpmFmZG5bUuGg5m9qiZVZnZ9rC2HDN7xczKvO/ZXruZ2XfNrNzMtprZ3LDHrPD6l5nZirD2eWa2zXvMdy3CkTirKJO9JxpobuuI5MuIiAyouqZWDtc0MbMwKyqv15sth58CN53Xdj/wmnNuMvCadx/gZmCy93UP8AMIhQnwILAQWAA82BkoXp97wh53/msNqJmFmbQHHbuPN0TyZUREBtT2I6cBYmfLwTm3Cqg5r3kZ8Jh3+zHg9rD2n7mQNUCWmRUANwKvOOdqnHO1wCvATd6yDOfcOy40w/KzsOeKiJlFodTdVlkXyZcRERlQW4+EPrNiJhx6MMo5dwzA+z7Say8EKsL6VXptF2qv7KY9YsZkppIzPFnzDiISV7YfqackJ43MtKSovN5AT0h3N1/g+tDe/ZOb3WNmpWZWWl3dt91RzYyZhZlsrVQ4iEj82FpZz8yi6Gw1QN/D4YQ3JIT3vcprrwSKw/oVAUcv0l7UTXu3nHMPOefmO+fm5+f36noV3ZpZmElZVaMmpUUkLtSeaaWy9mzUhpSg7+HwAtC5x9EK4Pmw9ru8vZYWAfXesNNKYKmZZXsT0UuBld6yBjNb5O2ldFfYc0XMzKJMOoKOncdOR/qlRET6rXMYfFYshYOZPQG8A0wxs0ozuxv4OnCDmZUBN3j3AV4E9gPlwI+BTwM452qArwHrva+vem0AnwIe9h6zD3hpYH60nnWm73bNO4hIHOgMh8ujGA4XvUyoc+7DPSxa0k1fB9zXw/M8CjzaTXspMONidQykgsxU8kYka95BROLCtsp6xuWmkTksOpPRMMSOkO7UeaS0thxEJB5sO1LPjChuNcAQDQcIjd3tPdHA2VZNSotI7DrV2MKRurPMiuKeSjCEw2FGYSZBBzuPaetBRGJX53yDthyiZNa5I6UVDiISu7YrHKJrVEYK+ekpmpQWkZi2tbKe8XnDyUiN3mQ0DOFwMDNmF2WyWedYEpEY5Zxjc0Uds6M83wBDOBwA5hRnsb/6DPVNbX6XIiLSxbH6ZqoaWphTHJ3TdIcb4uEQOmv4Fm09iEgM2lIR+myaU5J9kZ4Db0iHw6ziTMz+/AsQEYklmyvqSA4kMK0gPeqvPaTDISM1iYn5I9iscBCRGLSpoo5pYzJISQxE/bWHdDhAaN5hc0UdoTN/iIjEhvaOINsq67nCh/kGUDgwuziLU97pcEVEYkVZVSNn2zp8mYwGhcO5VN6koSURiSGdw90KB59MGZ1OSmKCJqVFJKZsPlxHVloSY3PTfHn9IR8OSYEEZhZmalJaRGJK6OC3LELXQYu+IR8OENps236knraOoN+liIjQ2NLO3qoG34aUQOEAhCalW9qD7D7W4HcpIiJsq6zHOZhTonDwVWc6b66o9bkSEZE/T0bPLlI4+Kooexh5I5K1x5KIxITNFbWMzU0jZ3iybzUoHAidofWKkmw2HVY4iIi/nHNsPFzn28FvnRQOnnljszlw8gynGlv8LkVEhrDK2rNUN7Qwb1yOr3UoHDzzxobOerhRWw8i4qPSQzUAzPPhTKzhFA6emYWZJAXs3C9GRMQPGw7VMiIlkSmjo38m1nAKB09qUoAZhZlsPKQ9lkTEPxsO1XFFSRaBBH8OfuukcAgzrySbLZX1tLbrYDgRib6G5jb2HD/NXJ+HlEDh8Bfmjc2mtT3IjqP1fpciIkPQ5oo6gg7mj1M4xJTOSekNGloSER9sOFSLmX9nYg2ncAgzMiOV4pxhCgcR8cWGQ7VMGZVOemqS36UoHM43rySb0kO1ujKciERVR9Cx6XBdTAwpgcKhi3ljs6luaNGV4UQkqvaeaKCxpf3c8Lbf+hUOZvZ5M9thZtvN7AkzSzWz8Wa21szKzOwpM0v2+qZ498u95ePCnucBr32Pmd3Yvx+pf+aNDR2VqKElEYmmUu8zZ16Jv0dGd+pzOJhZIfCPwHzn3AwgACwHvgF8yzk3GagF7vYecjdQ65ybBHzL64eZTfcedzlwE/B9Mwv0ta7+mjI6neHJAYWDiETVxkO15KenUJwzzO9SgP4PKyUCw8wsEUgDjgHXAc94yx8DbvduL/Pu4y1fYqFLHC0DnnTOtTjnDgDlwIJ+1tVngYTQSfjWH9SR0iISPesP1jCvJNu3K7+dr8/h4Jw7AvwncJhQKNQDG4A651y7160SKPRuFwIV3mPbvf654e3dPOYvmNk9ZlZqZqXV1dV9Lf2iFozPYc+JBuqaWiP2GiIinY7UnaWy9iwLxsfGkBL0b1gpm9Bf/eOBMcBw4OZuunbu9tNdHLoLtHdtdO4h59x859z8/Pz8Sy+6lxaMz8E5KD2ooSURibz1B0IjFYMiHIDrgQPOuWrnXBvwLLAYyPKGmQCKgKPe7UqgGMBbngnUhLd38xhfzCnOIjmQwDoNLYlIFKw9UEN6aiLTCjL8LuWc/oTDYWCRmaV5cwdLgJ3AG8AHvT4rgOe92y949/GWv+5CBxO8ACz39mYaD0wG1vWjrn5LTQowpziLtQcUDiISeesOnOLKcTm+n2wvXH/mHNYSmljeCGzznush4F+AL5hZOaE5hUe8hzwC5HrtXwDu955nB/A0oWD5A3Cfc66jr3UNlAXjc9h+pJ4zLe0X7ywi0kcnG1vYV30mpoaUILS3UZ855x4EHjyveT/d7G3knGsG7uzhef4d+Pf+1DLQFozP4b/fKGfj4VqumRy5+Q0RGdpicb4BdIR0j+aOzSaQYKzT0JKIRNDaAzUMSwowY0ym36X8BYVDD0akJDJjTAZr9yscRCRy1h6oYe7YLJITY+vjOLaqiTELJ+SyuaKO5jbfp0BEZBCqb2pj9/HTLByf63cpXSgcLmDBuBxaO4JsqajzuxQRGYRKD9XgXOzNN4DC4YKuHJeDGZp3EJGIWHeghuRAQkxc3Od8CocLyExLYsqodB3vICIRsfZADbOLM0lN8u1coz1SOFzEogm5lB6qobU96HcpIjKINDS3se1IfUzON4DC4aIWT8yluS3IZs07iMgAWn+who6gY/EkhUNcWjghlwSDt8tP+l2KiAwib5efIjkxgbklsXHlt/MpHC4ic1gSMwozeWffKb9LEZFBZPW+U8wfmx2T8w2gcOiVxRPz2FRRS1OrzrMkIv1Xc6aVXcdO865JeX6X0iOFQy8snphLW4djva7vICIDoHMk4qqJsTnfAAqHXpk/LpukgLF6n+YdRKT/Vu87yYiURGYVxtb5lMIpHHohLTmRK0qyWV2ueQcR6b/V+06xcHwOiYHY/QiO3cpizOKJuWw/Wk99U5vfpYhIHDtad5YDJ8/E9JASKBx6bfHEPJyDNQe09SAifdc537B4YuxORoPCodfmFGcxLCnAah3vICL98Pa+k+QMT2bq6HS/S7kghUMvJScmcOX4HN7W8Q4i0kfOOd7Zd4qrJuSSEEPXi+6OwuESXD0pl/KqRo7Vn/W7FBGJQ/uqGzlW3xzTxzd0UjhcgmsvC11L+k97NbQkIpfuj95nx7WXKRwGlSmj0hmVkcIfy6r9LkVE4tCqvdVMyB9OUXaa36VclMLhEpgZ10zO562yk3QEnd/liEgcaW7rYO2BU1w7Od/vUnpF4XCJrr0sn/qzbWyt1Cm8RaT31h+sobktyLsvUzgMSldPysMMVmneQUQuwaq91SQHElg4IfauF90dhcMlyhmezMzCTFZp3kFELsGqvSeZPy6btOREv0vpFYVDH1w7OZ/NFXXUn9WpNETk4o7XN7PnRMO5PR7jgcKhD669LJ+OoNPR0iLSK50jDfEyGQ0Khz65oiSLESmJrCpTOIjIxf2p7CT56SlMK4jtU2aEUzj0QVIggcUTc1m1txrntEuriPSsI+h4q6yaaybnYRbbp8wI169wMLMsM3vGzHab2S4zu8rMcszsFTMr875ne33NzL5rZuVmttXM5oY9zwqvf5mZrejvDxUN75kykiN1ZymvavS7FBGJYVsq66htaoubXVg79XfL4TvAH5xzU4HZwC7gfuA159xk4DXvPsDNwGTv6x7gBwBmlgM8CCwEFgAPdgZKLLtu6kgAXttd5XMlIhLLXt9VRSDBeM9lI/0u5ZL0ORzMLAO4FngEwDnX6pyrA5YBj3ndHgNu924vA37mQtYAWWZWANwIvOKcq3HO1QKvADf1ta5oGZ2ZyuVjMnh9l8JBRHr22u4q5o/NJjMtye9SLkl/thwmANXAT8xsk5k9bGbDgVHOuWMA3vfOuCwEKsIeX+m19dQe866bOpLSQzXUNbX6XYqIxKCjdWfZdez0uZGGeNKfcEgE5gI/cM5dAZzhz0NI3eluJsZdoL3rE5jdY2alZlZaXe3/QWjXTR1J0MEf9/pfi4jEnte9Yecl04ZWOFQClc65td79ZwiFxQlvuAjve1VY/+KwxxcBRy/Q3oVz7iHn3Hzn3Pz8fP8nd2YXZZE7PJnXNLQkIt14fXcVJTlpTMwf4Xcpl6zP4eCcOw5UmNkUr2kJsBN4Aejc42gF8Lx3+wXgLm+vpUVAvTfstBJYambZ3kT0Uq8t5iUkGO+dOpI391TR3hH0uxwRiSFnWzt4u/wk100dGVe7sHbq70k+/gF43MySgf3A3xEKnKfN7G7gMHCn1/dF4BagHGjy+uKcqzGzrwHrvX5fdc7V9LOuqFkydSTPbKhkw6FaFk7I9bscEYkRq/edpKU9GJdDStDPcHDObQbmd7NoSTd9HXBfD8/zKPBof2rxy9WT80gKGK/vrlI4iMg5r+2uYnhygAXj4+MsrOfTEdL9lJ6axMLxuecmnkREnHO8sbuKaybnk5IY8LucPlE4DIDrpo6krKqRgyfP+F2KiMSAHUdPc6y+mevidEgJFA4D4obpowBYueO4z5WISCxYueM4CRaak4xXCocBUJyTxozCDIWDiAChcFgwPofcESl+l9JnCocBcuP00Ww8XMeJ081+lyIiPtpf3cjeE43cePlov0vpF4XDALlxRuiN8PLOEz5XIiJ+Wrkj9BmwVOEgAJNHjmBC3nBe1tCSyJC2csdxZhVlUpg1zO9S+kXhMEDMjKWXj+adfaeob9K1pUWGouP1zWyuqIv7ISVQOAyoGy8fRXvQ8dpuDS2JDEUv7wyNHNx4+SifK+k/hcMAml2UxeiMVO21JDJErdxxnIn5w5k0Mn6uFd0ThcMASkgwll4+ij/uraaptd3vckQkiuqaWlmzv2ZQDCmBwmHA3TRjNM1tQd7YrWs8iAwlK3ccpyPouHlGgd+lDAiFwwBbOD6X/PQUfrul20tSiMgg9dstxxibGzogdjBQOAywQIJxy4zRvLGnisYWDS2JDAUnG1tYve8kt84qiMtrN3RH4RAB7589hpb2IK/qgDiRIeGl7ccJutD//cFC4RABc0uyKchM1dCSyBDx2y1HmTRyBFNGxf9eSp0UDhGQkGC8b2YBq8qqdUCcyCB34nQz6w/WDKohJVA4RMz7Z4+hrcOxcqeOeRAZzH6/9RjOwa2zBs+QEigcImZWUSYlOWkaWhIZ5H679SjTCjKYNHKE36UMKIVDhJgZ75tVwOp9pzjV2OJ3OSISAZW1TWw6XMetswbHsQ3hFA4RdNvsMXQEHb/beszvUkQkAp7fHBoZeP8gG1IChUNETSvIYFpBBs9urPS7FBEZYM45fr2xkgXjcijJTfO7nAGncIiwD8wtZEtlPeVVDX6XIiIDaEtlPfurz3DH3EK/S4kIhUOE3TZnDAkGz2484ncpIjKAnt1YSXJiArcMwvkGUDhE3Mj0VK69LJ/nNh0hGHR+lyMiA6C1PcgLW46ydPooMlKT/C4nIhQOUXDH3CKO1TezZv8pv0sRkQHwxp4q6pra+MDcIr9LiRiFQxQsnT6K9JREfq2hJZFB4dmNleSNSOGayXl+lxIxCocoSE0K8L5ZBby0/ZguAiQS52rPtPL67ipunzOGxMDg/QgdvD9ZjLljbhFNrR28uE2n0xCJZ89vPkJbh+OvBuleSp36HQ5mFjCzTWb2O+/+eDNba2ZlZvaUmSV77Sne/XJv+biw53jAa99jZjf2t6ZYdOW4bCbkDefJdYf9LkVE+sg5x5PrK5hZmMnlYzL9LieiBmLL4bPArrD73wC+5ZybDNQCd3vtdwO1zrlJwLe8fpjZdGA5cDlwE/B9MwsMQF0xxcxYvqCY0kO1lJ3QMQ8i8WhzRR27jzfw4QUlfpcScf0KBzMrAt4HPOzdN+A64Bmvy2PA7d7tZd59vOVLvP7LgCedcy3OuQNAObCgP3XFqg/MLSIpYDyxrsLvUkSkD55Yd5i05AC3zRl8p8s4X3+3HL4N/DMQ9O7nAnXOuc5Z10qgc2CuEKgA8JbXe/3PtXfzmEEld0QKS6eP5tlNlTS3dfhdjohcgobmNn675RjvnzWGESmJfpcTcX0OBzO7Fahyzm0Ib+6mq7vIsgs95vzXvMfMSs2stLq6+pLqjRXLFxRT19TGyh2amBaJJy9sOcrZtg6WLyj2u5So6M+Ww7uA28zsIPAkoeGkbwNZZtYZq0VA5wUNKoFiAG95JlAT3t7NY/6Cc+4h59x859z8/Pz8fpTun3dNzKM4ZxhPamhJJK48ua6CqaPTmVOc5XcpUdHncHDOPeCcK3LOjSM0ofy6c+4jwBvAB71uK4DnvdsvePfxlr/unHNe+3Jvb6bxwGRgXV/rinUJCcbyK0t4Z/8pDpw843c5ItIL24/Us+1IPR9eUDKoLgV6IZE4zuFfgC+YWTmhOYVHvPZHgFyv/QvA/QDOuR3A08BO4A/Afc65QT0gf+e8IhITjMfXHPK7FBHphV+sOURqUgK3zxmU06HdGpBZFefcm8Cb3u39dLO3kXOuGbizh8f/O/DvA1FLPBiZkcrNMwt4qrSCz99wGcOHwOSWSLyqPdPKc5uOcMfcIjLTBudJ9rqjI6R98reLx9LQ3M5zm3S+JZFY9lRpBS3tQVYsHut3KVGlcPDJ3JJsZhZm8tjqg4SmXkQk1rR3BPn5O4e4akIuU0dn+F1OVCkcfGJmrFg8jrKqRlbv06m8RWLRq7uqOFJ3lhWLx/ldStQpHHx066wCcoYn89PVB/0uRUS68djqgxRmDeP6aSP9LiXqFA4+Sk0K8DcLSnh11wkqapr8LkdEwuw+fpp39p/iY1eNHdSn5u7J0PuJY8xHFpUQMOMnbx/0uxQRCfPoWwdISUzgQ/OHxhHR51M4+Kwgcxi3zR7Dk+sPU9fU6nc5IgKcON3Mc5uO8KEri8kenux3Ob5QOMSAe949gabWDn6hg+JEYsKjbx+gI+j4xNUT/C7FNwqHGDB1dAbvmZLPT94+qLO1ivjsdHMbv1xzmFtmFlCSm+Z3Ob5ROMSIe989kVNnWnlmQ6XfpYgMaU+sPUxDSzv3vnui36X4SuEQIxaOz2F2cRY//tN+OoI6KE7EDy3tHTzy1gGunpTHjMLBfRnQi1E4xAgz41PvnsChU028tP2Y3+WIDEm/2XSEqoYWPvnuoTvX0EnhEENumD6aCfnD+e/Xywlq60Ekqto7gnz/zX3MKMzg6kl5fpfjO4VDDAkkGJ9dMpndxxt0pTiRKHtu0xEOnWric0suGzLXbLgQhUOMuXXWGCbkD+c7r5Vp60EkSto6gvy/18uZUZjBkiF4qozuKBxiTPjWwx+09SASFc9tOsLhGm01hFM4xKBzWw+vautBJNJCWw1lzCzM1FZDGIVDDOrcethzooGXtmvrQSSSntt4hIqas3zu+snaagijcIhRt84aw6SRI/ivV/bQ3hH0uxyRQam5rYNvv7qXWUWZXDdVWw3hFA4xKpBg/PONU9hffYanSiv8LkdkUHps9UGO1jdz/81TtdVwHoVDDLth+ijmj83mW6+Ucaal3e9yRAaVuqZWvvdGOe+Zks/iiTqu4XwKhxhmZjxwyzRONrbw8J8O+F2OyKDy/Tf30dDSzv03T/W7lJikcIhx88Zmc/OM0Ty0ah/VDS1+lyMyKFTWNvHTtw/ywblFTB2d4Xc5MUnhEAe+eOMUmtuDfPvVvX6XIjIo/OfKPZjBF5Ze5ncpMUvhEAcm5I/gY4vG8sS6w+w4Wu93OSJxbf3BGn6z+Sh/f80ECjKH+V1OzFI4xInP33AZ2WnJPPj8DpzTgXEifdERdPzr8zsYk5nKp987tK/XcDEKhziROSyJf7lpKqWHavnN5iN+lyMSl3659hC7jp3my7dOJy050e9yYprCIY58cF4Rs4uz+N8v7qahuc3vckTiyqnGFr65cg+LJ+Zy84zRfpcT8xQOcSQhwfjqbZdzsrGF77xa5nc5InHlmyv30NTawf+67XId8NYLCoc4M7s4i+VXlvDo2wfYWlnndzkicWHN/lM8ub6Cv3vXOCaPSve7nLjQ53Aws2Ize8PMdpnZDjP7rNeeY2avmFmZ9z3bazcz+66ZlZvZVjObG/ZcK7z+ZWa2ov8/1uB2/81TyRuRwj8/s5U2nXdJ5IKa2zq4/9dbKclJ4ws3TPG7nLjRny2HduCfnHPTgEXAfWY2HbgfeM05Nxl4zbsPcDMw2fu6B/gBhMIEeBBYCCwAHuwMFOle5rAk/u32Gew+3sCP/rjP73JEYtq3Xt3LwVNNfP2OmQxLDvhdTtzoczg454455zZ6txuAXUAhsAx4zOv2GHC7d3sZ8DMXsgbIMrMC4EbgFedcjXOuFngFuKmvdQ0VSy8fzftmFfDd18opr2rwuxyRmLStsp4fr9rP8iuLWazrQl+SAZlzMLNxwBXAWmCUc+4YhAIE6DwPbiEQfnrRSq+tp3a5iK+8/3LSUgJ88ZmtOq23yHla2jv44jNbyBuRwgO3TPO7nLjT73AwsxHAr4HPOedOX6hrN23uAu3dvdY9ZlZqZqXV1dWXXuwgk5+ewleXzWDT4Tq+94aGl0TC/efKPew+3sB/3DGTzGFJfpcTd/oVDmaWRCgYHnfOPes1n/CGi/C+V3ntlUBx2MOLgKMXaO/COfeQc26+c25+fn5+f0ofNG6bPYbb54zhu6+XsfFwrd/liMSEt8pO8uM/HeCji0pYMm2U3+XEpf7srWTAI8Au59z/DVv0AtC5x9EK4Pmw9ru8vZYWAfXesNNKYKmZZXsT0Uu9Numlr94+g9EZqXzuyc006roPMsTVnmnln361mYn5w/nSLdP9Lidu9WfL4V3Ax4DrzGyz93UL8HXgBjMrA27w7gO8COwHyoEfA58GcM7VAF8D1ntfX/XapJcyUpP49vI5VNY28ZUXdvhdjohvnHM88Ow2as608p3lV2jvpH7o88lFnHNv0f18AcCSbvo74L4enutR4NG+1iJw5bgcPvPeSXz39XIWjMvhr68svviDRAaZR98+yB92HOeBm6cyozDT73Limo6QHkQ+e/1lXD0pjy8/v53tR3Rqbxla1h+s4T9e3MUN00dxz7UT/C4n7ikcBpFAgvGd5XPIG57Mvb/YQF1Tq98liURFVUMz9z2+kaLsYfzXX8/WuZMGgMJhkMkdkcL3PjKXE6eb+dxTm+kI6toPMri1tgf5zC83cbq5jR98dB4ZqdptdSAoHAahK0qy+cptl/Pmnmr+7fc7/S5HJGKcc3zpuW2sO1DDNz4wi2kFuh70QNHVLgapjywcy76qMzz69gHG5w3nrqvG+V2SyID7/pv7+NWGSv5xyWSWzdGJFQaSwmEQ+9L7pnG4JrR7a3F2Gu+dOvLiDxKJE7/bepRvrtzDsjlj+Pz1k/0uZ9DRsNIg1jlBPa0gg/t+uZFNOoJaBol39p3iC09vYf7YbL7xgVmagI4AhcMgNzwlkUf/9kryRqTwtz9Zz+7jFzr9lUjs21xRxyceW8/YnDQeums+qUk60C0SFA5DwKiMVB7/xEKGJQX46MPrOHDyjN8lifTJ7uOnWfHoOnJHpPCLTywkZ3iy3yUNWgqHIaI4J41ffGIBQef46MNrOXyqye+SRC5JeVUDH3tkHalJCTz+iYWMykj1u6RBTeEwhEwamc7PPr6AM63t3Pmj1ZRXNfpdkkiv7Dhaz1//aA3OwS/uXkhxTprfJQ16CochZkZhJk/es4iOIHzoR++w86jmICS2bTxcy4cfWkNqYgK/uvcqJo9K97ukIUHhMARNHZ3B059cRHJiAssfeoe1+0/5XZJIt/64t5qPPbyW7OHJPH3vVYzPG+53SUOGwmGImpA/gqc/eRV56Sl87JF1/GbTEb9LEvkLv1x7mI//dD3FOWk8/cmrKMrWUFI0KRyGsOKcNJ791GKuKMnic09t5juvlhE6s7qIf4JBx3+8uIv/+dw2rpmcxzOfWqzJZx8oHIa4rLRkfn73Qu6YW8i3Xt3Lpx/fSENzm99lyRBVe6aVjz+2nh+t2s9HF5Xw8F3zGZGiEzn4QeEgJCcm8F93zuZLt0zj5Z0nuO2/39bBchJ1WyrquPX/vcXq8lN87fYZfG3ZDBID+ojyi9a8AGBm/P21E3ji7xdxpqWd27/3Nr9Yc0jDTBJxwaDj4T/t584fvgPAr+49NJXsAAAMlUlEQVS9io8tGqtTYvhM4SB/YcH4HH73j1dz5bgcvvyb7fztT9Zz4nSz32XJIFVZ28TfPLyGf/v9Lq69LI/f/cPVzC7O8rssASxe/zKcP3++Ky0t9buMQSsYdPxi7SH+94u7SEkM8OX3TeOD84r015wMiI6g44l1h/n6S7sB+Nf3T+dOvb8izsw2OOfm96qvwkEuZH91I198ZisbDtVy5bhs/u32mUwZrYOQpO+2H6nnS7/ZzpaKOhZPzOUbH5ilI56jROEgAyoYdPxqQwX/8dJuGpvb+eiisfzDdZPIHZHid2kSR6pON/OtV8t4av1hcoan8OX3TWPZnDHaWogihYNERM2ZVr65cg9Pl1YwLCnAJ6+dwN3XjCctWbsaSs8amtt4aNV+Hv7TAdqDQT6ycCyfv+EyMofpWs/RpnCQiCqvauT//GE3L+88QXZaEh9/13juumocmWn6zy5/VnOmlZ+8fYCfrj5IQ3M7t84q4Is3TmFsrk6B4ReFg0TFxsO1fP+Ncl7dVcWIlEQ+dGUxH1s0lnE6/82QVl7VwM/fOcTTpZU0t3dw0+Wjue+9k5hRmOl3aUOewkGiatex0/zwj/v4/dZjtAcd116Wz98sKOG9U/NJSdRVuoaC5rYOXt11gsfXHOad/adIDiRw25wx3PvuCUwaqR0YYoXCQXxRdbqZJ9ZV8Mt1hzhxuoXMYUncOquAZXMKmTc2m0CCJh4Hk/aOIOsO1PD85qO8uO0YDS3tFGYN4yOLSvjQ/GLtsBCDFA7iq/aOIH8qP8lzG4/w8s7jNLcFyR2ezPXTRnH99FEsmpBDeqrmJ+JR/dk23tl3kld2VvHa7hPUNbWRlhzgphmjueOKIq6amKs/AmLYpYSDdjORAZcYSOC9U0by3ikjaWxp543dVby88wS/33aMp0orSEwwrijJYvHEPK4cl8OckiydXC1GnW5uY9PhOkoP1vBW+Um2VNQRdJCRmsiSaaNYOn0U756Srz3WBqGY2XIws5uA7wAB4GHn3Ncv1F9bDvGnpb2DDYdqeavsJG+Vn2TbkXqcgwSDKaMzmFmYwYzCTKYXZDB5VLp2dYyyuqZW9p5oZMfRenYcPc32I/XsOdFw7nc0qyiLaybncfWkPOaOzSZJJ8WLO3E3rGRmAWAvcANQCawHPuyc29nTYxQO8a+h86/SQ7VsOlzLjqOnqTnTem75yPQUJo0cwdjc4ZTkpDE2N42CzFQKMoeRn56i4YtL1N4RpLqxhWP1zRytO8vhmiYOn2ri4KkzlFed4WRjy7m+eSNSmFGYwRXF2cwfl82c4iyGa+su7sXjsNICoNw5tx/AzJ4ElgE9hoPEv/TUJK69LJ9rL8sHwDnH8dPN7Dx6mvKqRsqqGtlX3cjLO45zKiw0AAIJRu7wZPJGpJCXnkJ2WhLZaclkDksiPTWRjNTQ97SURIYnBxiWHGBYUoCUpACpiQkkJyaQFEggOZBAQoyHTDDoaO0I0tYRpKU99NXc1sHZ1g7OtnXQ1NrBmZZ2GprbaGhu53RzO/VNrdQ2tVHb1MrJxlaqG1qoOdNC8Ly/BfNGJFOSk8Z1U/OZPDKdSSNHMH1Mhi6uIzETDoVARdj9SmChT7WIT8yMgsxhFGQOY8m0UX+xrKG5jYqasxyrP8ux+maO1zdT3dDCycbQ18GTZ6hrauV0c/slv24gwUhMMJICCQQSjECCkWBGIAESLHQ7IQEMI8FCdRqAlynm1R7WROdnsHPu3G1cqN05R9CBwxEMQtA5gs7R4d1u7wjSEXS0e18d53+i90LmsCSyvMAszBrG7KJM8tNTGJ2ZypjMYYzOTKUkJ01bA9KjWHlndPenW5f/EWZ2D3APQElJSaRrkhiSnprE9DFJTB+TccF+7R1BzrR0cNr7K/psWztnWkJ/Xbe0d9Dc1kFzW5DW9iCtHaHv7cEg7R3eh3FHkI7OD+qg8z64Qx/anR/0Qce561y4c/+EPuzDWVh6dL7BzbyAOXc7dD8UQKGQ6gyoxMCfQyspkECKt8WTmhggJSmB1KQAacmdX4mkpyaSnpLEiNREDblJv8VKOFQCxWH3i4Cj53dyzj0EPAShOYfolCbxJDGQQGZagk7lIdJPsbK7wXpgspmNN7NkYDnwgs81iYgMWTGx5eCcazezzwArCe3K+qhzbofPZYmIDFkxEQ4AzrkXgRf9rkNERGJnWElERGKIwkFERLpQOIiISBcKBxER6ULhICIiXcTEiff6wsyqgUN9fHgecHIAyxkoquvSqK5Lo7ouzWCsa6xzLr83HeM2HPrDzEp7e2bCaFJdl0Z1XRrVdWmGel0aVhIRkS4UDiIi0sVQDYeH/C6gB6rr0qiuS6O6Ls2QrmtIzjmIiMiFDdUtBxERuYBBGw5mdqeZ7TCzoJnNP2/ZA2ZWbmZ7zOzGHh4/3szWmlmZmT3lnUp8oGt8ysw2e18HzWxzD/0Omtk2r1/EL5xtZl8xsyNhtd3SQ7+bvHVYbmb3R6Gub5rZbjPbambPmVlWD/2isr4u9vObWYr3Oy733kvjIlVL2GsWm9kbZrbLe/9/tps+7zGz+rDf779Gui7vdS/4e7GQ73rra6uZzY1CTVPC1sNmMzttZp87r09U1peZPWpmVWa2Pawtx8xe8T6HXjGz7B4eu8LrU2ZmKwakINd5hatB9gVMA6YAbwLzw9qnA1uAFGA8sA8IdPP4p4Hl3u0fAp+KcL3/BfxrD8sOAnlRXHdfAf7HRfoEvHU3AUj21un0CNe1FEj0bn8D+IZf66s3Pz/waeCH3u3lwFNR+N0VAHO92+nA3m7qeg/wu2i9n3r7ewFuAV4idKG8RcDaKNcXAI4TOhYg6usLuBaYC2wPa/s/wP3e7fu7e88DOcB+73u2dzu7v/UM2i0H59wu59yebhYtA550zrU45w4A5cCC8A4WuiDwdcAzXtNjwO2RqtV7vb8GnojUa0TAAqDcObffOdcKPElo3UaMc+5l51znRaLXELpioF968/MvI/TegdB7aYl1Xmw6Qpxzx5xzG73bDcAuQtdojwfLgJ+5kDVAlpkVRPH1lwD7nHN9Pbi2X5xzq4Ca85rD30M9fQ7dCLzinKtxztUCrwA39beeQRsOF1AIVITdr6Trf55coC7sg6i7PgPpGuCEc66sh+UOeNnMNnjX0Y6Gz3ib9o/2sCnbm/UYSR8n9Fdmd6Kxvnrz85/r472X6gm9t6LCG8a6AljbzeKrzGyLmb1kZpdHqaSL/V78fk8tp+c/0PxYXwCjnHPHIBT8wMhu+kRkvcXMxX76wsxeBUZ3s+hLzrnne3pYN23n77LVmz690ssaP8yFtxre5Zw7amYjgVfMbLf3V0afXagu4AfA1wj9zF8jNOT18fOfopvH9nvXt96sLzP7EtAOPN7D0wz4+uqu1G7aIvY+ulRmNgL4NfA559zp8xZvJDR00ujNJ/0GmByFsi72e/FzfSUDtwEPdLPYr/XVWxFZb3EdDs656/vwsEqgOOx+EXD0vD4nCW3SJnp/8XXXZ0BqNLNE4A5g3gWe46j3vcrMniM0pNGvD7verjsz+zHwu24W9WY9Dnhd3mTbrcAS5w24dvMcA76+utGbn7+zT6X3e86k67DBgDOzJELB8Lhz7tnzl4eHhXPuRTP7vpnlOecieh6hXvxeIvKe6qWbgY3OuRPnL/BrfXlOmFmBc+6YN8RW1U2fSkLzIp2KCM219stQHFZ6AVju7UkyntBfAOvCO3gfOm8AH/SaVgA9bYn01/XAbudcZXcLzWy4maV33iY0Kbu9u74D5bxx3r/q4fXWA5MttFdXMqFN8hciXNdNwL8AtznnmnroE6311Zuf/wVC7x0IvZde7ynQBoo3p/EIsMs593976DO6c+7DzBYQ+hw4FeG6evN7eQG4y9traRFQ3zmkEgU9br37sb7ChL+HevocWgksNbNsbwh4qdfWP5Gegffri9CHWiXQApwAVoYt+xKhPU32ADeHtb8IjPFuTyAUGuXAr4CUCNX5U+De89rGAC+G1bHF+9pBaHgl0uvu58A2YKv35iw4vy7v/i2E9obZF6W6ygmNrW72vn54fl3RXF/d/fzAVwmFF0Cq994p995LE6Kwjq4mNKSwNWw93QLc2/k+Az7jrZsthCb2F0ehrm5/L+fVZcD3vPW5jbC9DCNcWxqhD/vMsLaory9C4XQMaPM+u+4mNEf1GlDmfc/x+s4HHg577Me991k58HcDUY+OkBYRkS6G4rCSiIhchMJBRES6UDiIiEgXCgcREelC4SAiIl0oHEREpAuFg4iIdKFwEBGRLv4/Uw5YHte+pP0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# decreasing noise on std\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "N=100\n",
    "ratio=0.1\n",
    "num_itr=50\n",
    "std_noise = 0.1\n",
    "\n",
    "f=lambda x: 3*x**2+x**4  # function of f(x)\n",
    "samples = np.random.normal(0,1,N) # x-value of f(x)\n",
    "print('true min: ', np.min(f(samples)))\n",
    "for i in range(num_itr):\n",
    "    # sample and select top 10% (max [-a:]or min [:a])\n",
    "    selected_samples = samples[np.array(f(samples)).argsort()[:int(N*ratio)]]\n",
    "    m=np.mean(selected_samples)\n",
    "    std=np.std(selected_samples)+std_noise*(num_itr-i)/num_itr  # decreasing noise on std\n",
    "#     print(std)\n",
    "    # fit the gaussian\n",
    "    samples=np.random.normal(m, std, N)\n",
    "    print(f(selected_samples))\n",
    "\n",
    "'''display the function f(x) '''\n",
    "x=np.linspace(-10,10,N)\n",
    "y=f(x)\n",
    "plt.plot(x,y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Results: the noise on std actually delays the convergence process of CE method and makes it more explorative."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}