time_series/TimeSeriesPrediction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Machine learning strategies for time series forecasting\n",
    "__Damian Leśniak, Igor Podolak__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<div id=\"toc\"></div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%javascript\n",
    "$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    }
   ],
   "source": [
    "%pylab inline\n",
    "import os\n",
    "\n",
    "from matplotlib import pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.neighbors import KNeighborsClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score, brier_score_loss, classification_report\n",
    "from sklearn.metrics import confusion_matrix, log_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Time series: a sequence S of historical measurements $y_t$ of an observable variable $y$\n",
    "\n",
    "Let there be a _dynamical system_ whose state evolves in a state space $\\Gamma\\subset\\mathbb{R}^G$ according to $$s(t)=\\mathcal{F}^t(s(0))$$\n",
    "\n",
    "* In the absence of _noise_ the time series is related to dynamical system by relation $$y_t=\\mathcal{G}(s(t))$$ where $$\\mathcal{G}:\\Gamma\\rightarrow{}\\mathbb{R}^D$$ is a measurement function (let $D=1$)\n",
    "  * if $D=1$ then we have __univariate time series__\n",
    "\n",
    "* $f:\\mathbb{R}^n\\rightarrow\\mathbb{R}$ is induced in the space of vectors\n",
    "$$y_t=f(y_{t-l}, y_{t-l-1},\\dots, y_{t-l-d+1}),\\tag{*}$$\n",
    "where\n",
    "* $(y_{t-l}, y_{t-l-1},\\dots, y_{t-l-n+1})$ is the __embeding vector__,\n",
    "* $l$ is the __lag time__,\n",
    "* $d$ is the __order__, or __embedding dimension__, ie. number of past values used.\n",
    "\n",
    "  * the reconstructed states can be used to __estimate__ $f$, and the $f$ can be used as alternative to $\\mathcal{F}$ and $\\mathcal{G}$\n",
    "  * $(*)$ does not take into account __noise__, thus\n",
    "  $$y_t=f(y_{t-l}, y_{t-l-1},\\dots, y_{t-l-d+1})+w(t)$$\n",
    "  with missing information in $w(t)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Auto Regressive Integrated Moving Average ARIMA model\n",
    "* a time series is __stationary__ if its statistical properties are constant over time\n",
    "  * is one whose properties __do not__ depend on the time at which series is observed\n",
    "  * it has no trend\n",
    "  * variations around its mean have a constant amplitude\n",
    "  * it wiggles in consistent fashion\n",
    "    * short term random time patterns look the same in some statistical sense\n",
    "    \n",
    "    \n",
    "    \n",
    "<img src=\"stationary.png\" width=\"100%\" />\n",
    "      * (a) Dow Jones on 292 conseccutive days\n",
    "        * __no__: trend rules\n",
    "      * (b) daily change in Dow Jones on 292 consecutive days\n",
    "        * __yes__: even though (a) was not stationary, it can be changed into stationary by differencing\n",
    "      * (c) annual number of strikes in US\n",
    "        * __no__: trend rules\n",
    "      * (d) monthly sales of new one family hauses in the US\n",
    "        * __no__: seasonal \n",
    "      * (e) price of a dozen eggs in US (inflation removed)\n",
    "        * __no__: trend rules\n",
    "      * (f)monthly total of pigs slaughtered in Victoria, Australia\n",
    "        * __no__: trend rules\n",
    "      * (g) annual total of lynx trapped in McKenzie river district in Canada\n",
    "        * __yes__: the cycles are aperiodic\n",
    "      * (h) monthly autralian beer production\n",
    "        * __no__: seasonal \n",
    "      * (i) monthly australian electricity production\n",
    "        * __no__: seasonal \n",
    "        * __no__: trend rules\n",
    "        * __no__: increasing variance\n",
    "* ARIMA is a class of models of forecasting time series, which can be made _stationary_\n",
    "$$\\hat{y}_{t+1}=\\mu+\\alpha_0y_t+\\dots+\\alpha_{d-1}y_{t-d+1}+\\phi_0\\epsilon_t+\\dots+\\phi_{d-1}\\epsilon_{t-d+1}$$\n",
    "* __ARIMA(p, d, q)__\n",
    "  * __p__ number of autoregressive terms\n",
    "  * __d__ number of nonseasonal differences needed for stationarity,\n",
    "  * __q__ number of lagged forecast errors\n",
    "* __ARIMA(1,0,0) first-order autoregressive__ model $$\\hat{y}_{t+1}=\\mu+\\alpha_0y_t$$\n",
    "  * last value plus some constant\n",
    "  * in __second-order ARIMA(2,0,0)__ $$\\hat{y}_{t+1}=\\mu+\\alpha_0y_t+\\alpha_1y_{t-1}$$\n",
    "* __ARIMA(0,1,0) random-walk__ $$\\hat{y}_{t+1}=\\mu+y_t$$\n",
    "  * the constant term is the average period-to-period change\n",
    "* __ARIMA(1,1,0) differenced first-order autoregressive__ model\n",
    "$$\\hat{y}_{t+1}=\\mu+y_t+\\alpha_0(y_t-y_{t-1})$$\n",
    "  * if errors of random walk are auto-correlated, then problem can be probably fixed by adding a term with second dependent variable\n",
    "  * it represents the first difference of $y$ __lagged__ by one period\n",
    "* __ARIMA(0, 1,1) simple exponential smoothing with growth__\n",
    "$$\\hat{y}_{t+1}=\\mu+y_t-\\phi_0\\epsilon_t=\\mu+y_t-\\phi_0(y_t-\\hat{y}_t)$$\n",
    "  * where $\\epsilon_t$ is the error of prediction at tle last previous step\n",
    "  \n",
    "\n",
    "* ARIMA models accurately predict __short-term__ temporal structures\n",
    "  * performance drops for long term\n",
    "  * ARIMA assumes stationarity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Multi step-ahead forecasting strategies\n",
    "[after some works of Bontempi et al.]\n",
    "* __multi_step__, or __long term__ prediction predicts $H$ values $$[y_{N+1},\\dots,y_{N+H}],$$ where $H>1$ of a historical time series $[y_1,\\dots,y_N]$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Recursive strategy (Iterated, Multi-stage)\n",
    "* single model is trained to perform __one-step ahead__ forecast\n",
    "$$\\boxed{y_{t+1}=f(y_t, \\dots,y_{t-d+1}+w\\;\\;}$$\n",
    "* if we forecast $H>1$ steps, the value just forecasted is __used__ as part of input for next predictions\n",
    "$$\\boxed{\n",
    "\\hat{y}_{N+h}=\n",
    "\\begin{cases}\n",
    "\\hat{f}(y_N,\\dots,y_{N-d+1}),& \\text{if } h=1\\\\\n",
    "\\hat{f}(\\hat{y}_{N+h-1},\\dots,\\hat{y}_{N+1},y_N,\\dots,y_{N-d+h}),&\\text{if } h\\in\\{2,\\dots,d\\}\\\\\n",
    "\\hat{f}(\\hat{y}_{N+h-1},\\dots,\\hat{y}_{N+h-d})&\\text{if }h\\in\\{d+1,\\dots,H\\}\n",
    "\\end{cases}\\;\\;}$$\n",
    "  * recursive strategy depends on noise present\n",
    "  * if __forecasting horizon__ $h$ exceeds $d$ (__embedding dimension__), then some future points will be predicted __solely__ on forecasted values!\n",
    "  * accumulation of errors __with__ the forecasting horizon\n",
    "  * ML methods frequently used\n",
    "    * neural networks\n",
    "    * nearest-neighbours"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Direct strategy (Independent)\n",
    "* forecast __each__ horizon independently from others\n",
    "* train $H$ models $f_h$ $$\\boxed{\\hat{y}_{N+h}=\\hat{f}_h(y_N,\\dots,y_{N-d+1})\\;\\;}$$\n",
    "  * no previously approximated values are used as input (as in recursive)\n",
    "* $H$ models are trained __independently__\n",
    "  * conditional independence of forecasts is induced\n",
    "  * strategy does not take into account dependencies between $\\hat{y}_{N+h}$\n",
    "    * e.g. let the best forecasted sequence be linear\n",
    "    * Direct might return a \"broken\" curve\n",
    "  * many models have to be learned\n",
    "* ML methods frequently used\n",
    "  * neural networks\n",
    "  * nearest neighbours\n",
    "  * decision trees\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  DirRec\n",
    "* combines Direct and Recursive strategies\n",
    "  * compute forecasts with different models for every horizon (as Direct)\n",
    "  * at each step enlarges inputs with forecasts of previous step (like Recursive)\n",
    "  $$\\boxed{\n",
    "  \\hat{y}_{N+h}=\\hat{f}_h(y_{N+h-1},\\dots,y_{N-d+1})\\;\\;}$$ for $t\\in\\{d, N-H\\}$ and $h\\in\\{1, H\\}$\n",
    "  * we have to learn models\n",
    "  $$\\boxed{\n",
    "\\hat{y}_{N+h}=\n",
    "\\begin{cases}\n",
    "\\hat{f}(y_N,\\dots,y_{N-d+1}),& \\text{if } h=1\\\\\n",
    "\\hat{f}(\\hat{y}_{N+h-1},\\dots,\\hat{y}_{N+1},y_N,\\dots,y_{N-d+h}),&\\text{if } h\\in\\{2,\\dots,d\\}\n",
    "\\end{cases}\\;\\;}$$\n",
    "  * forecasts from previous steps are added\n",
    "  * embedding size __is not the same__ for all horizons\n",
    "  ---\n",
    "  * DirRec outperformed previous approaches on some datasets\n",
    "  * not so popular yet - young"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### MIMO Multi-Input Multi-Output (or Joint)\n",
    "* previous are combinations of single output approaches\n",
    "* idea: learn prediction which __does not avoid existence of stochastic dependencies between future values__\n",
    "$$\\boxed{\n",
    "[\\hat{y}_{N+H},\\dots,\\hat{y}_{N+1}]=\\hat{f}(y_N,\\dots,y_{N-d+1})\n",
    "\\;\\;}$$\n",
    "* should preserve dependencies between predicted values\n",
    "* avoids \n",
    "  * conditional independence assumption (like it is in Direct)\n",
    "  * accumulation of errors (as it is in Recursive)\n",
    "* all horizons predicted with the same model structure - may be a drawback\n",
    "* several succesfull applications "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DIRMO \n",
    "* to preserve pros of DIRect and miMO\n",
    "  * forecast $H$ outputs in $n$ blocks each of size $s$ ($H = n * s$)\n",
    "  * $s\\in\\{1,\\dots,H\\}$\n",
    "    * the blocks do not overlap\n",
    "    * how about overlapping blocks?\n",
    "    * or blocks that are \"mixed\", eg. \n",
    "    $$\\begin{align}\n",
    "    b_1=[\\hat{y}_{N+1},\\hat{y}_{N+3},\\hat{y}_{N+6},\\hat{y}_{N+10}]\\\\\n",
    "    b_2=[\\hat{y}_{N+2},\\hat{y}_{N+4},\\hat{y}_{N+7},\\hat{y}_{N+11}]\\\\\n",
    "    b_3=[\\hat{y}_{N+3},\\hat{y}_{N+5},\\hat{y}_{N+8},\\hat{y}_{N+12}]\\\\\n",
    "    b_4=[\\hat{y}_{N+4},\\hat{y}_{N+6},\\hat{y}_{N+9},\\hat{y}_{N+13}]\n",
    "    \\end{align}$$\n",
    "* the output block size needs to be calibrated, eg. some sort of cross validation\n",
    "  * may give trade-off between\n",
    "    * stochastic dependency bewteen future values\n",
    "    * flexibility of predictor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Speed comparison\n",
    "* training of multi-output is longer than single output\n",
    "* multi prediction depends on number of models needed, thus\n",
    "  * Recursive is the fastest\n",
    "  * MIMO second fastest\n",
    "  * DIRMO third (only $H/s$ models)\n",
    "  * Direct next ($H$ single-output models)\n",
    "  * DirRec last ($H$ single output models because of change of embedding size)\n",
    "* DIRMO may be slowest because of the need to tune block size $s$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Comparisons\n",
    "* __Recursive__\n",
    "  * good for noise free environments, \n",
    "  * __but__ accumulates error\n",
    "* __Direct__\n",
    "  * does not accumulate errors,\n",
    "  * __But__ but assumes conditional independence of outputs\n",
    "* __DirRec__\n",
    "  * is a trade-off between Recursive and Direct, \n",
    "  * __but__ input grows with $H$\n",
    "* __MIMO__\n",
    "  * does not assume conditional independence of outputs, \n",
    "  * __but__ has reduced flexibility because of the same model for all horizons\n",
    "* __DIRMO__\n",
    "  * is a trade-off between independence and dependence of outputs,\n",
    "  * __but__ needs block size tuning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ML approximation methods used in time series forecasting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Nearest Neighbour KNN\n",
    "* find the pattern in history nearest to $[y_t,y_t-1,\\dots,y_{t-d+1}]$\n",
    "* select as  output the value $y_p$\n",
    "  * consider more neighbours"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lazy Learning LL\n",
    "* local method which adapts the size of neighbourhood on the basis of a cross-validation criterion\n",
    "* let $x_q=[y_t,\\dots,y_{t-d+1}]$ be the __query__\n",
    "  1. sort increasingly vectors $y_{[1]},y_{[2]},y_{[3]},\\dots$ in $X$ with respect to distance to $x_q$\n",
    "  2. determine optimal number $k$ of neighbours, e.g. using PRESS statistic:\n",
    "    1. estimate $$\\hat{y}^{(k)}_q=\\frac{1}{k}\\sum y_{[j]}$$\n",
    "    2. where $k$ is estimated from error $e_{LOO}$ (Leave-One-Out)\n",
    "    $$\\begin{align}\n",
    "    e_{LOO}(k)&=\\frac{1}{k}\\sum_j^k(e_j(k))^2\\\\\n",
    "    e_j(k))&=y_{[j]}-\\frac{\\sum_{i\\neq{}j}^k y_{[i]}}{k-1}=\n",
    "    \\frac{(k-1)y_{[j]}+y_{[j]}-y_{[j]}-\\sum_{i\\neq{}j}^k y_{[i]}}{k-1}\\\\\n",
    "    &=\\frac{ky_{[j]}-\\sum_i^ky_{[i]}}{k-1}=\\frac{ky_{[j]}-ky_q^{(k)}}{k-1}=k\\frac{y_{[j]}-y_q^{(k)}}{k-1}\\\\\n",
    "    k^{\\ast}&=\\underset{k\\in\\{1,\\dots,K\\}}{\\arg\\,\\min}e_{LOO}(k)\n",
    "    \\end{align}$$\n",
    "  3. calculate prediction $\\hat{y}_{t+1}$\n",
    "* to use lazy learning in multi-step forecasting\n",
    "  * extend algorithm to us composition of one-step ahead aproach\n",
    "  * use other measures of discrepancy between forecasted values and training time sereis from history\n",
    "  \n",
    "* two main approaches to model selection in LL\n",
    "  * __winner-takes_all__ select the best $k$\n",
    "  * __combine__ average models $$\\hat{y}_q=\\sum_{k=2}^Kp_ky_q^{(k)} \\Big{/} \\sum_{k=2}^Kp_k$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Multi layer perceptron MLP\n",
    "* high number of parameters (number of hidden layers, number of neurons in layers)\n",
    "* relation between inputs in the embedding is not given __explicitely__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bayesian Neural Network BNN\n",
    "* network parameters (weights) treated as random variables obeying some _a priori_ distribution\n",
    "* distribution favours models with low complexity by minimizing $$J=\\alpha{}E_D+(1-\\alpha)E_W$$\n",
    "* given observed data, new distributions can be computed\n",
    "* predictions reflect \n",
    "  * __smoothness__ imposed through the prior\n",
    "  * __fitness accuracy__ imposed by observed data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Radial Basis Function Neural Networks RBF\n",
    "* a single layer neural network with hidden layer using Gaussian type activation\n",
    "$$\\hat{y}(x)=\\sum_j w_j\\exp\\left(-\\frac{\\|x-c_j\\|^2}{\\sigma^2}\\right)$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generalized Regression Neural Network GRNN (Nadaraya-Watson)\n",
    "* prediction given by the __average__ of target outputs of training points in vicinity of $x$\n",
    "* __local average__ constructed by weighting points by their distance from $x$ using some kernel function\n",
    "$$\\boxed{\n",
    "\\begin{align}\n",
    "\\hat{y}(x)&=\\sum_jw_jy_j\\\\\n",
    "w_j(x)&=\\frac{K\\left(\\frac{\\|x-x_j\\|}{h}\\right)}{\\sum_{k}K\\left(\\frac{\\|x-x_k\\|}{h}\\right)}\n",
    "\\end{align}\n",
    "\\;\\;}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Classification And Regression Trees CART\n",
    "* tree-like partition of the input space\n",
    "* terminal leaves return a value\n",
    "  * step-like function returned\n",
    "* needs pruning\n",
    "* low generalization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Support Vector Regression SVR\n",
    "* minimize the error function\n",
    "$$J=\\frac{1}{2}\\|w\\|^2+C\\sum_j|y_j-f(x_j)|_\\epsilon$$\n",
    "* the model uses _support vectors_ only\n",
    "$$f(x)=\\sum_j(\\alpha_j^*-\\alpha_j)K(x_j^Tx)+b$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gaussian Processes GP\n",
    "* nonparametric method based on modelling observed responses of different training points as __multivariate random variable__\n",
    "* posterior distribution of function to be predicted is found using assumed prior distribution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocessing\n",
    "* preprocessing may have __a big impact__ on performance, eg.\n",
    "  * deseasonalization of data\n",
    "  * detrending\n",
    "  * log-transformation\n",
    "  * series differencing\n",
    "    * well understood for linear models (eg. ARIMA)\n",
    "    * not always a good strategy for non-stationary series\n",
    "    * usually gives good results for the stationary case\n",
    "  * taking moving averages\n",
    "    * may smooth out noise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Model selection\n",
    "* K-fold cross-validation is preferred\n",
    "  * probably double"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Metrics\n",
    "* __symmetric mean absolute precentage error SMAPE__\n",
    "$$SMAPE=\\frac{1}{M}\\sum_m\\frac{|\\hat{y}_m-y_m|}{(|\\hat{y}_m|+|y_m|)/2}$$\n",
    "* ranking methods for comparing approaches\n",
    "  * compute the rank $R_q(p)$ of each method $q$ on each time series $p$\n",
    "  \n",
    "---\n",
    "* there are problems with comparisons\n",
    "* two groups used the same deterministic algorithm (1-NN) on the same data\n",
    "  * results differed by as much as 6%!\n",
    "  * the only difference was that one group rounded data to 6 decimal places while the other to 8 places!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comparisons\n",
    "[after Ahmed 2010]\n",
    "\n",
    "\n",
    "\n",
    "|LAG|   |DIFF |    |MAVG|\n",
    "---  |---  |--- |---  |--- |---  |---\n",
    "     |SMAPE|Rank|SMAPE|Rank|SMAPE|Rank\n",
    "MLP  |0.086|__2.78__|0.179|5.02|0.083|__2.88__\n",
    "BNN  |0.103|_3.42_|0.175|4.58|0.086|__2.94__\n",
    "RBF  |0.125|5.33|0.213|5.24|0.158|6.28\n",
    "GRNN |0.104|5.24|0.156|__3.92__|0.103|4.99\n",
    "KNN  |0.104|5.10|0.169|4.57|0.103|4.95\n",
    "CART |0.121|6.77|0.152|__3.93__|0.117|6.31\n",
    "SVR  |0.099|4.19|0.171|4.59|0.104|4.41\n",
    "GP   |0.095|_3.17_|0.165|_4.15_|0.096|_3.26_\n",
    "\n",
    "* best models within 95% confidence are in __bold__\n",
    "* second best are given in _italics_\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ensembles of model\n",
    "[after Oliveira & Torgo]\n",
    "* _embedding dimension_ is critical to solutions\n",
    "* task: find an __optimal__ value for embedding dimension\n",
    "  * there may not exists a __single__ correct value\n",
    "  * non-stationary series and changes of regime shifts may lead to the size to be __time-dependent__\n",
    "* diversity is a key ingredient to  boost performance\n",
    "  * generate ensemble models of different characteristics (model tried by authors)\n",
    "    1. __simple bagging__ several models of same embedding size; diversity from bootstrap samples\n",
    "    2. __extra predictors__ additional predictors added; values convey information on dynamics, eg. mean and variance values for the given embedding\n",
    "      * last week Damian proposed adding information on the regime a machine is working in\n",
    "    3. __extended diversity__ by varying embedding dimensions; on top of bootstrap samples\n",
    "    4. __extended + extra__ varying embedding dimensions plus extra predictors\n",
    "    5. __extended in groups__ mixture of the last two: only half of models have extra predictor input values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def _ls(datasets_dir):\n",
    "    return os.listdir(datasets_dir)\n",
    "\n",
    "def load_dataset(dataset):\n",
    "#    train = pd.read_csv(os.path.join(DATASETS_DIR, dataset, dataset + \"_TRAIN\"), header=None)\n",
    "#    test = pd.read_csv(os.path.join(DATASETS_DIR, dataset, dataset + \"_TEST\"), header=None)\n",
    "    train = pd.read_csv(os.path.join(DATASETS_DIR, dataset + \"_TRAIN\"), header=None)\n",
    "    test = pd.read_csv(os.path.join(DATASETS_DIR, dataset + \"_TEST\"), header=None)\n",
    "\n",
    "    X_train = train.ix[:,1:].values\n",
    "    y_train = train[0].values\n",
    "    X_test = test.ix[:,1:].values\n",
    "    y_test = test[0].values\n",
    "    return X_train, y_train, X_test, y_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#DATASETS_DIR = \"./UCR_TS_Archive_2015\"\n",
    "DATASETS_DIR = \".\"\n",
    "dataset = \"ElectricDevices\"\n",
    "X_train, y_train, X_test, y_test = load_dataset(dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import timeit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train: X.shape = (8926, 96), y.shape = (8926,)\n",
      "Test : X.shape = (7711, 96), y.shape = (7711,)\n"
     ]
    }
   ],
   "source": [
    "print(\"Train: X.shape = {}, y.shape = {}\".format(X_train.shape, y_train.shape))\n",
    "print(\"Test : X.shape = {}, y.shape = {}\".format(X_test.shape, y_test.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Time series classification\n",
    "[after Bagnall, Bostrom, Large, Lines]\n",
    "\n",
    "Let $(x, y)$ be an observation where $x$ is an $m$ element __time series__ TS\n",
    "\n",
    "* let $a=[a_1,\\dots,a_m], b=[b_1,\\dots,b_m]$ be series that we want to measure distance between\n",
    "* $M(a,b)$ is a pointwise distance matrix\n",
    "  * $M_{ij}=(a_i-b_j)^2$\n",
    "## Euclidean distance\n",
    "  $$d_E(a,b)=\\sum_i(a_i-b_i)^2$$\n",
    "  * distance along the diagonal\n",
    "## Dynamic Time Warping DTW\n",
    "  * __warping path__ $$P=[(e_1,f_1),\\dots,(e_s,f_s)]$$ is a set of pairs of indexes that define the traversal of $M$\n",
    "    * $(e_1,f_1)=(1,1)$ and $(e_s,f_s)=(m, m)$: start at the first point of sequences, and end at the last\n",
    "    * $0\\leq{}e_{i+1}-e_i\\leq1$ and $0\\leq{}f_{i+1}-e_i\\leq1$: stay in the same point (of $a$ or $b$) or proceed to the next\n",
    "  * the distance for $P$ is $$d=\\sum_iM_{a_{e_i}b_{f_i}}$$\n",
    "  * DTW is the distance over path $p^\\ast$ minimizing distance\n",
    "  * requires dynamic programming for exact values; approximations possible"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from dtw import dtw\n",
    "from fastdtw import fastdtw"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  1.   0.   1.   1.   4.   9.]\n",
      " [  0.   1.   0.   0.   1.   4.]\n",
      " [  4.   9.   4.   4.   1.   0.]\n",
      " [  1.   4.   1.   1.   0.   1.]\n",
      " [  4.   9.   4.   4.   1.   0.]\n",
      " [  9.  16.   9.   9.   4.   1.]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "a = np.array([0, 1, 3, 2, 3, 4]).reshape(-1, 1)\n",
    "b = np.array([1, 0, 1, 1, 2, 3]).reshape(-1, 1)\n",
    "m = np.zeros((len(a), len(b)))\n",
    "for i in range(len(a)):\n",
    "    for j in range(len(b)):\n",
    "        m[i, j] = (a[i] - b[j]) ** 2\n",
    "print(m)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x11addffd0>"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgUAAAFkCAYAAACw3EhvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3X+UnHV96PH3JwihERJ7boSggigoEKR4s5iC/AgpgiIQ\n0gLRLeGXhlbw56q19dxcktZjEFGiFNHbSwtRZCFqAuIvEAUiAg3dDUI1gBaRgEsket1QSRDJ9/4x\ns7hZdnbnmZ155pmZ9+ucPcd99pmdL3MwfDLPe75PpJSQJEma1OwFSJKkYnAokCRJgEOBJEkqcyiQ\nJEmAQ4EkSSpzKJAkSYBDgSRJKnMokCRJgEOBJEkqcyiQJEnABIeCiPiHiNgWEZeMc97REdEXEVsj\n4qGIOGsizytJkuqv5qEgIt4A/A3wo3HO2xv4BvA94GDgs8AVEXFsrc8tSZLqr6ahICJ2Aa4GFgG/\nHef084CHU0ofSSk9mFL6HPBVoKeW55YkSY1R6zsFnwNuTCl9v4pzDwVuGXHsJuCwGp9bkiQ1wIuy\nPiAi3g68HjikyofMADaOOLYRmBoRk1NKz4zyHP8DeDPwCLA16xolSepgOwN7AzellH6d5YGZhoKI\neAXwGeBNKaVnszw2ozcDX27g75ckqd2dDlyT5QFZ3ynoAl4K9EdElI/tABwVEe8BJqeU0ojHPAHs\nPuLY7sDm0d4lKHsE4Oqrr+aAAw7IuETVqqenh+XLlzd7GR3F1zx/vub58zVvvK1b4WMfg+98B/7q\nr9azatVCKP+3NIusQ8EtwEEjjl0FrAc+McpAAHAXcPyIY8eVj1eyFeCAAw5g1qxZGZeoWk2bNs3X\nO2e+5vnzNc+fr3ljPfEEzJ8P990HK1fCPvvAqlVADZffMw0FKaXfAT8Zfiwifgf8OqW0vvz9MuDl\nKaWhvQi+ALw7Ii4C/g04BjgVeGvWxUqSpD9atw7mzYNt22DNGjjkEOjvr/331WNHw5HvDuwB7Pn8\nD1N6BDgBeBNwL6WPIr4zpTTyEwmSJKlKq1bBEUfAjBmwdm1pIJiozJ8+GCml9Bcjvj9nlHPWUOoR\nJEnSBKQEy5bB4sWwYAFceSVMmVKf3+29D/S87u7uZi+h4/ia58/XPH++5vWzZQssXFgaCJYuhWuv\nrd9AABCjt4HNFRGzgL6+vj7jFEmS2D4oXLECTjtt9PP6+/vp6uoC6EopZSoMJnz5oFkeffRRNm3a\n1OxlFNL06dPZa6+9mr0MSVKdjBYUNkJLDgWPPvooBxxwAE8//XSzl1JIU6ZMYf369Q4GktQGVq2C\nM86AmTPh+uvh5S9v3HO15FCwadMmnn76aTc3GsX69etZuHAhmzZtciiQpBbWyKCwkpYcCoa4uZEk\nqR1t2QKLFsE115SCwgsugOf3EW6glh4KJElqNyN3KKwUFDaCQ4EkSQWRV1BYifsUSJJUAI3YoTAr\nhwJJkpooJfj4x+GUU+DEE+H22xv7CYOxOBRIktQkjd6hMCubAkmSmqCZQWElDgWSJOWs2UFhJV4+\nkCQpR0UICitxKCiYRx99lPPPP5/999+fKVOmMH36dBYsWMAvfvGLZi9NkjQBRQoKK/HyQcHcc889\n3H333XR3d/OKV7yCRx55hMsvv5y5c+fyk5/8hJ133rnZS5QkZdSsHQqzcigomBNPPJFTTjllu2Mn\nnXQShx56KF/72tc4/fTTm7QySVItihgUVtIRQ8HTT8MDDzT2Ofbfvz4fI5k8efLz//sPf/gDmzdv\n5tWvfjUveclL6O/vdyiQpBZS1KCwko4YCh54ALq6GvscfX1Qj3szbd26lWXLlnHVVVfx+OOPk1IC\nICIYHByc+BNIknKR5y2P66UjhoL99y/9R7vRz1EP73nPe1ixYgU9PT0ceuihTJs2jYjgbW97G9u2\nbavPk0iSGqYZtzyul44YCqZMqc/f4vPwta99jbPPPptPfvKTzx975pln+O1vf9vEVUmSqtEqQWEl\nHTEUtJIddtjhBe8IXHrppTz33HNNWpEkqRqtFBRW4lBQMCeeeCJf+tKXmDp1KjNnzuSuu+7ie9/7\nHtOnT2/20iRJFbRaUFiJQ0HBXHrppbzoRS/immuuYevWrRxxxBHccsstvPnNbyZa6T0oSeoQrRgU\nVuJQUDBTp07liiuueMHxhx9+uAmrkSRV0spBYSVucyxJUkZFu+VxvfhOgSRJGbRDUFiJQ4EkSVVq\nl6CwkkyXDyLiXRHxo4gYLH/dGRFvGeP8ORGxbcTXcxGx28SXLklSfop8y+N6ydoUbAD+HpgFdAHf\nB26IiAPGeEwCXgPMKH/tkVL6VQ1rlSQpd61wy+N6yXT5IKX0zRGHFkfEecChwPoxHvpkSmlz1sVJ\nktRMrb5DYVY1NwURMQlYAEwB7hrrVODeiNgZ+E9gaUrpzlqfV5KkPLRzUFhJ5qEgIl5HaQjYGXgK\n+MuUUqUbEw8Afwv8BzAZOBe4LSJmp5TurW3JkiQ1VrsHhZXU8k7BA8DBwDTgVOCLEXHUaINBSukh\n4KFhh+6OiH2AHuCs8Z6op6eHadOmbXesu7ub/fbbr4ZlS5I0vlbaobC3t5fe3t7tjg0ODtb8+zIP\nBSmlPwBD2+uti4jZwPuB86r8FWuBw6s5cfny5cwa5faG/f39VT6VJEnVacUdCru7u+nu7t7uWH9/\nP11dXTX9vnrsaDiJ0qWBar2e0mUFSZIKoV13KMwq6z4FyyLiyIh4ZUS8LiIuBOYAV5d/fmFErBh2\n/vsjYl5E7BMRB0bEZ4C5wGX1/IdoN0uXLmXSpEn85je/afZSJKntPfEEzJ0Lq1eXgsIlS9r7EwZj\nyXr5YDdgBbAHMAjcBxyXUvp++eczgD2Hnb8T8GngZcDT5fOPSSmtmcii211EeEdEScpBpwaFlWTd\np2DROD8/Z8T3FwMX17AuSZIaqpWCwrx4l0RJUkfppB0Ks3IoKLAnn3ySBQsWMG3aNKZPn84HPvAB\nnnnmmWYvS5JalkHh2LxLYkGllFiwYAGvetWr+MQnPsHdd9/NpZdeym9/+1uuuuqqZi9PklpOJ+5Q\nmJVDQYHts88+rFq1CoDzzjuPXXfdlc9//vN8+MMf5nWve12TVydJrcOgsDodMRQ8/ezTPLCp0k7M\n9bH/9P2ZsmP93oOKCN797ndvd+y9730vl19+Od/61rccCiSpSgaF1euIoeCBTQ/Q9S+17e5Urb6/\n6WPWHi/cfXEi9t133+2+32effZg0aRKPPPJIXZ9HktpRK+5Q2GwdMRTsP31/+v6mr+HP0WjuXSBJ\n1em0Wx7XS0cMBVN2nFL3v8Xn4ac//SmvfOUrn//+Zz/7Gdu2bWPvvfdu3qIkqeAGBkpB4f33GxRm\n5UcSCyqlxOc+97ntjl166aVEBMcff3yTViVJxbZuHcyeDY89VgoKHQiy6Yh3ClrVz3/+c04++WTe\n8pa3cOedd/LlL3+ZhQsXctBBBzV7aZJUOAaFE+c7BQU1adIkrrvuOiZPnsxHP/pRvv3tb/O+972P\nK664otlLk6RCcYfC+vGdggJasmQJS5YsAWDlypVNXo0kFZdBYX05FEiSWpJBYf05FEiSWo47FDaG\nTYEkqaWsWgVHHAEzZsDatQ4E9eRQIElqCQaFjedQIEkqPG95nA+bAklSoRkU5sehQJJUWAaF+fLy\ngSSpkAwK89fS7xSsX7++2UsoHF8TSa3OWx43T0sOBdOnT2fKlCksXLiw2UsppClTpjB9+vRmL0OS\nMnOHwuZqyaFgr732Yv369WzatKnZSymk6dOns9deezV7GZKUiUFh87XkUAClwcD/8ElSezAoLAZD\nQ0lSUxkUFodDgSSpKdyhsHgcCiRJuXOHwmJq2aZAktSaDAqLK9M7BRHxroj4UUQMlr/ujIi3jPOY\noyOiLyK2RsRDEXHWxJYsSWpV69bB7Nnw2GOloNCBoFiyXj7YAPw9MAvoAr4P3BARB4x2ckTsDXwD\n+B5wMPBZ4IqIOLbG9UqSWpRBYfFlGgpSSt9MKX0npfRfKaWfpZQWA/8NHFrhIecBD6eUPpJSejCl\n9Dngq0DPxJYtSWoVBoWto+bQMCImRcTbgSnAXRVOOxS4ZcSxm4DDan1eSVLrMCjMX0qp5sdmHgoi\n4nUR8RTwDHA58JcppQcqnD4D2Dji2EZgakRMzvrcUrv4/e/h3HPh299u9kqkxhkYgLlzYfXqUlC4\nZIlbFjfawFMDnHvjuTU/vpZPHzxAqQ+YBpwKfDEijhpjMKhZT08P06ZN2+5Yd3c33d3d9X4qKVfL\nl8MVV8C//it86lPQ0+Mflmov7lCYj97eXnp7ewEY3DrI2l+uZduWbTX/vpjI2wwAEfFd4GcppfNG\n+dntQF9K6YPDjp0NLE8p/ekYv3MW0NfX18esWbMmtD6paDZsgP33L71TsPPOcNFF8I53wOc/Dzvt\n1OzVSRO3ahWccQbMnAnXX28/kIdV61dxxuozmPnSmXxsv49x/JzjAbpSSv1Zfk89Ni+aBFS6FHAX\ncMyIY8dRuUGQ2t6HPgRTp8I//RN84hOwYgVcfTUceyx4jy+1MoPC/KWU+Piaj3PKylM48bUncvvZ\nt7PbLrvV/Puy7lOwLCKOjIhXltuCC4E5wNXln18YESuGPeQLwKsj4qKI2C8izqd0yeGSmlcstbDv\nfhe+8pXSJYOpU0vHzjwTbr0V1q8vfX77xz9u7hqlWhgU5m/Ls1tYuHohi29dzNI5S7n2lGuZsuPE\nXvSsTcFuwApgD2AQuA84LqX0/fLPZwB7Dp2cUnokIk4AlgPvAx4D3plSGvmJBKnt/f738N73wlFH\nwV//9fY/e+Mb4Z574KST4LDDoLcXTjihOeuUsnKHwvwNPDXA/Ovmc//G+1l56kpOO7A+L3qmoSCl\ntGicn58zyrE1lDY6kjra8uXws5+V3ikYLSp85Svhhz8s/W3rpJMMENUaDArzt25gHfOunce2tI01\n56zhkJfV70X3hkhSDjZsKDUE73sfHHRQ5fN23bX08a2PfKTUHixaVHqHQSoidyjM36r1qzjiyiOY\nscsM1i5aW9eBABwKpFwMxYVLl45/7qRJBogqNoPC/I0WFL58av1fdIcCqcFGiwurYYCoIjIozF8j\ngsJKHAqkBhorLqzGUIC4yy6lAPGb36z/GqVqDQzA0Ue7Q2GeBp4a4OgVR7N6/WpWnrqSJUcvIRr4\nojsUSA00FBdedlntf3gOBYhz55YCxEsuKb19K+XJWx7nb93AOmZfMZvHNj/GmnPW1O0TBmNxKJAa\npNq4sBoGiGomg8L8NToorMShQGqQLHFhNQwQlTeDwvzlFRRW4lAgNUCtcWE1DBCVB4PC/OUZFFbi\nUCDV2UTjwmoYIKqRDArzl3dQWIlDgVRn9YgLq2GAqEYwKMxfM4LCShwKpDqqZ1xYDQNE1ZNBYf6a\nFRRW4lAg1VG948JqGCBqogwK89fsoLAShwKpThoZF1bDAFG1MCjMXxGCwkocCqQ6yCMurIYBorIw\nKMxfUYLCShwKpDrIKy6shgGiqmFQmL8iBYWVOBRIE5R3XFgNA0SNxaAwf0ULCitxKJAmqBlxYTUM\nEDWSQWH+ihoUVuJQIE1As+PCahggCgwKm6HIQWElDgVSjYoSF1bDALGzGRTmr+hBYSUOBVKNihQX\nVsMAsTMZFOavFYLCShwKpBoUMS6shgFiZzEozF+rBIWVOBRINShqXFgNA8T2Z1CYv1YLCitxKJAy\naoW4sBoGiO3JoDB/rRgUVuJQIGXQSnFhNQwQ24tBYf5aNSisxKFAyqDV4sJqGCC2B4PC/LVyUFiJ\nQ4FUpVaNC6thgNjaDArz1+pBYSUOBVKVWjkurIYBYusxKMxfuwSFlWQaCiLioxGxNiI2R8TGiFgd\nEa8d5zFzImLbiK/nImK3iS1dyk+7xIXVMEBsDQaF+WunoLCSrO8UHAn8M/DnwJuAHYGbI+JPxnlc\nAl4DzCh/7ZFS+lXG55aaot3iwmoYIBabQWH+2i0orORFWU5OKb11+PcRcTbwK6ALuGOchz+ZUtqc\naXVSAQzFhV/5Smf9wTsUIC5cWAoQP/Up6OnprNegiNatg3nzYNu2UlBoP9B46wbWMe/aeWxL21hz\nzpq26QdGM9Gm4CWU3gX4zTjnBXBvRPwyIm6OiDdO8HmlXLRzXFgNA8RiMSjMX7sGhZXUPBRE6X2T\nzwB3pJR+MsapA8DfAqcAfwVsAG6LiNfX+txSXto9LqyGAWLzGRTmr92DwkoyXT4Y4XJgJnD4WCel\nlB4CHhp26O6I2AfoAc6awPNLDTUUF159dfvHhdU480zYd1+YP78UIN54Ixx4YLNX1f62bCm9Q3PN\nNaXh9IILvITTaFue3cKiGxdxzf3XsHTOUi6Yc0Fb9gOjiVTDLiURcRlwEnBkSunRGh7/SeDwlNKo\nA0VEzAL6jjrqKKZNm7bdz7q7u+nu7s68ZimL3/8e/uzPYPfd4bbb/EN4uF/8otQYPPII9PbCCSc0\ne0Xta2CgNITdf3/pnRo3JGq8gacGmH/dfO7feD8r5q8o/IZEvb299Pb2bndscHCQNWvWAHSllPqz\n/L7MQ0F5IDgZmJNSejjTg//4O24GNqeUTq3w81lAX19fH7NmzarlKaQJuegi+F//qxR1dWJLMJ6n\nnioFiDfeaIDYKMODwhtusB/Iw/Cg8Ia339Cy/UB/fz9dXV1Qw1CQdZ+Cy4HTgb8GfhcRu5e/dh52\nzrKIWDHs+/dHxLyI2CciDoyIzwBzgcuyPLeUl06PC6ux666l6M0AsTEMCvPXaUFhJVlDw3cBU4Hb\ngF8O+1ow7Jw9gD2Hfb8T8GngvvLjDgKOSSndVsuCpUYzLqzODjsYINabQWH+OjUorCTrPgXjDhEp\npXNGfH8xcHHGdUlNYVyYnQFifRgU5q+Tg8JKvPeBVNaJOxfWizsgTow7FOavU3YozMqhQCprx9si\n58lbMNfGWx7nrx1veVwvDgUSxoX1YoCYjUFh/gwKx+ZQIGFcWE8GiOMzKMyfQWF1HArU8Trptsh5\n8hbMo/OWx/nrhFse14tDgTqacWFjGSBuz6AwfwaF2TgUqKMZFzaeAWKJQWH+DAqzcyhQxzIuzE+n\nB4gGhfkzKKyNQ4E61gc/aFyYp04MEA0K82dQODEOBepI3/0ufPWrxoXN0CkBokFh/gwKJ86hQB3H\nuLD52j1ANCjMn0FhfTgUqOMYFxZDuwaIBoX5MyisH4cCdRTjwmJptwDRoDB/BoX15VCgjmJcWDzt\nECAaFObPoLAxHArUMYwLi61VA0SDwvwZFDaOQ4E6gnFha2i1ANGgMH8GhY3lUKCOYFzYOlolQDQo\nzJ9BYeM5FKjtGRe2nqIHiAaF+TMozIdDgdqecWFrKmKAaFCYP4PCfDkUqK0ZF7a+ogSIBoX5MyjM\nn0OB2pZxYftodoBoUJg/g8LmcChQ2zIubC/NChANCvNnUNg8DgVqS8aF7SnvANGgMH8Ghc3lUKC2\nZFzYvvIIEA0K82dQWAwOBWo7xoWdoVEBokFh/gwKi8OhQG3FuLCz1DtANCjMn0FhsTgUqK0YF3ae\negWIBoX5MygsHocCtQ3jws410QDRoDB/BoXFlGkoiIiPRsTaiNgcERsjYnVEvLaKxx0dEX0RsTUi\nHoqIs2pfsjQ648LOVkuAaFCYP4PCYsv6TsGRwD8Dfw68CdgRuDki/qTSAyJib+AbwPeAg4HPAldE\nxLE1rFcalXGhhlQbIBoU5s+gsPhelOXklNJbh38fEWcDvwK6gDsqPOw84OGU0kfK3z8YEUcAPcB3\nM61WGoVxoUYaChBPOqkUIPb2wgkn/PHnAwMwfz7cf38pKLQfaLyBpwaYf9187t94PytPXWk/UFAT\nbQpeAiTgN2Occyhwy4hjNwGHTfC5JcC4UKOrFCAaFObPoLB11DwUROkzI58B7kgp/WSMU2cAG0cc\n2whMjYjJtT6/BMaFGtvIAHHePIPCvBkUtpZMlw9GuByYCRxep7W8QE9PD9OmTdvuWHd3N93d3Y16\nSrUY40KNZyhAnDkTzj23dNngyivtBxotpcSyHyxj8a2LWXDgAq48+Ur7gQbo7e2lt7d3u2ODg4M1\n/75INXygNyIuA04CjkwpPTrOubcDfSmlDw47djawPKX0pxUeMwvo6+vrY9asWZnXp87w3e/CcceV\nSvPTT2/2atQKNm8uvXvgZabG2vLsFhbduIhr7r+GpXOWcsGcC9yQKEf9/f10dXUBdKWU+rM8NvM7\nBeWB4GRgzngDQdldwPEjjh1XPi7VxLhQtfCTKY1nUNjaMg0FEXE50A3MA34XEbuXfzSYUtpaPmcZ\n8PKU0tBeBF8A3h0RFwH/BhwDnAps90kGKYuhuPArX/FvfVJRrBtYx7xr57EtbWPNOWvsB1pQ1tDw\nXcBU4Dbgl8O+Fgw7Zw9gz6FvUkqPACdQ2tfgXkofRXxnSmnkJxKkqhgXSsVjUNgesu5TMO4QkVI6\nZ5RjayjtZSBNmHGhVBwGhe1lIp8+kHI3tHPh1Vd7fVhqNoPC9uNQoJZhXCgVh0Fhe3IoUMswLpSK\nwaCwfXnrZLUE40KpGAwK25tDgVqCcaHUXN7yuDN4+UCFZ1woNZdBYedwKFChGRdKzWVQ2FkcClRo\nxoVS8xgUdh6bAhWWcaHUPAaFncmhQIVlXCjlz6Cws3n5QIVkXCjlz6BQDgUqHONCKX8GhQKHAhWQ\ncaGUL4NCDbEpUKEYF0r5MijUcA4FKhTjQikfBoUajZcPVBjGhVI+DApViUOBCsG4UMrHE//9BPOv\nnc99G+8zKNQLOBSoEIwLpcYzKNR4bArUdMaFUuMZFKoaDgVqOuNCqXEMCpWFlw/UVMaFUuMYFCor\nhwI1zTPPwHveY1woNYJBoWrhUKCmWb4c/uu/Su8U+JcXqX4MClUrmwI1xYYN8LGPGRdK9WZQqIlw\nKFBTGBdK9WVQqHrw8oFyZ1wo1ZdBoerFoUC5Mi6U6sugUPXkUKBcGRdK9WNQqHrL3BRExJER8fWI\neDwitkXEvHHOn1M+b/jXcxGxW+3LVisyLpTqx6BQjVBLaPhi4F7gfCBV+ZgEvAaYUf7aI6X0qxqe\nWy3MuFCaOINCNVLmywcppe8A3wGIbCXLkymlzVmfT+3BuFCaOINCNVpeTUEA90bEzsB/AktTSnfm\n9NxqMuNCaeIMCpWHPIaCAeBvgf8AJgPnArdFxOyU0r05PL+azLhQmhiDQuWl4UNBSukh4KFhh+6O\niH2AHuCssR7b09PDtGnTtjvW3d1Nd3d33depxjAulCZm1fpVnLH6DGa+dCbXv+16+wFtp7e3l97e\n3u2ODQ4O1vz7IqVqW8FRHhyxDZifUvp6xsd9Ejg8pXR4hZ/PAvr6+vqYNWtWzetT8512GtxxBzz4\noC2BlEVKiWU/WMbiWxez4MAFXHnylUzZcUqzl6UW0N/fT1dXF0BXSqk/y2ObtU/B6yldVlAbMy6U\namNQqGbJPBRExIuBfSnFgwCvjoiDgd+klDZExIXAy1JKZ5XPfz/wc+DHwM6UmoK5wLF1WL8KyrhQ\nqo1BoZqplncKDgFupbT3QAI+XT6+AngHpX0I9hx2/k7lc14GPA3cBxyTUlpT45rVAowLpewMCtVs\ntexTcDtjbHqUUjpnxPcXAxdnX5palXGhlJ1BoYrAWyer7ty5UKqeOxSqSLwhkurKuFCqnkGhisah\nQHVjXChVz6BQReRQoLoxLpSqY1CoorIpUF0YF0rV8ZbHKjKHAtWFcaE0NoNCtQIvH2jCjAulsRkU\nqlU4FGhCjAulsRkUqpU4FGhCjAulygwK1WpsClQz40KpMoNCtSKHAtXMuFB6IYNCtTIvH6gmxoXS\nCxkUqtU5FCgz40LphQwK1Q4cCpSZcaG0PYNCtQubAmViXChtz6BQ7cShQJkYF0olBoVqR14+UNWM\nC6USg0K1K4cCVcW4UCoxKFQ7cyhQVYwLJYNCtT+bAo3LuFAyKFRncCjQuIwL1ckMCtVJvHygMRkX\nqpMZFKrTOBSoIuNCdTKDQnUihwJVZFyoTmVQqE5lU6BRGReqUxkUqpM5FGhUxoXqNAaFkpcPNArj\nQnUag0KpxKFA2zEuVKcxKJT+KPPlg4g4MiK+HhGPR8S2iJhXxWOOjoi+iNgaEQ9FxFm1LVeNNhQX\nXnaZcaHa37qBdbzh/76BDZs3sOacNQ4E6ni1NAUvBu4FzgfSeCdHxN7AN4DvAQcDnwWuiIhja3hu\nNZBxoTqJQaH0QpkvH6SUvgN8ByCqu+h2HvBwSukj5e8fjIgjgB7gu1mfX41jXKhOkFJi2Q+WsfjW\nxSw4cAFXnnwlU3ac0uxlSYWQR1NwKHDLiGM3ActzeG5VybhQncCgUBpbHkPBDGDjiGMbgakRMTml\n9EwOa9AYBgbg3e82LszTwFMD9NzUwxP//USzl9JRHn/qcR7f/LhBoVRBoT99cNo7T+PAPQ/c7lh3\ndzfd3d1NWlH7WbcO5s2DbdvgX/7FuDAPQ7vlPbftOf7iVX/R7OV0lL1fsjfvnf1eul7W1eylSHXR\n29tLb2/vdscGBwdr/n15DAVPALuPOLY7sHm8dwkenv0wc0+Yy+UnXM5OO+zUsAV2qlWr4IwzYOZM\nuP56eLn7tDTcqvWrOGP1GRww/QBuePsNbo4jaUJG+4tyf38/XV21Db557Gh4F3DMiGPHlY+P6R/n\n/iNfuu9LHPulY9n09KaGLK4TpQQf/ziccgqceCLcfrsDQaMN3y3vhNecwJpz1jgQSCqcWvYpeHFE\nHBwRry8fenX5+z3LP78wIlYMe8gXyudcFBH7RcT5wKnAJeM914mvPZFbz7qV9U+uZ/b/nc2Pf/Xj\nrMvVCFu2wOmnw+LFpU8ZXHstTDG8bqgtz25h4eqFLL51MUvmLOHaU6+1dpdUSLW8U3AIsA7oo7RP\nwaeBfuAfyz+fAew5dHJK6RHgBOBNlPY36AHemVIa+YmEUb1xzzdyz7n3sMtOu3DYvx7Gt376rRqW\nLCgFhUcfXbpUsHIlLFliQ9BoA08NcPSKo1m1fhXXnXodS49eyqTwliOSiqmWfQpuZ4xhIqV0zijH\n1gA1lz3AvtLKAAANT0lEQVSvfMkr+eE7fsjC1Qs5qfckPnXsp/jAoR/wo0QZDA8K16yBQ9ynpeGG\nB4Vrzl7DG17+hmYvSZLG1DJ/Zdl18q6sfttq/u6Nf8cHb/4g5954Lr9/7vfNXlZLWLUKjjgCZsyA\ntWsdCPIwtFve7i/enXvOvceBQFJLaJmhAGBSTOITb/oEK+avMECsgkFh/gwKJbWylhoKhpx58JkG\niOMwKMyfQaGkVteSQwEYII7FoDB/BoWS2kFL/6k1FCDOfdVcTuo9ieV3LSelcW/c2NbWrYPZs+Gx\nx0pB4Wnu5Npw6wbWMfuK2WwY3MCas9ew4MAFzV6SJNWkpYcCMEAczqAwfwaFktpJyw8FYIBoUJg/\ng0JJ7agthoIhnRggGhTmb3hQuHTOUq479TqDQkltoa2GAuisANGgMH8jg8IlRy9xEy1JbaPthgLo\njADRoDB/BoWS2l1bDgXQ3gGiQWH+DAoldYK2HQqg/QJEg8L8GRRK6iRtPRQMaYcA0aAwfwaFkjpN\nRwwF0NoBokFh/gwKJXWijhkKoDUDRIPC/BkUSupUHTUUQGsFiAaF+TMolNTJOm4ogOIHiAaF+TMo\nlKQOHQqGFDFANCjMn0GhJJV09FAAxQoQDQrzZ1AoSX/U8UMBFCNANCjMn0GhJG3PoaCsmQGiQWH+\nDAol6YUcCobJO0A0KMzf8KDwxNeeaFAoScM4FIwijwBxyxZYuNCgME8jg8JrT7nWoFCShnEoqKCR\nAeJQULh6tUFhXgwKJWl8DgVjaESAaFCYP4NCSaqOQ8E46hkgGhTmz6BQkqrnUFCFiQaIBoX5MyiU\npOxqGgoi4t0R8fOI2BIRd0dExb9+RcSciNg24uu5iNit9mU3Ry0BokFh/gwKJak2mYeCiHgb8Glg\nCfA/gR8BN0XE9DEeloDXADPKX3uklH6VfbnNlyVANCjMn0GhJNWulncKeoD/k1L6YkrpAeBdwNPA\nO8Z53JMppV8NfdXwvIVRTYBoUJi/oaDwsc2P8YNzfmBQKEkZZRoKImJHoAv43tCxVPqv4S3AYWM9\nFLg3In4ZETdHxBtrWWyRjBUgGhTmbygonLHLDNYuWsshL/NFl6SsXpTx/OnADsDGEcc3AvtVeMwA\n8LfAfwCTgXOB2yJidkrp3ozPXyhDAeLMl87k3BvP5ae//imHP/41Lvzf01mwAK680n6g0VJKLPvB\nMhbfupgFBy7gypOvtB+QpBplHQoySyk9BDw07NDdEbEPpcsQZzX6+fNw5sFnsueL9+X4FfNZ89Rs\nzl96I5ddcKD9QINteXYLi25cxDX3X8PSOUu5YM4F9gOSNAFZh4JNwHPA7iOO7w48keH3rAUOH++k\nnp4epk2btt2x7u5uuru7MzxV4w0MwD+c/kb4xT3s9YGT+FI6jBN+di1vfc1bm720tjXw1ADzr5vP\nfRvv47pTr7MfkNSRent76e3t3e7Y4OBgzb8vsu7QFxF3A/+eUnp/+fsAHgUuTSldXOXvuBnYnFI6\ntcLPZwF9fX19zJo1K9P68rZuHcybB9u2wQ03wH4HPcXC1Qv5xkPf4OJjL6bn0B7/9lpn6wbWMe/a\neWxL27jh7TfYD0jSMP39/XR1dQF0pZT6szy2lk8fXAKcGxFnRsT+wBeAKcBVABFxYUSsGDo5It4f\nEfMiYp+IODAiPgPMBS6r4bkLZbSgcNfJu7JqwSr+7o1/x4du/hCLvr4ot1swdwKDQklqnMxDQUpp\nJfBh4J+AdcCfAW9OKT1ZPmUGsOewh+xEaV+D+4DbgIOAY1JKt9W86iYbb4fCHSbt8PwOiFfff3XD\nb8HcCUbuUHj72be7Q6Ek1Vnmywd5KPLlgy1bYNEiuOaa0g6FF1ww9oZEd264k/nXzmeXnXbhxu4b\nOXC3A3Nba7swKJSk6uV9+aBj1bJDYSNvwdwJ3KFQkvLjUFCliexQOHIHxEvuumTCt2DuBO5QKEn5\nciioQj12KDRAzMagUJLy51Awhnrf8tgAcXwGhZLUPA4FFTTylse13IK5E3jLY0lqLoeCUeRxy2MD\nxO0ZFEpS8zkUjJDnLY8NEEsMCiWpGBwKhmnGLY87PUA0KJSk4nAooP5BYVadGCAaFEpS8XT8UNDI\noDCrTgkQDQolqZg6eih44gmYO7exQWFW7R4gGhRKUnF17FCwbh284Q2wYUPjg8Ks2jVANCiUpGLr\nyKGgGUFhVu0WIBoUSlLxddRQ0OygMKt2CBANCiWpdXTMUFCkoDCrVg0QDQolqbV0xFBQxKAwq5EB\n4jcf+mazlzSmoaBw9frVrDx1pUGhJLWAth8KihwUZtUqAeLwoHDNOWs47cAWftElqYO09VDQCkFh\nVkMB4kcO/0ghA0SDQklqXW05FLRaUJhVEQNEg0JJan1tNxS0clCYVVECRINCSWoPbTUUtENQmFWz\nA0SDQklqH20zFLRTUJhVswJEg0JJai9tMRS0Y1CYVd4BokGhJLWflh4K2j0ozCqPANGgUJLaV8sO\nBZ0UFGbVqADRoFCS2ltLDgWdGBRmVe8A0aBQktpfyw0FnRwUZlWvANGgUJI6Q0sNBQaF2WUJEHt7\ne19wzKCwsUZ7zdVYvub58zVvHTUNBRHx7oj4eURsiYi7I+IN45x/dET0RcTWiHgoIs7K8nwGhRNT\nbYA4/P+4BoX58A/L/Pma58/XvHVkHgoi4m3Ap4ElwP8EfgTcFBHTK5y/N/AN4HvAwcBngSsi4thq\nns+gsH6qDRANCiWpM9XyTkEP8H9SSl9MKT0AvAt4GnhHhfPPAx5OKX0kpfRgSulzwFfLv2dMmzYZ\nFNbbeAGiQaEkda5MQ0FE7Ah0UfpbPwCpVK7dAhxW4WGHln8+3E1jnP+8M84wKGyESgHi4NZBg0JJ\n6mAvynj+dGAHYOOI4xuB/So8ZkaF86dGxOSU0jOjPGZngClT1vP5z8OkSdDfn3GlGtfifRfzp//v\nT/nQVR/i+luv54c//SH7zd6PS467hElPTKL/CV/0RhscHKTff7lz5WueP1/zfK1fv37of+6c9bGR\n5SNqEbEH8DhwWErp34cdvwg4KqX0gr/9R8SDwL+llC4adux4Sp3BlNGGgoj4a+DLWf5BJEnSdk5P\nKV2T5QFZ3ynYBDwH7D7i+O7AExUe80SF8zdXeJcASpcXTgceAbZmXKMkSZ1sZ2BvSv8tzSTTUJBS\nejYi+oBjgK8DRKlCOwa4tMLD7gKOH3HsuPLxSs/zayDTdCNJkp53Zy0PquXTB5cA50bEmRGxP/AF\nYApwFUBEXBgRK4ad/wXg1RFxUUTsFxHnA6eWf48kSSqIrJcPSCmtLO9J8E+ULgPcC7w5pfRk+ZQZ\nwJ7Dzn8kIk4AlgPvAx4D3plSGvmJBEmS1ESZQkNJktS+WureB5IkqXEcCiRJElDAoSDrzZY0MRFx\nZER8PSIej4htETGv2WtqZxHx0YhYGxGbI2JjRKyOiNc2e13tLCLeFRE/iojB8tedEfGWZq+rk0TE\nP5T/fDEwb5CIWFJ+jYd//STr7ynUUJD1ZkuqixdTikXPBwxMGu9I4J+BPwfeBOwI3BwRf9LUVbW3\nDcDfA7MobdP+feCGiDigqavqEOW/2P0NpT/P1Vj/SekDADPKX0dk/QWFCg0j4m7g31NK7y9/H5T+\nD31pSumTTV1cB4iIbcD8lNLXm72WTlEeeH9FaUfQO5q9nk4REb8GPpxSurLZa2lnEbEL0Efpxnj/\nG1iXUvpgc1fVniJiCXBySmnWRH5PYd4pqPFmS1Krewmld2h+0+yFdIKImBQRb6e0t0rFDdRUN58D\nbkwpfb/ZC+kQrylfCv6viLg6IvYc/yHby7xPQQPVcrMlqWWV3wn7DHBHSinztT9VLyJeR2kI2Bl4\nCvjL8q3f1SDl4ev1wCHNXkuHuBs4G3gQ2ANYCqyJiNellH5X7S8p0lAgdZrLgZnA4c1eSAd4ADgY\nmEZpR9UvRsRRDgaNERGvoDTwviml9Gyz19MJUkrD73PwnxGxFvgFsACo+jJZkYaCWm62JLWkiLgM\neCtwZEppoNnraXcppT8AD5e/XRcRs4H3U7rWrfrrAl4K9JffEYPSO8FHRcR7gMmpSEFbG0opDUbE\nQ8C+WR5XmKagPE0O3WwJ2O5mSzXd2EEqovJAcDIwN6X0aLPX06EmAZObvYg2dgtwEKXLBweXv/4D\nuBo42IGg8cqR575Apr90FOmdAijdJOmq8p0Y1wI9DLvZkuovIl5M6V+coWn+1RFxMPCblNKG5q2s\nPUXE5UA3MA/4XUQMvTM2mFLyNuENEBHLgG8DjwK7Urot+xxKd2tVA5SvYW/XyUTE74Bfp5TWN2dV\n7S0iLgZupHTJ4OXAPwLPAr1Zfk+hhoIqbrak+jsEuJVSAZ8o7RMBsAJ4R7MW1cbeRel1vm3E8XOA\nL+a+ms6wG6V/n/cABoH7gOMs4nPnuwON9QrgGuB/AE8CdwCHppR+neWXFGqfAkmS1DyFaQokSVJz\nORRIkiTAoUCSJJU5FEiSJMChQJIklTkUSJIkwKFAkiSVORRIkiTAoUCSJJU5FEiSJMChQJIklf1/\nPv+XAhFd7xUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11a925490>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot(a, label='a')\n",
    "plot(b, label='b')\n",
    "legend(loc='best')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def squared_norm(x, y):\n",
    "    return (x - y) ** 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def l1_norm(x, y):\n",
    "    return np.abs(x - y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dist = 0.25\n",
      "cost = [[ 1.  0.  1.  1.  2.  3.]\n",
      " [ 0.  1.  0.  0.  1.  2.]\n",
      " [ 2.  3.  2.  2.  1.  0.]\n",
      " [ 1.  2.  1.  1.  0.  1.]\n",
      " [ 2.  3.  2.  2.  1.  0.]\n",
      " [ 3.  4.  3.  3.  2.  1.]]\n",
      "acc = [[  1.   1.   2.   3.   5.   8.]\n",
      " [  1.   2.   1.   1.   2.   4.]\n",
      " [  3.   4.   3.   3.   2.   2.]\n",
      " [  4.   5.   4.   4.   2.   3.]\n",
      " [  6.   7.   6.   6.   3.   2.]\n",
      " [  9.  10.   9.   9.   5.   3.]]\n",
      "path = (array([0, 0, 1, 1, 2, 3, 4, 5]), array([0, 1, 2, 3, 4, 4, 5, 5]))\n"
     ]
    }
   ],
   "source": [
    "dst, cst, acc, ptha = dtw(a, b, dist=lambda x, y: norm(x - y, ord=1))\n",
    "print('dist = {}\\ncost = {}\\nacc = {}\\npath = {}'.format(dst, cst, acc, pth))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x11bccf550>"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgUAAAFkCAYAAACw3EhvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3Xl4VPXZ//H3nUCAICAKSEBQFkVcEdSqgErdKiCCCxDU\nBwQF0VaN+mtR69pWxT5WBVlstRUFUgXsIxD3lQoqkqCgoCCLyKKAaIIgS5Lv74+TgSxkmcnMnFk+\nr+vKdZkzc865MwJz5zvnc25zziEiIiKS4ncBIiIiEhvUFIiIiAigpkBERERKqCkQERERQE2BiIiI\nlFBTICIiIoCaAhERESmhpkBEREQANQUiIiJSQk2BiIiIALVsCsxsjJkVm9nfqnneOWaWa2a7zGyF\nmQ2tzXlFREQk/EJuCszsVGAk8Fk1zzsSmAu8DZwEPAE8bWbnh3puERERCb+QmgIzOwiYClwL/FTN\n00cDq51zv3fOfeWcmwDMBLJCObeIiIhERqgrBROAOc65d2rw3NOBt8ptex04I8Rzi4iISATUCXYH\nMxsMdAFOqeEuLYHvy237HmhsZvWcc7sPcI5DgQuBtcCuYGsUERFJYvWBI4HXnXM/BLNjUE2BmR0O\nPA6c55zbG8y+QboQmBbB44uIiCS6K4HpwewQ7EpBN6A5kGdmVrItFTjLzH4L1HPOuXL7fAccVm7b\nYUDBgVYJSqwFmDp1Kp07dw6yxMSSlZXFY4895ncZMUGvhUevgyfZX4fvv4f/+R9o0QKaNs1i3Ljk\nfS0Ckv3PBF9+CS+8wPIBA7jqmmug5L00GME2BW8BJ5Tb9iywHHj4AA0BwIfAReW2XVCyvTK7ADp3\n7kzXrl2DLDGxNGnSJOlfgwC9Fh69Dp5kfh22b4cRIyA9Hd5+G0aOTN7XorRk/jPB+vVw8cXQqhUc\ndVRga9AfvwfVFDjndgDLSm8zsx3AD8655SXfPwi0ds4F7kUwGbjRzMYC/wTOBS4HegdbrIhIsiss\nhMxMWL0a5s+Hli39rkh8t3079O0LqakwezZs2hTyocJxR8PyqwMZQJt9Dzq3FugDnAd8ihdFHOGc\nK59IEBGRatx6K7z2GsyYAccf73c14rvSXWJODmRk1OpwQacPynPO/brc99cc4Dnz8K5HEBGREI0b\nB+PHw+TJcMEFflcjMSHQJebkwAnlP90PXq2bAomszMxMv0uIGXotPHodPMn2OsydC1lZcPvtMGpU\n2ceS7bWoTNK9DoEucdIkuPDCsBzSDnxtoL/MrCuQm5ubm7wXjYiIlFi8GHr29FYHZs6EFI2yk+3b\noVMnGDIE/vd/yzyUl5dHt27dALo55/KCOWzcrhSsW7eOrVu3+l1G3GrWrBlt27b1uwwRqcb69d41\nZMceC1OnqiGQEo0awaJFcFj5xH/txGVTsG7dOjp37szOnTv9LiVupaens3z5cjUGIjFs+3YvZVan\njndReXq63xVJTGnVKuyHjMumYOvWrezcuVM3NwrR8uXLueqqq9i6dauaApEYpeih+CEum4IA3dxI\nRBJV4KLyV15R9FCiJ66bAhGRRKToofhFl6yIiMSQqqKHkqR+/DFqp1JTICISIxYvhsGD4ZJLYOxY\nv6uRmDBnDrRrB198EZXTqSkQEYkBih5KBYsXe1eb/vrXEKWL6vXHTkTEZ4oeSgWBLrFz56h2ibrQ\nUETER4oeSgWlu8Q5c6LaJaopEBHxkaKHUkagS1y1ChYsiHqXqKZARMQnih5KBaWnHvrQJeqaghiz\nbt06brjhBo455hjS09Np1qwZAwcO5JtvvvG7NBEJI0UPpQLnIDUVnnwybFMPg6WVghjzySef8NFH\nH5GZmcnhhx/O2rVrmThxIr169WLZsmXUr1/f7xJFpJYUPZQDMoPHHvO1BDUFMaZv375cdtllZbZd\nfPHFnH766cyaNYsrr7zSp8pEJBwUPZRYlhRNwc6d8OWXkT3HMceE5wLRevXq7fvvwsJCCgoKaN++\nPQcffDB5eXlqCkTimKKHEuuSoin48kvo1i2y58jNhXDMZtq1axcPPvggzz77LBs2bMA5B4CZkZ+f\nX/sTiIgvFD2UeJAUTcExx3hv2pE+Rzj89re/ZcqUKWRlZXH66afTpEkTzIxBgwZRXFwcnpOISNQp\neijxICmagvT08PwWHw2zZs1i2LBhPPLII/u27d69m59++snHqkSkNhQ9lAr+/W+47DKoW9fvSsrQ\nJS4xJjU1tcKKwLhx4ygqKvKpIhGpDUUPpYJx47zPknJy/K6kgqRYKYgnffv25fnnn6dx48Yce+yx\nfPjhh7z99ts0a9bM79JEJEiKHkoFc+Z4XeJtt0H//n5XU4Gaghgzbtw46tSpw/Tp09m1axc9evTg\nrbfe4sILL8TM/C5PRGpI0UOpIDD1sF+/mO0S1RTEmMaNG/P0009X2L569WofqhGRUCh6KBWUn3qY\nmup3RQekpkBEJIx8nmcjsWj7dq8hSE31usSGDf2uqFJqCkREwsjneTYSi0aP3n+DiowMv6upkj7l\nEhEJk0D0cMIE3+bZSCz64x/h//4PTjjB70qqFVRTYGbXm9lnZpZf8rXAzH5TxfPPNrPicl9FZtai\n9qWLiMQORQ+lUsccA7/+td9V1EiwHx98C/wBWAkYMAx42cy6OOeWV7KPA44Gtu/b4Nzm4EsVEYlN\nih5KogiqKXDOlb/Twh/NbDRwOlBZUwCwxTlXEGxxIiKxTtFDSSQh//E1sxQzGwykAx9W9VTgUzPb\naGZvmNmZoZ5TRCSWKHooiSbo9IGZHY/XBNTH+0hggHOussHEm4BRwCKgHnAd8J6Zneac+zS0kkVE\n/KfooSSiUCKJXwInAU2Ay4HnzOysAzUGzrkVwIpSmz4ysw5AFjC0uhNlZWXRpEmTMtsyMzPp1KlT\nCGWLiISPoodSRlERDBsGw4dDr15RO212djbZ2dlltuXn54d8vKCbAudcIRC4vd5iMzsNuBkYXcND\nLAS61+SJjz32GF0PMN4wLy+vhqcSEQm/0lMPFT0UwIueZGfDVVdF9bSZmZlkZmaW2ZaXl0e3bt1C\nOl44bl6UgvfRQE11wftYQUQk7ih6KBUkUJcY7H0KHjSznmZ2hJkdb2YPAWcDU0sef8jMppR6/s1m\n1s/MOpjZcWb2ONALeDKcP0Siue+++0hJSWHbtm1+lyIipSh6KBUEusTbbkuILjHYlYIWwBQgA8gH\nlgAXOOfeKXm8JdCm1PPTgEeBVsDOkuef65ybV5uiE52ZaSKiSIxR9FAqCHSJMTz1MFjB3qfg2moe\nv6bc938F/hpCXSIiMUPRQ6kgTqYeBksDkUREqqDooRzQ88/HxdTDYGkBLIZt2bKFgQMH0qRJE5o1\na8Ytt9zC7t27/S5LJKkEooczZih6KKWMGQOLFsX81MNgqSmIUc45Bg4cyJ49e3j44Yfp06cP48aN\nY1QCXMgiEi809VAqZQYtEm+2nz4+iGEdOnTgpZdeAmD06NE0atSISZMmcfvtt3O8fmURiShFDyUZ\nJUVTsHPvTr7cWtmdmMPjmGbHkF43fFcfmRk33nhjmW2/+93vmDhxIq+88oqaApEIUvRQklVSNAVf\nbv2Sbn8P7e5ONZU7MpeuGRXvvlgbHTt2LPN9hw4dSElJYe3atWE9j4jsp+ihJLOkaAqOaXYMuSNz\nI36OSNO9C0QiS9FDqWDrVmjaNGEih9VJiqYgvW562H+Lj4aVK1dyxBFH7Pv+66+/pri4mCOPPNK/\nokQSlKKHUsH27XDeeXD66d4tjJOAFsZilHOOCRMmlNk2btw4zIyLLrrIp6pEEpeih1JGYaF3Ycnq\n1VDu+q5ElhQrBfFqzZo1XHLJJfzmN79hwYIFTJs2jauuuooTTjjB79JEEkoCzbORcMnKgtdf92Zj\nJ9G/uVopiFEpKSm88MIL1KtXjzvuuINXX32Vm266iaefftrv0kQSiqKHUsG4cfDkk95XknWJWimI\nQffeey/33nsvAC+++KLP1YgkLkUPpYI5c/ZPPbz+er+riTqtFIhIUio/z0bRQ2HxYu9q0wSaehgs\n/TUQkaRTOno4Z46ih1IiNRXOPz+hph4GSx8fiEhSUfRQKnXiifCf//hdha/UFIhIUglED3NyFD0U\nKU9NgYgkDUUPRaqmawpEJCkoeihSPTUFIpLwFD0UqRk1BSKS0BQ9lAo+/RQ+/NDvKmKSrikQkYSl\n6KFUsH499OkDHTrA+++Dps+WoaZARBKSoodSwc8/e11iaiq88IIaggNQUyAiCUnRQymjqMi7sGTV\nKpg/HzIy/K4oJqkpEJGEo+ihVJCVtb9LTKKph8FSUyAiCUXRQ6lAXWKN6TpcEUkYih5KBYsW7Z96\nqC6xWlopEJGEoOihHFC3bvD88zBokN+VxAU1BSIS9xQ9lEqZwZAhflcRN4Lqpc3sejP7zMzyS74W\nmNlvqtnnHDPLNbNdZrbCzIbWruTksHHjRkaMGEHr1q2pX78+7du354YbbqCwsJAff/yR22+/nRNP\nPJFGjRrRpEkTevfuzZIlS/wuWyTqSkcPc3IUPRSpjWBXCr4F/gCsBAwYBrxsZl2cc8vLP9nMjgTm\nAhOBIcB5wNNmttE592boZSe2TZs2ceqpp1JQUMCoUaPo1KkTGzZsYObMmezcuZPVq1cze/Zsrrji\nCtq1a8f333/PU089xTnnnMOyZctoqX8VJYkoeigSPkE1Bc65nHKb/mhmo4HTgQpNATAaWO2c+33J\n91+ZWQ8gC4huU7Bpk/dVmfr14dhjqz7GsmWwa1fF7RkZYc28jhkzhs2bN7Nw4UJOPvnkfdvvu+8+\nAE488URWrFhRZp+rr76aTp068cwzz3DXXXeFrRaRWKaLykXCK+RrCswsBRgIpAOV3UT6dOCtctte\nBx4L9bwhe+opuP/+yh8/9lj44ouqj3HFFV5jUN6990LJG3ZtOed4+eWX6devX5mGoLS6devu++/i\n4mJ++ukn0tPT6dSpE3l5eWGpQyTWKXrocc5hujOfhEnQTYGZHY/XBNQHtgMDnHNfVvL0lsD35bZ9\nDzQ2s3rOud3Bnj9ko0ZBv36VP16/fvXHmDGj8pWCMNmyZQsFBQUcd9xxlT7HOcfjjz/OpEmTWLNm\nDUVFRQCYGc2aNQtbLbHip5+8NNEf/gBHH+13NRILvpi7hm0D/sRVv3mEsWMT7898TRUVF3H5jMu5\npNMlDOsyzO9yfPXmK6+Qs2oVj48cCfXqVfq8yz7/nB/27q308d+2bs3lLVpU+vjSn3/mdytXVlnL\nzOOOo1laWqWPj1+/nllbtlT6+AkHHcT4o46q8hxV/RwdN2yoct+qhLJS8CVwEtAEuBx4zszOqqIx\nCFlWVhZNmjQpsy0zM5NOnToFf7BwLPFX9/FClPzlL3/hnnvu4dprr+XPf/4zhxxyCCkpKdx8880U\nFxf7XV7Y/eMf8M9/wrx53mCzBOx7JAjr18PiQQ/xP4X/4sqfVpKy960q3wQSWdbrWcz5ag6jTxnt\ndym+WrxoEQPMuKBBAyi1knogGWlpNKgir3pQamqV+6elpHB4NX/eUqtZuTm4Tp0qj9G8mp8B9v8c\na3NyWPvKK2Ue+3bnzmr3r5RzrlZfeNcGTKrksfeBv5XbNgz4sZpjdgVcbm6uO5Dc3FxX1ePxrLi4\n2DVp0sQNGDCg0ud06dLFnXvuuRW2H3744a5Xr17VniOeXr+9e51r29a5Cy90rnlz53r0cG7XLr+r\nEr8UFDh31vE/uJ3WwO28+Arn6tVz7qqrnCsu9ru0qHvioycc9+EmfzLZ71J89e2aNa7VrFnu1ClT\n3I6CAr/LiQmBf+OBri7I9/Rw3N4jBais5fkQOLfctguo/BqEpGdm9O/fnzlz5lR6fUBqamqgedpn\nxowZbKjFklGsmjMH1q2Dv/wFXn4ZPvkEhg+Hcj++JIFA9LDHyn9Sr04RDZ5+Ep57zrtT0QMP+F1e\nVM1dMZes17O4/YzbGXVK8l5Q8fNPP3HxBx9Qp7iY2b16kd6okd8lxb2gPj4wsweBV4F1QCPgSuBs\nvDd6zOwhoJVzLnAvgsnAjWY2FvgnXoNwOdA7LNUnqAcffJA333yTs846i5EjR9K5c2c2btzIzJkz\nmT9/Pn379uWBBx5g+PDhnHnmmSxdupRp06bRoUMHv0sPu3Hj4MwzvZuSgfceMGgQHHVU2K7tlDgR\niB5mvXAtKQ2PhxYtYOBA+Ppr2LLF6xST4IK7xZsWM3jmYC7pdAljz0/eezkX7d3L4FmzWJWRwYKM\nDFq2aeN3SQkh2GsKWgBTgAwgH1gCXOCce6fk8ZbAvv8zzrm1ZtYHL21wE7AeGOGcK59IkFJatWrF\nxx9/zN1338306dMpKCigdevW9O7dm/T0dO6880527tzJ9OnTefHFF+nWrRuvvPIKY8aMSairkJcu\nhffeg3//e/+2gQO9m9TceSd06ABXX+1beRJFpaOH5152MFDqnml33JEUzQDA+oL19M3uy7HNj2Xq\npVNJseS9l3PWlCm81q4dOUVFHF9JUkuCF+x9Cq6t5vFrDrBtHtAtyLqS3uGHH86//vWvSh9/5JFH\neOSRR8pse+eddyp5dnwaPx5atYJLLy27fcwY75fDESPgiCPgrLP8qU+io9roYZI0BD/v+ZmLsy+m\nTkodZmfOJr1u8t7LedW33/J0mzZM2LKFCwcP9ruchKLZBxKTtm3zPiq+886KFxObwaRJsHYtDBjg\nJRIUVUxMmnroKSouYvDMwaz+cTXzh8+n5UHJfdfSDm3a8FVaGm0OO8zvUhJO8q49SUx75hkoKoKR\nIw/8eFoazJzpfazcpw9s3Rrd+iTyNPVwv6zXs3jt69eYccUMjm+hezkDaggiJIn/mkmsKiyEJ5/0\nrjSv4h4iNG3q3e8+P99bMdgdvVthSYRp6uF+4z4ex/iF45nQewIXdLjA73IkwakpkJgTiCH+7nfV\nP7d9e0UVE03Yph7u3ev9YYpjih5KtKkpkJhTPoZYnTPO8KKK06dXPd5C4kMgejhjRsnUw6IiWLQo\n+AO9+KJ3a/OpU8NeYzQoeih+UFMgMSUQQ7zppuD2GzgQHnzQawri9D1A2B89nDCh1NTDOXPg1FOr\nH1hW3pAhcM013hLSvHlhrzWSFD3c76dvvvG7hKSSvH/SJCZVFkOsiTFjvH//4/A9QKgiehhYOqpi\nSNgBmXk3NujRw7vopNy48Vil6OF+46ZN47hFi/hhzRq/S0kaagokZgRiiKNHVzvT5IACUcWePePq\nPUCoInq4dCm8+27wS0cBaWkwa1bcxFRKRw9zhuQkdfRwzquvktWyJZk//cSh7dr5XU7SUFMgMaO6\nGGJNKKoYf6qMHtZm6SggjmIqih56Fi9aRCbQb+1axg4b5nc5SUVNgcSEmsYQayKO3gOSXpXRwx9+\n8LqEG24IbemotNIxlVtuqd2xIkTRQ8/6tWvpu24dnbduZerAgaRWM8pYwktNgcSEYGKINaGoYuyr\nNnoYWDq67rrwnPCMM7xEws03h+d4YaTooWd7fj59588n1Tlmn3MODTX1MOrUFMShKVOmkJKSUulo\n5XgUbAyxJhRVjG0VooelFRZ6EYRwLB2V1q8fHHNM+I4XBooeegoLC8mcOZPVTZuS0749GZp66AvN\nPohTiT4NMVxKT1Xs2BGuuir855DglZ56uC96WNrOnd5Vh9dUmLGWUBQ93G9vcTENGjRgRp06nKCp\nh75RUyC+C8e1ZFUJTFUcPhzattVURb9VO/UQoHFjr3NIYIoeltUgLY0ZQ4b4XUbSS962VGJCbWOI\nNaGoYuzQ1EOPoocSq9QUxKiNGzcyYsQIWrduTf369Wnfvj033HADhYWF+56zY8cORo0aRbNmzWjS\npAlDhw7lp59+8rHq4IUjhlgTiir6T1MP91P0UGKVPj6IQZs2beLUU0+loKCAUaNG0alTJzZs2MDM\nmTPZuXMnAM45fvvb39K0aVPuv/9+vvrqKyZOnMi6det49913ff4JaiacMcSaCEQVTz/dWzF46y2o\nVy/y55U4mHr47rveTY7Gj/eWliIoED2c3GdyUkcPJTapKYhBY8aMYfPmzSxcuJCTS11wc99995V5\nXv369Xn77bf35Xjbtm3LH/7wB+bOnUvfvn2jWXJIwh1DrIlAVLFXLxgxAp5/PuLvAUmvdPRwwYJa\nTD2MpM2bvbRD8+Zw770RO42ihxLrkqYp2LR7N5v27Kn08fopKRzbsGGVx1i2Ywe7iosrbM9ISyMj\nTL9yOud4+eWX6devX5mG4EBGjhxZ5sYeo0eP5s477+SVV16Ji6YgEjHEmghEFQcN8hIJ5XotCbNA\n9DAn5wDRw1gxaJDXtdx1F3ToEJGYiqKHnqLCQmZMn86gQYMwLdXFnKRpCp7auJH7q5i2dWx6Ol+c\ndlqVx7jiiy9YVrJ8X9q9RxzBfWG6N/eWLVsoKCjguGqGv5gZHTt2LLOtYcOGZGRksHbt2rDUEkmR\njCHWhKKK0VFt9LC04mJ/LzS44w4vpjJiRNhjKooe7pf17LNMbNeOYxct4sTu3f0uR8pJmqZgVKtW\n9GvWrNLH69fgH6MZxx1X6UqBBCfSMcSaUFQxsmoUPSztoou8iMgf/xjx2g4oMFVx7VrvopOPPoKj\njqr1YRU93G/ctGmM79iRyd99x4mDB/tdjhxA0jQFGfXq1XqJv7qPF8KhefPmNG7cmM8//7zK5znn\nWLlyJWefffa+bTt27GDTpk306dMn0mXWSiCGeOedkYsh1kQgqhh4D/jwQzj6aP/qSSRBRw+XLoU3\n3vA6ND8FpiqeeSb07u01BoceGvLhAtHDVdtWsWDEgqSOHs4tmXp426pVjBoxwu9ypBLJu4YVo8yM\n/v37M2fOnGpvY/z3v/+9TERx4sSJFBUV0bt370iXWSvRiiHWhKKK4RdS9DAWlo4CSk/UevbZWh1K\n0UPP4kWLGAz0W7OGsUOH+l2OVCFpVgriyYMPPsibb77JWWedxciRI+ncuTMbN25k5syZzJ8/f9/z\n9uzZw7nnnsvAgQP58ssvmTRpEj179ozpiwyjHUOsCUUVwyek6GFg6eiuu/xdOiqtfXvIy4PWrUM+\nROno4YUdq7ugInHtm3q4Y4c39bCO3nZimf7vxKBWrVrx8ccfc/fddzN9+nQKCgpo3bo1vXv3Jr3k\nX1kz48knn2TatGnce++97N27lyuvvJInnnjC5+qr5kcMsSYUVay9kKOHzzzjXWQYC0tHpR1+eMi7\nKnrocc5x+QcfkNqggTf1sHFjv0uSaqgpiFGHH344//rXvw742NChQxlasgTXo0cPJk2aFM3SasWv\nGGJNKKpYOyFFDwNLR4MHe/cISACKHu5nZjxy4ok0LS7W1MM4oaZAosbvGGJNKKoYmqCih6XF6tJR\niBQ9rOisE0/0uwQJQlB/Ys3sDjNbaGYFZva9mf3HzKq8XtvMzjaz4nJfRWYWI58oS7TE0rVkVRkz\nxrsIfvhwmDfP72piX9DRw9Ly8qBHj9hcOgqSooeSCIJdKegJjAcWlez7EPCGmXV2zv1SxX4OOBrY\nvm+Dc5uDPLfEsViJIdZE+ahimOLqCanWUw//9CfYvTvsdUVUURHs2OGNdw5sUvRQEkRQKwXOud7O\nueedc8udc0uBYUBboCZt/hbn3ObAVwi1ShyLpRhiTZSOKvbuDT/84HdFsSdsUw/jLepx3XXeD16q\nmVH0UBJFbT/wOhhvFWBbNc8z4FMz22hmb5jZmbU8r8SRWIwh1kTpuHr//vH3C20kxfzUw0i67jpY\nuBCuvRac2xc9nNB7QlJHD3HO7wokDEJuCszMgMeBD5xzy6p46iZgFHAZcCnwLfCemXUJ9dwSX+L5\nWrJAVPGTT7yoov7dKxs9zMmJ0amHkXTGGTBlCkydylc3X6XoIbA9P58BTz3F8vfe87sUqaXapA8m\nAscCVU60cM6tAFaU2vSRmXUAsgDd2ioJxHIMsSYUVSwrLqYeRtqgQWz89L90engCj97YjZuSOHpY\nWFhI5syZzGvblsImTfwuR2oppKbAzJ4EegM9nXObQjjEQqppJgCysrJoUu4PWWZmJp06dQrhlOKH\neIgh1oSiip6Qo4cJZn3Ben7V7CUmndmMm/+xFBv4QdJO1Lr12Wd5rV07coqKOKGace8SftnZ2WRn\nZ5fZlp+fH/Lxgm4KShqCS4CznXPrQjxvF7yPFar02GOP0bVr1wrbAzMBli9fHuLpk1s0X7d4iSHW\nRGCqYgQm68aFWkUPE8j23du96GFqXU6b8xF2+bCkjakEph5O+u47LtTUQ19kZmaSmZlZZlteXh7d\nQlyaDaopMLOJQCbQD9hhZoeVPJTvnNtV8pwHgdbOuaEl398MrAG+AOoD1wG9gPNDqhho1qwZ6enp\nXJWsv66FQXp6Os2qGCUdDvEUQ6yJZI4q1jp6GPD5596y0ZgxcNBBYasvWgqLC8mclbk/enhIW2+q\n4qBBsGuX3+VFVemph9dr6mHCCHal4Hq8tMF75bZfAzxX8t8ZQOn7WaYBjwKtgJ3AEuBc51zIt4Vp\n27Yty5cvZ6tG2oWsWbNmtG3bNqLniLcYYk0EoophmqwbFwLRw2OPrWX0ELzPH3Jy4N57w1ZfNN36\n+q289vVr5AzJ2R89bNrUG/ucRPZNPVy7lrHDhvldjoRRUE2Bc67afw6cc9eU+/6vwF+DrKtabdu2\njfibmoQuXmOINVF6qmL//ok9VbF09HD27FpGD2NxGmIQNPVwvydWrOCYoiJv6mFqqt/lSBhp9oFE\nRDzHEGsiGaYqhjz1sDKBpaPrrgtLfdGkqYdl/WPQIAq2b6dho0Z+lyJhpmkdEhHxHkOsiUBUcdo0\nuP9+v6sJv0D0cMaMMEQP43jpSFMPK6qbmsqhBx/sdxkSAVopkLBLlBhiTSRqVDHs0cM4XTrS1ENJ\nNmoKJOwSKYZYE4kWVYxI9HD8+LhbOtoXPazN1MOPPoJDDoGjqxwmKxIz1PZKWAWuJRs9Oi6vJQtJ\nIKrYvbsXVVy50u+KQhe26GFpy5bBu+/CTTeF6YCRVzp6mDMkJ7Sph8XFcOON0KcPKCklcUJNgYRV\nIsYQayIJ70OXAAAgAElEQVQtzYurx/NUxbBGD0vr3NmLaMTR0lEgelirqYcpKd4FGfn5XrcYhxO1\ntqxahdu71+8yJIrUFEjYxPG1ZGERz1MVwxo9LM8Mzj03bpaOwjr1MI4naq1fu5Yun33GY08/7Xcp\nEkVqCiRs4vRasrAq/R5QMlk35iX91MNSIhI9jMOYys8//cTFH3xAqnNk9u3rdzkSRbrQUMImGWKI\nNVF+qmKs37xPUw89EY0eDhzoXY16110xH1Mp2ruXwbNmsSojg/kZGWS0aVP9TpIw1BRIWCRTDLEm\nSkcVO3SI3fcATT30BKKHnZt3jlz08I479sdUTj4Zjjsu/OcIg6wpUzT1MImpKZCwSLYYYk3EelRR\nUw89paOHczLnhBY9rAkzr/vq0cO7mjMGBaYeTtbUw6Slawqk1pIxhlgTsRxVjEj0MA6FJXoYjLQ0\nGD48Ju+JXXrq4Sg1BElLTYHUWrLGEGsiFqOKEYseljZ/PvzySwQOHF5hiR4miLotWnDVt98yduhQ\nv0sRH6kpkFpJ9hhiTcRSVDGi0cOAbdvg/PO9CxZiWFijhwngwm7dmDJsGKl19KlyMlNTILWiGGLN\nxEJUMWrRw8DS0TXXVP9cn2jqociBqSmQWlEMseYCUcWpU+GBB6J//rBOPaxMHCwdaeqhSOXUFEjI\nAjHEOLqlve8GDoQHH4T77vOag2gJRA8nTIhw9DDGl46iEj0Mxa5dMHQozJvndyWS5GLkb4TEI8UQ\nQzNmjHcB+ogR0XkPiGr0MIaXjqIWPQxFSgp8+60XU1mxwu9qJImpKZCQKIYYumhGFaMaPYzhpaOo\nRw+DVTqmEuGpim++9hqrFiyI2PElvqkpkJAohlg70YgqRiV6WFoMLx3FRfSwdEwlQlMVFy9axADn\n+MuSJWE/tiQGNQUStDi4liwuRPI9ICrRw/K6d4c//Snmlo7iKnpYOqYyfHhYYyrr166l77p1dN66\nlfFXXhm240piUVMgQYvxa8niSuA9YOHC8EUVfZt6OHSo90YWQ+IyenjGGTBlCkyfHrapitt/+om+\n8+eT6hyze/WiYaNGYTmuJB41BRK0GL6WLC6FO6oYlehhHIjr6OGgQfCXv3izErZtq9WhCvfuZfBL\nL7G6aVNyOnQg4/DDw1SkJCLdukqCommIkVF6sm5tpipq6qEnZqOHwbjjDm/56JBDanWYrOee4/Uj\nj/SmHnbpEqbiJFGpKZCgxPC1ZHGv9GTdI46Anj2D219TDz0xHT0MhlmtL9p54Y03eLJDB009lBqL\nw/ZZ/KIYYmQFJut27+7NSAgmqqiph56Yjx5GWf9zzyW7sFBTD6XG1BRIjSmGGHmhRBWjHj2MYXER\nPYyieqmpDD7vPL/LkDiSxP98SDAUQ4yeYKKKvkQPA/bu9WeyUyXiKnooEqOCagrM7A4zW2hmBWb2\nvZn9x8yOrsF+55hZrpntMrMVZqaB3XFGMcToqklU0bfoYcDjj8Npp3nLRz6Ly+hhbWze7HcFkqCC\nXSnoCYwHfgWcB9QF3jCzBpXtYGZHAnOBt4GTgCeAp83s/BDqFZ8ohhh91UUVfY0eBpaOjjsOUlOj\nfPKy4jp6GIqVK72ISjQnaknSCCp94JzrXfp7MxsGbAa6AR9UsttoYLVz7vcl339lZj2ALODNoKoV\nXyiG6J/Kooq+Rw9jZOkoIaKHwerYEa64wouptG0LZ53lbXfOu1pVpBZqG0k8GHBAVXfXOB14q9y2\n14HHanluiRLFEP1VPqqYnx8D0cMYWDpKmOhhsAIxlW++8S46+egjxi9cyIbvv+ehm27C6ihpLqEL\n+U+PmRnwOPCBc25ZFU9tCXxfbtv3QGMzq+ecC//UDwmbQAzxzjsVQ/RL4D1g7Vovqrh7t8/RwyVL\nfF862ltUyG9mZ/FVYQOeu/TlaqOH8/Pz2VtcXOnjR6Wn07pevUof/3HvXj77+ecqz9G9SRPqVhH9\nWLFzJxuruGr04Dp16FLN7YfL/BzPPgs33sjK++/nlmuuIWvXLjUEUmu1+RM0ETgW6B6mWirIysqi\nSZMmZbZlZmaSmZkZqVNKOYohxoZAVLFHDzjoIJ+jh08+6fvS0SnvPsuS5ldBc5iwDS5vW/XzL1m6\nlB8KCyt9fMJRR3FD69aVPv7pzz/z688+q/IcW848k2ZpaZU+/vj69UzauLHSx885+GDereaOgxV+\njltvBeDSr79m7LBhVe4riSk7O5vs7Owy2/Lz80M+nrkQIkVm9iRwMdDTObeumue+D+Q6524ttW0Y\n8Jhzrmkl+3QFcnNzc+natWvQ9Ul4FBZ6n2P36uX9UiL+273bawZ8W7X54Qdo08a7yOGuu3wp4bL5\n03hpb2suT9vMQ1360iA1tcrf8gFW//ILxVX8W9c8LY0mVfyWvbOoqMrf8gHaNWhAahWf6W/es4eC\nKhqTUH8O27WL9gcfjCXzDSqkjLy8PLp5H+11c87lBbNv0CsFJQ3BJcDZ1TUEJT4ELiq37YKS7RLD\nYuRaMimlmveMyPv8c+9GCj4tHd2z5FVe2tOSU4pXM+PMmk9kbN+g0oBUjaSnptKxljeBaJGWRosq\nVhJq4oA/R1RvTiGJLtj7FEwErgSGADvM7LCSr/qlnvOgmU0ptdtkoL2ZjTWzTmZ2A3A58Lcw1C8R\nFAPXkkmsOftsr1Ns3jzqp85es4g/bYGM3Wv4sJdudSISCcGuFFyPlzZ4r9z2a4DnSv47A2gTeMA5\nt9bM+uClDW4C1gMjnHPlEwkSQxRDlEr5cF+Cb/O/ZeiyT0lPrceScwZRJ8XfeyOIJKpg71NQ7cqC\nc+6aA2ybh3cvA4kTiiFKrNi+ezv9/t2P5nuLefXKHJrVr/oKfREJnfIrUkEghnjXXYohir9KTz1c\nMGIBxzc93O+SRBKamgKpIBBDvO46vyuRZBeYepgzJEdTD0WiQE2BlKFpiBIrAlMPJ/eZrKmHIlGi\nYKuUoRiixIKkm3ooEiPUFEgZiiFKGYWF3uzmpUujdsqFG/OSa+qhSAxRUyD7BGKIN93kdyUSM+bM\n8S4y2bMnKqf7ZOtazlyyglZHXJI8Uw9FYoj+xsk+iiFKBVFcOtq08yfOXvQBpNRlxkVjk2fqoUgM\n0YWGAiiGKAcQxTtY7S7aS5d5s/ilTgazjm7FSYcoeijiBzUFAiiGKAcQxaWj096dwuZ67fhz8yIu\nbVv1pEARiRx9fCCKIUpFgaWj0aMjvnR02fxpLKnTkSH1t3DX8RdE9FwiUjU1BaIYolQUWDqK8DTE\nfVMPi1Yz7YzBET2XiFRPTYEohihlOQdPPRXxpaPFmxbz8OIXydi9hgWaeigSE3RNQZLTNESpwAze\nfReKiyN2ivUF6+mb3ZeTGrXinV8/SV1NPRSJCWoKkpxiiHJAbdpU/5wQbd+9nYuzL6ZOSh3mZM6h\nUVrDiJ1LRIKjjw+SWOBashtuUAxRoqP01MOcITm0PKil3yWJSClaKUhiiiFKtGnqoUhsU1OQpBRD\nlGjT1EOR2KePD5KUYogSTX/7/BVueeP3mnooEuO0UpCkFEOUaMles4jbvjc6nfwAY8+/3e9yRKQK\nagqSkGKIUsFnn8FBB0GHDmE97Cdb13LVynWkF+/gv+eN1tRDkRinpiAJKYYoFdxyC+zdCx98ELZD\nbiyZemgpDVhwWi+a128UtmOLSGSoKUgymoYoFURg6WhX0V5O1tRDkbijpiDJKIYoFURg6ehXmnoo\nEpf0AV8SUQxRKojANMRB86dq6qFInFJTkEQUQ5QKwjwNce6Kucz44PecU7hMUw9F4pCagiSiGKKU\nEealo8WbFjN45mD6H3k6b597fRgKFJFo0zUFSUIxRKkgjEtHgamHnZt3ZuqlUxU9FIlT+pubJBRD\nlArat4e776710lH5qYfpddPDVKCIRFvQTYGZ9TSz2Wa2wcyKzaxfNc8/u+R5pb+KzEyXukWJpiHK\nAZ10EjzwQK0OoamHIokllJWChsCnwA2Aq+E+DjgKaFnyleGc2xzCuSUEiiFKpASmHs64YoamHook\ngKCvKXDOvQa8BmBmFsSuW5xzBcGeT2pHMUSJlIHzpzLj60+Y3HuCph6KJIhoXVNgwKdmttHM3jCz\nM6N03qSnGKJEwj1LXmXGngxOOfZaTT0USSDRaAo2AaOAy4BLgW+B98xMtzmLAsUQJdyy1yziT1sg\nY/daFvQa5nc5IhJGEY8kOudWACtKbfrIzDoAWcDQqvbNysqiSZMmZbZlZmaSmZkZ9joTkWKIEm6l\npx5+ds5A6qak+l2SSFLLzs4mOzu7zLb8/PyQj2fO1fRawQPsbFYM9HfOzQ5yv0eA7s657pU83hXI\nzc3NpWvXriHXl+xGjoScHFi7VqkDKbFzJ6SHFhnc9Es+Hd6fw56UBuSe8isNORKJUXl5eXTzloe7\nOefygtnXr/sUdMH7WEEiRDFEqWDpUmjZEhYvDnrX3UWFnPT+LH6pcwgvHtNBDYFIggr64wMzawh0\nxLt4EKC9mZ0EbHPOfWtmDwGtnHNDS55/M7AG+AKoD1wH9ALOD0P9UgnFEKWC8eOhcWM4Pvjo4KVv\n/S9b6p2qqYciCS6UlYJTgMVALt79Bx4F8oD7Sx5vCbQp9fy0kucsAd4DTgDOdc69F1LFUi3FEKWC\nWkxDHPfxOF756A7uabhOUw9FElwo9yl4nyqaCefcNeW+/yvw1+BLk1AphigVPPMMFBcHPQ1x7oq5\nZL2exe1n3M79p11T/Q4iEtc0+yABjRsH3bsrhiglAktHgwdD8+Y13i0w9fCSTpcw9vyxESxQRGKF\npiQmGMUQpYIQlo4CUw+PbX6sph6KJBH9TU8wmoYoFQS5dPTznp/3TT2cnTlbUw9FkoiaggSiGKJU\nsGOHd2+CGq4S7CkqZODMIZp6KJKk9PFBAlEMUSpo2BA+/hhqeJOyU999liX1z+KVy2/U1EORJKSm\nIEEohihVqsFA08vmT2NJnY4MyfiOi47S1EORZKSPDxKEYohSG/cseZWX9rTklKLVTDtjsN/liIhP\n1BQkCMUQJVRlpx5WOaNMRBKcPj5IAIohSqg09VBEStNKQQJQDFFC8d0v+Zy96AMMx4LTetG8fiO/\nSxIRn6kpiHOKIUooioqLGDJ7NLsLd2nqoYjso6YgzimGKBU88ABMmVLlU7Jez2LeVy+S06mNph6K\nyD5qCuKYYohSwbZt8PDD8O23lT5l3MfjGL9wPBN6T+A3ih6KSClqCuKYYohSQWDpqJJpiKWnHo46\nZVSUixORWKemII4phihlVLN0pKmHIlIdRRLjlGKIUkEVS0eaeigiNaF/GeKUYohSwfjxB1w62vRL\nPr3+7yZNPRSRammlIA4FYoh33aUYopRYuhTefbfC0tHuokK6vD+TLRnD+PD8+zT1UESqpKYgDimG\nKBX8618HXDo67d1n2VyvHX9uXsSvMk70qTgRiRf6+CDOKIYoB/Tww/Dmm2WWjvZNPay/hbuOv8DH\n4kQkXqgpiDOKIcoBpaXBscfu+1ZTD0UkFGoK4oxiiFIdTT0UkVDpmoI4ohiiVGfptm809VBEQqaV\ngjiiGKJU5ec9P3P1jP40+X6Oph6KSEjUFMQJTUOUqhQVFzF45mBWb1vFvAtv09RDEQmJPj6IE4oh\nSlWyXs/ita9fI2dIDse3ON7vckQkTmmlIA4ohigVbNvmRVGKispMPbywo6Yeikjo1BTEAcUQpYJn\nnoHLL+eNhdmaeigiYRN0U2BmPc1stpltMLNiM+tXg33OMbNcM9tlZivMTDmpICiGKGWULB390P9C\nLn33ek09FJGwCWWloCHwKXAD4Kp7spkdCcwF3gZOAp4Anjaz80M4d9IJxBC1SiD7zJnD+p07Oan3\nSXRq0UVTD0UkbIK+0NA59xrwGoCZWQ12GQ2sds79vuT7r8ysB5AFvBns+ZONYohS3o+TJnDuo2P5\nrmVjXjl1lKYeikjYROPXi9OBt8ptex04IwrnjmuKIUp5uz9dzNW9zmVNy5bMOKYjJzZV9FBEwica\nkcSWwPfltn0PNDazes653VGoIS6NH68YIkDu5x9z63vz2JHWYN+249auJOulKQD8v9tOZG/div3t\n0hZDKEpJ47AfdtFoR2GFx4e+8R96fp7LFx0a848r2ld4vCCtNasP8T7l6vDtz9gBPiwbN+HPpO/e\nxbQ+bfnkhEMqPP5t4zP5If1oGv5SSMaWXRUeD/bnqLO3Ll+ddhoPHLqXAW27VCxIRKQWYvo+BcNu\nGMWRLTLKbMvMzCQzM9OniqJn7lx44AH4wx+SO4b43bbvuDr3c37MaMeJ61fu255uhew4rCkArRq3\npqhuxdv5rk8ppsgKOWTvThr/coDes2EaOw5rijVryuGNK/7G/WPqIfxoXjNx6C/bD9gU/NK8CW5v\nAxo3bcnhjSv+j9pVrwHFVkjD4t0c+sv2Co8H/XPk7+ZqK2DMuVkVixGRpJOdnU12dnaZbfn5+SEf\nz5yr9lrBync2Kwb6O+dmV/Gc94Fc59ytpbYNAx5zzjWtZJ+uQG6d8Q+x7OprOKrJYSHXGI8WL4ae\nPeGCC2DmTEhJ0mvI9hQV0ubNKfxkLZlZby8Xn9Pf75JERGJeXl4e3by4WjfnXF4w+0bj7eZD4Nxy\n2y4o2V6lwpQ0TvnwVQr2/BKRwmLR+vXQty907uxdT5CsDQHAeW9PYHO9I7mndV01BCIiURDKfQoa\nmtlJZhb4QLN9yfdtSh5/yMymlNplcslzxppZJzO7Abgc+Ft157qr1cEUpLXipHenUVxcHGypcWf7\ndrj4YqhTx7thUXoSX1Q+7uNx/Pej/8dNDTZy1/EX+F2OiEhSCOX30FOAxUAu3n0KHgXygPtLHm8J\ntAk82Tm3FugDnId3f4MsYIRzrnwioYJL23bhlia/sLZeR34977kQSo0fhYXebYxXrYKcHGjZ0u+K\n/DN3xVyyXs/ittNv4onTr/a7HBGRpBHKfQrep4pmwjl3zQG2zQNCuh/fY90uYdn7U3jDHclNi2Yx\n7pTLQjlMzLv1VnjtNa8hOD6J59ks3rSYwTMH069TP8aep7v0iYhEU1x8Yv1qz6s59ZdcJr05innf\nzPO7nLAbN86LH06YABcm8Tyb9QXr6Zvdl87NOzN1wFRSUypeiS8iIpETF01BSkoKH1zwO3q2OpEB\nLwxg5Q8rq98pTsydC1lZcNttMCqJ59n8vO073rq8GwftMWYPnk3DtIZ+lyQiknTioikASEtNY9bA\nWbRo2ILe03vzw84f/C6p1hYvhsGD4ZJL4JFH/K7GP4V7drPs3JO4dN5m5nSfQEajjOp3EhGRsIub\npgCgaYOm5AzJIX9XPv1f6M/uwvi9GaKih/sNmvAwOSf9hhWT/szRPS/xuxwRkaQVd29F7Zu25+XB\nL/PJhk8YMXsEtbn5kl8UPdzvpvH389LJZ7PjtI6ccs1dfpcjIpLU4q4pADijzRk8N+A5pi2dxv3v\n31/9DjFE0cP9/jb1CSYc24OBi9/jf2+42+9yRESSXkzPPqjKwOMGsmrbKu5c9l++bziLSafGR1RR\n0UPPS2/O4J5Dj6bnslym3niH3+WIiAhxulIQMKbHGI5qdymTtzdi/Jfv+11OtRQ99HyyZQ3X/2wc\nuelbZgz5H+qm1fO7JBERIc6bAjMjr9dVHLxnAzevL+DNjcv9LqlSih56vvsln7Nz55PfAKafeQrN\nD03iz09ERGJMXDcFAAfVrU9e977ULfqZPkuXsDL/e79LqkDRQ09RcREDcv7ArpSDmN65Ayce09Xv\nkkREpJS4bwoA2jVqzusndaMwpX7MTVVU9HC/rNez+GTZ0/znyAZcdsTJfpcjIiLlJMxb1Dktj2by\nkc1iaqqioof7jft4HOMXjmdC7wlccnQSX1AhIhLDEqYpABh5VHdubvILa9Pacd28x32tRdHD/fZN\nPTzjNkadksQXVIiIxLiEagoAHu92CbemLuGf79/G1CVTfasjED2cMSO5o4eL1y3U1EMRkTiRcE0B\nwP/2vIXhXYYz/OXhvkxVVPTQs2n5Jxzc7UyGbW6lqYciInEgIZsCM2NS30n0aNsj6lMVFT30bNq8\ngZWDBlC3CO7+7QxNPRQRiQMJ2RSAP1MVFT307N69iytm/JtRt/2Z7S+9yGEdT/K7JBERqYGEbQog\nulMVFT3cb8jEsXx0TBduqLeDzudc6nc5IiJSQwn/1lV6quLQ2SMjElVU9HC/wNTDm7/6kN8NvtHv\nckREJAgJ3xSAN1Xxmf7P8ULd7vx63nNhPbaih/s9um/q4fs8esMf/S5HRESClBRNAcCVxw/kgsb1\neJ8juf6Tl8J2XEUPPdlrFnFni6PpsTyP53+rqYciIvEoaZoCgFd7Xs1Re1byVJimKip66Plk61qu\nWrmOOsVbmTHsWtLqpvldkoiIhCCpmoKUlBTyel3NwbvXc/P6At7YuCzkYyl66Pl5z8+MeGkQDQo+\nY/5p59DioKZ+lyQiIiFKqqYASqYq9vCmKvZdupQVIUxVVPTQU1RcxOCZg1n7w3I+7Hk5XQ5p43dJ\nIiJSC0nXFEDZqYqnBjlVUdHD/bJez+K1r19jxhUzOOGwE/wuR0REailp39ICUxW3797O8Jzf4Zyr\ndp9A9DA1FWbPTu7oYWDq4ZO9n+TCjkl8QYWISAJJ2qYAvKmK2R0OY9aSZ7j//furfG756GFGRpSK\njEH/nfUYY+bewm1n3Mb1p1zvdzkiIhImITUFZnajma0xs1/M7CMzO7WK555tZsXlvorMrEXoZYfP\noOMH8pdf/4X737+/yqmKpaOHJyTxSvmXb2Rz8pBbeXbZ0Zp6KCKSYOoEu4OZDQIeBUYCC4Es4HUz\nO9o5t7WS3RxwNLB93wbnNgdfbmTc0eMOvt72NcNfHk7bJm0564izyjweiB5Onpzc0cPczz/m1Wdn\nMiAjnT5Pz9PUQxGRBBPKSkEW8JRz7jnn3JfA9cBOYHg1+21xzm0OfIVw3ogxMyb3nXzAqYqKHnq+\n27yBq3M/Z9KAQdR9/b80bBoTCz0iIhJGQTUFZlYX6Aa8HdjmvCv03gLOqGpX4FMz22hmb5jZmaEU\nG0kHmqqo6KFnz+5dXD7j36xv1pLJzdM4ulNXv0sSEZEICHaloBmQCpQP938PVHbX/03AKOAy4FLg\nW+A9M+sS5LkjrvRUxTNn3U7vS39R9BDILJl6+Jfta7n4nP5+lyMiIhES9DUFwXLOrQBWlNr0kZl1\nwPsYYmikzx+s9k3bM77P/zF403ZS75rKootGkJ6evB3B7yb+iZdOPptbv/gvv7vxbr/LERGRCAq2\nKdgKFAGHldt+GPBdEMdZCHSv7klZWVk0adKkzLbMzEwyMzODOFVwCgvh+d+fSd3DXmbv1Udx5cop\nvNf6moidL5bds+RVnux8Jv2XL+BRNQQiIjEnOzub7OzsMtvy8/NDPp7V5KY9ZXYw+wj42Dl3c8n3\nBqwDxjnn/lrDY7wBFDjnLq/k8a5Abm5uLl27Rvfz65tugokTvXsRPFp/Cm+6I7i+4TYmnXppVOvw\n23/X53HW8g1kFH3P2vOHkZYa8UUlEREJg7y8PLp16wbQzTmXF8y+oayL/w24zsz+x8yOASYD6cCz\nAGb2kJlNCTzZzG42s35m1sHMjjOzx4FewJMhnDuiyk89fK3n1Ry152smh2mqYrxYX7CewS9ezDHf\nTeHTsweqIRARSRJBNwXOuReB24EHgMXAicCFzrktJU9pCZSejJOGd1+DJcB7wAnAuc6590KuOgIO\nFD30pipexcF7aj9VMV78vOdnLs6+mFRL5Z3+42nRoLHfJYmISJSE9Cugc24iMLGSx64p9/1fgRp9\nrOCXqqKHB9WtT173vhyz4C36Ll3KmsbNaX1Qc38KjbDA1MNV21Yxf/h8Mhol8b2cRUSSUPJeVl+i\nJlMPA1MV66//N4NnXMruwt3RLzQKNPVQRCS5JXVTEMzUw3NaHs3rF/yeTzZ8wojZI2o0VTGevPOH\ngaycpqmHIiLJLGmbglCmHp7R5gym9J/CtKXTqp2qGE8+mnwPZ/91Br/fc4qmHoqIJLGkvaw8MPUw\nJye4qYeDjh/Eqh9Xcdc7d9HxkI5cdeJVkSsyCl56cwb31W3L/eefQL/pC/wuR0REfJSUTUFtpx4G\npiqOmD3igFMV40Xu5x9zY4FxqBXTffqbpNap63dJIiLio6T7+CAcUw8DUxW7t+leYapivNi0eQNX\n5X5OSnEx07ufSotDy9+kUkREkk1SNQXhnHoYmKp4SOMOdP3vS6zILz8jKnbt3r2LK2b8mw3NWjK5\neT1O7HSy3yWJiEgMSJqmoCbRw2A1bdCUf/Sfyo6Gx3Dqh69SsOeX2h80Cnq/MKHU1MNL/C5HRERi\nRFI0BcFED4N1TsujmXxkMwrSWnHSu9MoLi4O38Ej4LL503inbTeG7VjJ7wbf6Hc5IiISQxK+KQgl\nehiskUd15+YmO1lbryO/nvdc+E8QJrO/mstL32+kW9Eqnu6n6KGIiJSV8E1BIHo4Y0Zw0cNgPd6t\nP+fbN7zPkYz+5KXInShEizctZsiswfTb+xELzhnqdzkiIhKDEropKD/1MNL2T1U8KKamKq4vWE/f\n7L50bt6Z6QOe09RDERE5oIRtCsIRPQzW/qmKG7h1xVJWbF0RnRNXofTUw9mDZ9MwraHfJYmISIxK\nyKYgED3s1w/Gjo3uuQ+qW5/F3S+m3YZn6JPdhx92/hDdAkop2ruHK/89kFXbVpEzJEdTD0VEpEoJ\n1xSUjx6mpka/hiMbNeO1wbPI35VP/xf6+zZV8b+Xn8boP73KjMtf1NRDERGpVkI1BeWjhw19XClv\n37Q9Lw9+2bepiv+6dyTnzP6M9MszufCo30T13CIiEp8S5oqz0tHD+fMjEz0MVmCq4uBZg+l4SEfu\nO+e+qJz3b1Of4P+dNYgNt+/mjw9Nico5RUQk/iVMUxDq1MNIi/ZUxZfenME9hx5Nj+V5/P4vf4/o\nuVnbGV4AAAkWSURBVEREJLEkRFNQ26mHkVZ6quIhjdrQu93ZETlPYOrhkfnfMmPw1aSl1YvIeURE\nJDHF/TUFfkQPgxWYqnj00SPo+/VPvLlxedjP8d3mDVy9qGTq4Zmn0KJZy7CfQ0REEltcNwV+Rg+D\nlZaaxuyL7qdu8U76LF3CyjBOVdxTVMhpH+awrkVLJjdP48Rjuobt2CIikjzitimIhehhsNo1as7r\nJ3WjMKU+p4RxquJp7z7Lt407cEebulx8Tv+wHFNERJJPXDYFsRQ9DFa4pyqO+3gcny1/mqvqb+au\nEy4IU5UiIpKM4q4piMbUw0gbeVR3bgrDVMW5K+aS9XoWt3XqwfNnZIaxQhERSUZx1xREa+phpD1R\ny6mKizctZvDMwfTr1I+x58X4BRUiIhIX4qopiPbUw0h7refVHL1nBf/49DnmfTOvxvuVnno4dcBU\nUlPi4IIKERGJeXHTFMRD9DBYKSkpLD13GGelFTDghQGs/GFltfv8/MMm/vjnX2vqoYiIhF1cNAXx\nFD0MVlpqGrMGzqJFwxb0nt67wlTF7Ozsff9dtGc3y87rwqMTvubVS2Yk3dTD0q9FMtPr4NHrsJ9e\nC49eh9oLqSkwsxvNbI2Z/WJmH5nZqdU8/xwzyzWzXWa2wsyG1vRc8Rg9DFbTBk3JGZJzwKmKpf+Q\n3zD2DppuTmHNk3/iuPa/8qNUX+kvvEevg0evw356LTx6HWov6KbAzAYBjwL3AicDnwGvm1mzSp5/\nJDAXeBs4CXgCeNrMzq/uXDt2xG/0MFjVTVW8afz9/L1nP/5x52hOueYun6oUEZFEFspKQRbwlHPu\nOefcl8D1wE5geCXPHw2sds793jn3lXNuAjCz5DhVuvPO+I4eBiswVXHa0mnc//79+7b/beoTTDi2\nBwMXv8cjN97jY4UiIpLIghqIZGZ1gW7Ag4FtzjlnZm8BZ1Sy2+nAW+W2vQ48Vt35FiyAV16J7+hh\nsPZNVZz3MDsPOo5NWzbum3r4/I13+F2eiIgksGCnJDYDUoHyN+7/HuhUyT4tK3l+YzOr55zbfYB9\n6gPccsFLNP9mEXkHmgDcrRuYVV7p6tWwbVvljx98MHTsWPnjALm5UG4Zv4x27eDQQyt//McfvaWO\nqhzg57gw/UKebfIjf120mToFv9D24/nc3fMsPv/8i6qPleDy8/PJy8vzuwzf6XXw6HXYT6+FR6+D\nZ/nyfUP36ge7r5X/7LrKJ5tlABuAM5xzH5faPhY4yzlXYbXAzL4C/umcG1tq20V41xmkH6gpMLMh\nwLRgfhAREREp40rn3PRgdgh2pWArUAQcVm77YcB3lezzXSXPL6hklQC8jxeuBNYCu4KsUUREJJnV\nB47Eey8NSlBNgXNur5nlAucCswHMzEq+H1fJbh8CF5XbdkHJ9srO8wMQVHcjIiIi+ywIZadQ0gd/\nA64zs/8xs2OAyUA68CyAmT1kZlNKPX8y0N7MxppZJzO7Abi85DgiIiISI4L9+ADn3Isl9yR4AO9j\ngE+BC51zW0qe0hJoU+r5a82sD17a4CZgPTDCOVc+kSAiIiI+CupCQxEREUlccTH7QERERCJPTYGI\niIgAMdgUBDtsKRGZWU8zm21mG8ys2Mz6+V2TH8zsDjNbaGYFZva9mf3HzI72uy4/mNn1ZvaZmeWX\nfC0ws9/4XZffzGxMyd+RpLpw2czuLfm5S38t87suv5hZKzN73sy2mtnOkr8rXf2uK5pK3jfL/5ko\nNrPxwRwnppqCYIctJbCGeBdw3gAk80UfPYHxwK+A84C6wBtm1sDXqvzxLfAHoCvercbfAV42s86+\nVuWjkl8YRuL9O5GMPse72LtlyVcPf8vxh5kdDMwHdgMXAp2B24Af/azLB6ew/89CS+B8vPePF4M5\nSExdaGhmHwEfO+duLvne8P4xHOece8TX4nxiZsVAf+fcbL9r8VtJc7gZ7+6ZH/hdj9/M7Afgdufc\nv/yuJdrM7CAgF2/g2t3AYufcrf5WFT1mdi9wiXMuqX4bPhAzexjvLrtn+11LLDGzx4HezrmgVldj\nZqWg1LCltwPbnNexVDVsSZLLwXidbxVDLRKfmaWY2WC8+4NUehOwBDcBmOOce8fvQnx0VMlHjKvM\nbKqZtal+l4R0MbDIzF4s+Zgxz8yu9bsoP5W8n14JPBPsvjHTFFD1sKWW0S9HYknJqtHjwAfOuaT8\n7NTMjjez7XjLpBOBASXjy5NKSUPUBUjmsaEfAcPwlsuvB9oB88ysoZ9F+aQ93orRV3h3y50EjDOz\nq32tyl8DgCbAlOqeWF7QNy8S8clE4Figu9+F+OhL4CS8v+yXA8+Z2VnJ1BiY2eF4zeF5zrm9ftfj\nF+dc6Xvaf25mC4FvgIFAsn2clAIsdM7dXfL9Z2Z2PF6z9Lx/ZflqOPCqc66ymUSViqWVglCGLUkS\nMLMngd7AOc65TX7X4xfnXKFzbrVzbrFz7i68C+xu9ruuKOsGNAfyzGyvme0FzgZuNrM9JStKScc5\nlw+sAKqZB5+QNgHLy21bDrT1oRbfmVlbvAuz/xHK/jHTFJR0/YFhS0CZYUshDXaQ+FfSEFwC9HLO\nrfO7nhiTAtTzu4goews4Ae/jg5NKvhYBU4GTXCxdOR1FJRdedsR7g0w284FO5bZ1wls5SUbD8T52\nfyWUnWPt44O/Ac+WTGJcCGRRathSsij5XLAjEPitp72ZnQRsc859619l0WVmE4FMoB+ww8wCq0j5\nzrmkGqltZg8CrwLrgEZ4FxGdjfcZatJwzu0AylxTYmY7gB+cc+V/W0xYZvZXYA7eG19r4H5gL5Dt\nZ10+eQyYb2Z34MXvfgVcC1zna1U+KPlFehjwrHOuOJRjxFRTUINhS8niFOBdvCvtHd69G8C7aGS4\nX0X54Hq8n/+9ctuvAZ6LejX+aoH3/z8DyAeWABck+dX3Acm4OnA43nj5Q4EtwAfA6SVj55OKc27R\n/2/vjgkQiIEACG68YQAl2EAKrpASij8FNPliRsGVW9wla61n9e46T/1Wr7335+xkRzy6PiT8e6/k\nVu8UAADn3GanAAA4SxQAAJUoAACGKAAAKlEAAAxRAABUogAAGKIAAKhEAQAwRAEAUIkCAGD8AHit\n1mA1gCvCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11b937f90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ca = [a[x] for x in ptha[0]]\n",
    "cb = [b[x] for x in ptha[1]]\n",
    "plot(a, label='a')\n",
    "plot(b, label='b')\n",
    "plot(ca, '--', label='ca')\n",
    "plot(cb, '--', label='cb')\n",
    "legend(loc='best')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dist = 0.25\n",
      "cost = [[  1.   0.   1.   1.   4.   9.]\n",
      " [  0.   1.   0.   0.   1.   4.]\n",
      " [  4.   9.   4.   4.   1.   0.]\n",
      " [  1.   4.   1.   1.   0.   1.]\n",
      " [  4.   9.   4.   4.   1.   0.]\n",
      " [  9.  16.   9.   9.   4.   1.]]\n",
      "acc = [[  1.   1.   2.   3.   7.  16.]\n",
      " [  1.   2.   1.   1.   2.   6.]\n",
      " [  5.  10.   5.   5.   2.   2.]\n",
      " [  6.   9.   6.   6.   2.   3.]\n",
      " [ 10.  15.  10.  10.   3.   2.]\n",
      " [ 19.  26.  19.  19.   7.   3.]]\n",
      "path = (array([0, 0, 1, 1, 2, 3, 4, 5]), array([0, 1, 2, 3, 4, 4, 5, 5]))\n"
     ]
    }
   ],
   "source": [
    "dst, cst, acc, pthb = dtw(a, b, dist=squared_norm)\n",
    "print('dist = {}\\ncost = {}\\nacc = {}\\npath = {}'.format(dst, cst, acc, pth))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x11be6df50>"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgUAAAFkCAYAAACw3EhvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3Xl4VPXZ//H3nUCAICAKSEBQFkVcEdSqgErdKiCCCxDU\nBwQF0VaN+mtR69pWxT5WBVlstRUFUgXsIxD3lQoqkqCgoCCLyKKAaIIgS5Lv74+TgSxkmcnMnFk+\nr+vKdZkzc865MwJz5zvnc25zziEiIiKS4ncBIiIiEhvUFIiIiAigpkBERERKqCkQERERQE2BiIiI\nlFBTICIiIoCaAhERESmhpkBEREQANQUiIiJSQk2BiIiIALVsCsxsjJkVm9nfqnneOWaWa2a7zGyF\nmQ2tzXlFREQk/EJuCszsVGAk8Fk1zzsSmAu8DZwEPAE8bWbnh3puERERCb+QmgIzOwiYClwL/FTN\n00cDq51zv3fOfeWcmwDMBLJCObeIiIhERqgrBROAOc65d2rw3NOBt8ptex04I8Rzi4iISATUCXYH\nMxsMdAFOqeEuLYHvy237HmhsZvWcc7sPcI5DgQuBtcCuYGsUERFJYvWBI4HXnXM/BLNjUE2BmR0O\nPA6c55zbG8y+QboQmBbB44uIiCS6K4HpwewQ7EpBN6A5kGdmVrItFTjLzH4L1HPOuXL7fAccVm7b\nYUDBgVYJSqwFmDp1Kp07dw6yxMSSlZXFY4895ncZMUGvhUevgyfZX4fvv4f/+R9o0QKaNs1i3Ljk\nfS0Ckv3PBF9+CS+8wPIBA7jqmmug5L00GME2BW8BJ5Tb9iywHHj4AA0BwIfAReW2XVCyvTK7ADp3\n7kzXrl2DLDGxNGnSJOlfgwC9Fh69Dp5kfh22b4cRIyA9Hd5+G0aOTN7XorRk/jPB+vVw8cXQqhUc\ndVRga9AfvwfVFDjndgDLSm8zsx3AD8655SXfPwi0ds4F7kUwGbjRzMYC/wTOBS4HegdbrIhIsiss\nhMxMWL0a5s+Hli39rkh8t3079O0LqakwezZs2hTyocJxR8PyqwMZQJt9Dzq3FugDnAd8ihdFHOGc\nK59IEBGRatx6K7z2GsyYAccf73c14rvSXWJODmRk1OpwQacPynPO/brc99cc4Dnz8K5HEBGREI0b\nB+PHw+TJcMEFflcjMSHQJebkwAnlP90PXq2bAomszMxMv0uIGXotPHodPMn2OsydC1lZcPvtMGpU\n2ceS7bWoTNK9DoEucdIkuPDCsBzSDnxtoL/MrCuQm5ubm7wXjYiIlFi8GHr29FYHZs6EFI2yk+3b\noVMnGDIE/vd/yzyUl5dHt27dALo55/KCOWzcrhSsW7eOrVu3+l1G3GrWrBlt27b1uwwRqcb69d41\nZMceC1OnqiGQEo0awaJFcFj5xH/txGVTsG7dOjp37szOnTv9LiVupaens3z5cjUGIjFs+3YvZVan\njndReXq63xVJTGnVKuyHjMumYOvWrezcuVM3NwrR8uXLueqqq9i6dauaApEYpeih+CEum4IA3dxI\nRBJV4KLyV15R9FCiJ66bAhGRRKToofhFl6yIiMSQqqKHkqR+/DFqp1JTICISIxYvhsGD4ZJLYOxY\nv6uRmDBnDrRrB198EZXTqSkQEYkBih5KBYsXe1eb/vrXEKWL6vXHTkTEZ4oeSgWBLrFz56h2ibrQ\nUETER4oeSgWlu8Q5c6LaJaopEBHxkaKHUkagS1y1ChYsiHqXqKZARMQnih5KBaWnHvrQJeqaghiz\nbt06brjhBo455hjS09Np1qwZAwcO5JtvvvG7NBEJI0UPpQLnIDUVnnwybFMPg6WVghjzySef8NFH\nH5GZmcnhhx/O2rVrmThxIr169WLZsmXUr1/f7xJFpJYUPZQDMoPHHvO1BDUFMaZv375cdtllZbZd\nfPHFnH766cyaNYsrr7zSp8pEJBwUPZRYlhRNwc6d8OWXkT3HMceE5wLRevXq7fvvwsJCCgoKaN++\nPQcffDB5eXlqCkTimKKHEuuSoin48kvo1i2y58jNhXDMZtq1axcPPvggzz77LBs2bMA5B4CZkZ+f\nX/sTiIgvFD2UeJAUTcExx3hv2pE+Rzj89re/ZcqUKWRlZXH66afTpEkTzIxBgwZRXFwcnpOISNQp\neijxICmagvT08PwWHw2zZs1i2LBhPPLII/u27d69m59++snHqkSkNhQ9lAr+/W+47DKoW9fvSsrQ\nJS4xJjU1tcKKwLhx4ygqKvKpIhGpDUUPpYJx47zPknJy/K6kgqRYKYgnffv25fnnn6dx48Yce+yx\nfPjhh7z99ts0a9bM79JEJEiKHkoFc+Z4XeJtt0H//n5XU4Gaghgzbtw46tSpw/Tp09m1axc9evTg\nrbfe4sILL8TM/C5PRGpI0UOpIDD1sF+/mO0S1RTEmMaNG/P0009X2L569WofqhGRUCh6KBWUn3qY\nmup3RQekpkBEJIx8nmcjsWj7dq8hSE31usSGDf2uqFJqCkREwsjneTYSi0aP3n+DiowMv6upkj7l\nEhEJk0D0cMIE3+bZSCz64x/h//4PTjjB70qqFVRTYGbXm9lnZpZf8rXAzH5TxfPPNrPicl9FZtai\n9qWLiMQORQ+lUsccA7/+td9V1EiwHx98C/wBWAkYMAx42cy6OOeWV7KPA44Gtu/b4Nzm4EsVEYlN\nih5KogiqKXDOlb/Twh/NbDRwOlBZUwCwxTlXEGxxIiKxTtFDSSQh//E1sxQzGwykAx9W9VTgUzPb\naGZvmNmZoZ5TRCSWKHooiSbo9IGZHY/XBNTH+0hggHOussHEm4BRwCKgHnAd8J6Zneac+zS0kkVE\n/KfooSSiUCKJXwInAU2Ay4HnzOysAzUGzrkVwIpSmz4ysw5AFjC0uhNlZWXRpEmTMtsyMzPp1KlT\nCGWLiISPoodSRlERDBsGw4dDr15RO212djbZ2dlltuXn54d8vKCbAudcIRC4vd5iMzsNuBkYXcND\nLAS61+SJjz32GF0PMN4wLy+vhqcSEQm/0lMPFT0UwIueZGfDVVdF9bSZmZlkZmaW2ZaXl0e3bt1C\nOl44bl6UgvfRQE11wftYQUQk7ih6KBUkUJcY7H0KHjSznmZ2hJkdb2YPAWcDU0sef8jMppR6/s1m\n1s/MOpjZcWb2ONALeDKcP0Siue+++0hJSWHbtm1+lyIipSh6KBUEusTbbkuILjHYlYIWwBQgA8gH\nlgAXOOfeKXm8JdCm1PPTgEeBVsDOkuef65ybV5uiE52ZaSKiSIxR9FAqCHSJMTz1MFjB3qfg2moe\nv6bc938F/hpCXSIiMUPRQ6kgTqYeBksDkUREqqDooRzQ88/HxdTDYGkBLIZt2bKFgQMH0qRJE5o1\na8Ytt9zC7t27/S5LJKkEooczZih6KKWMGQOLFsX81MNgqSmIUc45Bg4cyJ49e3j44Yfp06cP48aN\nY1QCXMgiEi809VAqZQYtEm+2nz4+iGEdOnTgpZdeAmD06NE0atSISZMmcfvtt3O8fmURiShFDyUZ\nJUVTsHPvTr7cWtmdmMPjmGbHkF43fFcfmRk33nhjmW2/+93vmDhxIq+88oqaApEIUvRQklVSNAVf\nbv2Sbn8P7e5ONZU7MpeuGRXvvlgbHTt2LPN9hw4dSElJYe3atWE9j4jsp+ihJLOkaAqOaXYMuSNz\nI36OSNO9C0QiS9FDqWDrVmjaNGEih9VJiqYgvW562H+Lj4aVK1dyxBFH7Pv+66+/pri4mCOPPNK/\nokQSlKKHUsH27XDeeXD66d4tjJOAFsZilHOOCRMmlNk2btw4zIyLLrrIp6pEEpeih1JGYaF3Ycnq\n1VDu+q5ElhQrBfFqzZo1XHLJJfzmN79hwYIFTJs2jauuuooTTjjB79JEEkoCzbORcMnKgtdf92Zj\nJ9G/uVopiFEpKSm88MIL1KtXjzvuuINXX32Vm266iaefftrv0kQSiqKHUsG4cfDkk95XknWJWimI\nQffeey/33nsvAC+++KLP1YgkLkUPpYI5c/ZPPbz+er+riTqtFIhIUio/z0bRQ2HxYu9q0wSaehgs\n/TUQkaRTOno4Z46ih1IiNRXOPz+hph4GSx8fiEhSUfRQKnXiifCf//hdha/UFIhIUglED3NyFD0U\nKU9NgYgkDUUPRaqmawpEJCkoeihSPTUFIpLwFD0UqRk1BSKS0BQ9lAo+/RQ+/NDvKmKSrikQkYSl\n6KFUsH499OkDHTrA+++Dps+WoaZARBKSoodSwc8/e11iaiq88IIaggNQUyAiCUnRQymjqMi7sGTV\nKpg/HzIy/K4oJqkpEJGEo+ihVJCVtb9LTKKph8FSUyAiCUXRQ6lAXWKN6TpcEUkYih5KBYsW7Z96\nqC6xWlopEJGEoOihHFC3bvD88zBokN+VxAU1BSIS9xQ9lEqZwZAhflcRN4Lqpc3sejP7zMzyS74W\nmNlvqtnnHDPLNbNdZrbCzIbWruTksHHjRkaMGEHr1q2pX78+7du354YbbqCwsJAff/yR22+/nRNP\nPJFGjRrRpEkTevfuzZIlS/wuWyTqSkcPc3IUPRSpjWBXCr4F/gCsBAwYBrxsZl2cc8vLP9nMjgTm\nAhOBIcB5wNNmttE592boZSe2TZs2ceqpp1JQUMCoUaPo1KkTGzZsYObMmezcuZPVq1cze/Zsrrji\nCtq1a8f333/PU089xTnnnMOyZctoqX8VJYkoeigSPkE1Bc65nHKb/mhmo4HTgQpNATAaWO2c+33J\n91+ZWQ8gC4huU7Bpk/dVmfr14dhjqz7GsmWwa1fF7RkZYc28jhkzhs2bN7Nw4UJOPvnkfdvvu+8+\nAE488URWrFhRZp+rr76aTp068cwzz3DXXXeFrRaRWKaLykXCK+RrCswsBRgIpAOV3UT6dOCtctte\nBx4L9bwhe+opuP/+yh8/9lj44ouqj3HFFV5jUN6990LJG3ZtOed4+eWX6devX5mGoLS6devu++/i\n4mJ++ukn0tPT6dSpE3l5eWGpQyTWKXrocc5hujOfhEnQTYGZHY/XBNQHtgMDnHNfVvL0lsD35bZ9\nDzQ2s3rOud3Bnj9ko0ZBv36VP16/fvXHmDGj8pWCMNmyZQsFBQUcd9xxlT7HOcfjjz/OpEmTWLNm\nDUVFRQCYGc2aNQtbLbHip5+8NNEf/gBHH+13NRILvpi7hm0D/sRVv3mEsWMT7898TRUVF3H5jMu5\npNMlDOsyzO9yfPXmK6+Qs2oVj48cCfXqVfq8yz7/nB/27q308d+2bs3lLVpU+vjSn3/mdytXVlnL\nzOOOo1laWqWPj1+/nllbtlT6+AkHHcT4o46q8hxV/RwdN2yoct+qhLJS8CVwEtAEuBx4zszOqqIx\nCFlWVhZNmjQpsy0zM5NOnToFf7BwLPFX9/FClPzlL3/hnnvu4dprr+XPf/4zhxxyCCkpKdx8880U\nFxf7XV7Y/eMf8M9/wrx53mCzBOx7JAjr18PiQQ/xP4X/4sqfVpKy960q3wQSWdbrWcz5ag6jTxnt\ndym+WrxoEQPMuKBBAyi1knogGWlpNKgir3pQamqV+6elpHB4NX/eUqtZuTm4Tp0qj9G8mp8B9v8c\na3NyWPvKK2Ue+3bnzmr3r5RzrlZfeNcGTKrksfeBv5XbNgz4sZpjdgVcbm6uO5Dc3FxX1ePxrLi4\n2DVp0sQNGDCg0ud06dLFnXvuuRW2H3744a5Xr17VniOeXr+9e51r29a5Cy90rnlz53r0cG7XLr+r\nEr8UFDh31vE/uJ3WwO28+Arn6tVz7qqrnCsu9ru0qHvioycc9+EmfzLZ71J89e2aNa7VrFnu1ClT\n3I6CAr/LiQmBf+OBri7I9/Rw3N4jBais5fkQOLfctguo/BqEpGdm9O/fnzlz5lR6fUBqamqgedpn\nxowZbKjFklGsmjMH1q2Dv/wFXn4ZPvkEhg+Hcj++JIFA9LDHyn9Sr04RDZ5+Ep57zrtT0QMP+F1e\nVM1dMZes17O4/YzbGXVK8l5Q8fNPP3HxBx9Qp7iY2b16kd6okd8lxb2gPj4wsweBV4F1QCPgSuBs\nvDd6zOwhoJVzLnAvgsnAjWY2FvgnXoNwOdA7LNUnqAcffJA333yTs846i5EjR9K5c2c2btzIzJkz\nmT9/Pn379uWBBx5g+PDhnHnmmSxdupRp06bRoUMHv0sPu3Hj4MwzvZuSgfceMGgQHHVU2K7tlDgR\niB5mvXAtKQ2PhxYtYOBA+Ppr2LLF6xST4IK7xZsWM3jmYC7pdAljz0/eezkX7d3L4FmzWJWRwYKM\nDFq2aeN3SQkh2GsKWgBTgAwgH1gCXOCce6fk8ZbAvv8zzrm1ZtYHL21wE7AeGOGcK59IkFJatWrF\nxx9/zN1338306dMpKCigdevW9O7dm/T0dO6880527tzJ9OnTefHFF+nWrRuvvPIKY8aMSairkJcu\nhffeg3//e/+2gQO9m9TceSd06ABXX+1beRJFpaOH5152MFDqnml33JEUzQDA+oL19M3uy7HNj2Xq\npVNJseS9l3PWlCm81q4dOUVFHF9JUkuCF+x9Cq6t5vFrDrBtHtAtyLqS3uGHH86//vWvSh9/5JFH\neOSRR8pse+eddyp5dnwaPx5atYJLLy27fcwY75fDESPgiCPgrLP8qU+io9roYZI0BD/v+ZmLsy+m\nTkodZmfOJr1u8t7LedW33/J0mzZM2LKFCwcP9ruchKLZBxKTtm3zPiq+886KFxObwaRJsHYtDBjg\nJRIUVUxMmnroKSouYvDMwaz+cTXzh8+n5UHJfdfSDm3a8FVaGm0OO8zvUhJO8q49SUx75hkoKoKR\nIw/8eFoazJzpfazcpw9s3Rrd+iTyNPVwv6zXs3jt69eYccUMjm+hezkDaggiJIn/mkmsKiyEJ5/0\nrjSv4h4iNG3q3e8+P99bMdgdvVthSYRp6uF+4z4ex/iF45nQewIXdLjA73IkwakpkJgTiCH+7nfV\nP7d9e0UVE03Yph7u3ev9YYpjih5KtKkpkJhTPoZYnTPO8KKK06dXPd5C4kMgejhjRsnUw6IiWLQo\n+AO9+KJ3a/OpU8NeYzQoeih+UFMgMSUQQ7zppuD2GzgQHnzQawri9D1A2B89nDCh1NTDOXPg1FOr\nH1hW3pAhcM013hLSvHlhrzWSFD3c76dvvvG7hKSSvH/SJCZVFkOsiTFjvH//4/A9QKgiehhYOqpi\nSNgBmXk3NujRw7vopNy48Vil6OF+46ZN47hFi/hhzRq/S0kaagokZgRiiKNHVzvT5IACUcWePePq\nPUCoInq4dCm8+27wS0cBaWkwa1bcxFRKRw9zhuQkdfRwzquvktWyJZk//cSh7dr5XU7SUFMgMaO6\nGGJNKKoYf6qMHtZm6SggjmIqih56Fi9aRCbQb+1axg4b5nc5SUVNgcSEmsYQayKO3gOSXpXRwx9+\n8LqEG24IbemotNIxlVtuqd2xIkTRQ8/6tWvpu24dnbduZerAgaRWM8pYwktNgcSEYGKINaGoYuyr\nNnoYWDq67rrwnPCMM7xEws03h+d4YaTooWd7fj59588n1Tlmn3MODTX1MOrUFMShKVOmkJKSUulo\n5XgUbAyxJhRVjG0VooelFRZ6EYRwLB2V1q8fHHNM+I4XBooeegoLC8mcOZPVTZuS0749GZp66AvN\nPohTiT4NMVxKT1Xs2BGuuir855DglZ56uC96WNrOnd5Vh9dUmLGWUBQ93G9vcTENGjRgRp06nKCp\nh75RUyC+C8e1ZFUJTFUcPhzattVURb9VO/UQoHFjr3NIYIoeltUgLY0ZQ4b4XUbSS962VGJCbWOI\nNaGoYuzQ1EOPoocSq9QUxKiNGzcyYsQIWrduTf369Wnfvj033HADhYWF+56zY8cORo0aRbNmzWjS\npAlDhw7lp59+8rHq4IUjhlgTiir6T1MP91P0UGKVPj6IQZs2beLUU0+loKCAUaNG0alTJzZs2MDM\nmTPZuXMnAM45fvvb39K0aVPuv/9+vvrqKyZOnMi6det49913ff4JaiacMcSaCEQVTz/dWzF46y2o\nVy/y55U4mHr47rveTY7Gj/eWliIoED2c3GdyUkcPJTapKYhBY8aMYfPmzSxcuJCTS11wc99995V5\nXv369Xn77bf35Xjbtm3LH/7wB+bOnUvfvn2jWXJIwh1DrIlAVLFXLxgxAp5/PuLvAUmvdPRwwYJa\nTD2MpM2bvbRD8+Zw770RO42ihxLrkqYp2LR7N5v27Kn08fopKRzbsGGVx1i2Ywe7iosrbM9ISyMj\nTL9yOud4+eWX6devX5mG4EBGjhxZ5sYeo0eP5s477+SVV16Ji6YgEjHEmghEFQcN8hIJ5XotCbNA\n9DAn5wDRw1gxaJDXtdx1F3ToEJGYiqKHnqLCQmZMn86gQYMwLdXFnKRpCp7auJH7q5i2dWx6Ol+c\ndlqVx7jiiy9YVrJ8X9q9RxzBfWG6N/eWLVsoKCjguGqGv5gZHTt2LLOtYcOGZGRksHbt2rDUEkmR\njCHWhKKK0VFt9LC04mJ/LzS44w4vpjJiRNhjKooe7pf17LNMbNeOYxct4sTu3f0uR8pJmqZgVKtW\n9GvWrNLH69fgH6MZxx1X6UqBBCfSMcSaUFQxsmoUPSztoou8iMgf/xjx2g4oMFVx7VrvopOPPoKj\njqr1YRU93G/ctGmM79iRyd99x4mDB/tdjhxA0jQFGfXq1XqJv7qPF8KhefPmNG7cmM8//7zK5znn\nWLlyJWefffa+bTt27GDTpk306dMn0mXWSiCGeOedkYsh1kQgqhh4D/jwQzj6aP/qSSRBRw+XLoU3\n3vA6ND8FpiqeeSb07u01BoceGvLhAtHDVdtWsWDEgqSOHs4tmXp426pVjBoxwu9ypBLJu4YVo8yM\n/v37M2fOnGpvY/z3v/+9TERx4sSJFBUV0bt370iXWSvRiiHWhKKK4RdS9DAWlo4CSk/UevbZWh1K\n0UPP4kWLGAz0W7OGsUOH+l2OVCFpVgriyYMPPsibb77JWWedxciRI+ncuTMbN25k5syZzJ8/f9/z\n9uzZw7nnnsvAgQP58ssvmTRpEj179ozpiwyjHUOsCUUVwyek6GFg6eiuu/xdOiqtfXvIy4PWrUM+\nROno4YUdq7ugInHtm3q4Y4c39bCO3nZimf7vxKBWrVrx8ccfc/fddzN9+nQKCgpo3bo1vXv3Jr3k\nX1kz48knn2TatGnce++97N27lyuvvJInnnjC5+qr5kcMsSYUVay9kKOHzzzjXWQYC0tHpR1+eMi7\nKnrocc5x+QcfkNqggTf1sHFjv0uSaqgpiFGHH344//rXvw742NChQxlasgTXo0cPJk2aFM3SasWv\nGGJNKKpYOyFFDwNLR4MHe/cISACKHu5nZjxy4ok0LS7W1MM4oaZAosbvGGJNKKoYmqCih6XF6tJR\niBQ9rOisE0/0uwQJQlB/Ys3sDjNbaGYFZva9mf3HzKq8XtvMzjaz4nJfRWYWI58oS7TE0rVkVRkz\nxrsIfvhwmDfP72piX9DRw9Ly8qBHj9hcOgqSooeSCIJdKegJjAcWlez7EPCGmXV2zv1SxX4OOBrY\nvm+Dc5uDPLfEsViJIdZE+ahimOLqCanWUw//9CfYvTvsdUVUURHs2OGNdw5sUvRQEkRQKwXOud7O\nueedc8udc0uBYUBboCZt/hbn3ObAVwi1ShyLpRhiTZSOKvbuDT/84HdFsSdsUw/jLepx3XXeD16q\nmVH0UBJFbT/wOhhvFWBbNc8z4FMz22hmb5jZmbU8r8SRWIwh1kTpuHr//vH3C20kxfzUw0i67jpY\nuBCuvRac2xc9nNB7QlJHD3HO7wokDEJuCszMgMeBD5xzy6p46iZgFHAZcCnwLfCemXUJ9dwSX+L5\nWrJAVPGTT7yoov7dKxs9zMmJ0amHkXTGGTBlCkydylc3X6XoIbA9P58BTz3F8vfe87sUqaXapA8m\nAscCVU60cM6tAFaU2vSRmXUAsgDd2ioJxHIMsSYUVSwrLqYeRtqgQWz89L90engCj97YjZuSOHpY\nWFhI5syZzGvblsImTfwuR2oppKbAzJ4EegM9nXObQjjEQqppJgCysrJoUu4PWWZmJp06dQrhlOKH\neIgh1oSiip6Qo4cJZn3Ben7V7CUmndmMm/+xFBv4QdJO1Lr12Wd5rV07coqKOKGace8SftnZ2WRn\nZ5fZlp+fH/Lxgm4KShqCS4CznXPrQjxvF7yPFar02GOP0bVr1wrbAzMBli9fHuLpk1s0X7d4iSHW\nRGCqYgQm68aFWkUPE8j23du96GFqXU6b8xF2+bCkjakEph5O+u47LtTUQ19kZmaSmZlZZlteXh7d\nQlyaDaopMLOJQCbQD9hhZoeVPJTvnNtV8pwHgdbOuaEl398MrAG+AOoD1wG9gPNDqhho1qwZ6enp\nXJWsv66FQXp6Os2qGCUdDvEUQ6yJZI4q1jp6GPD5596y0ZgxcNBBYasvWgqLC8mclbk/enhIW2+q\n4qBBsGuX3+VFVemph9dr6mHCCHal4Hq8tMF75bZfAzxX8t8ZQOn7WaYBjwKtgJ3AEuBc51zIt4Vp\n27Yty5cvZ6tG2oWsWbNmtG3bNqLniLcYYk0EoophmqwbFwLRw2OPrWX0ELzPH3Jy4N57w1ZfNN36\n+q289vVr5AzJ2R89bNrUG/ucRPZNPVy7lrHDhvldjoRRUE2Bc67afw6cc9eU+/6vwF+DrKtabdu2\njfibmoQuXmOINVF6qmL//ok9VbF09HD27FpGD2NxGmIQNPVwvydWrOCYoiJv6mFqqt/lSBhp9oFE\nRDzHEGsiGaYqhjz1sDKBpaPrrgtLfdGkqYdl/WPQIAq2b6dho0Z+lyJhpmkdEhHxHkOsiUBUcdo0\nuP9+v6sJv0D0cMaMMEQP43jpSFMPK6qbmsqhBx/sdxkSAVopkLBLlBhiTSRqVDHs0cM4XTrS1ENJ\nNmoKJOwSKYZYE4kWVYxI9HD8+LhbOtoXPazN1MOPPoJDDoGjqxwmKxIz1PZKWAWuJRs9Oi6vJQtJ\nIKrYvbsXVVy50u+KQhe26GFpy5bBu+/CTTeF6YCRVzp6mDMkJ7Sph8XFcOON0KcPKCklcUJNgYRV\nIsYQayIJ70OXAAAgAElEQVQtzYurx/NUxbBGD0vr3NmLaMTR0lEgelirqYcpKd4FGfn5XrcYhxO1\ntqxahdu71+8yJIrUFEjYxPG1ZGERz1MVwxo9LM8Mzj03bpaOwjr1MI4naq1fu5Yun33GY08/7Xcp\nEkVqCiRs4vRasrAq/R5QMlk35iX91MNSIhI9jMOYys8//cTFH3xAqnNk9u3rdzkSRbrQUMImGWKI\nNVF+qmKs37xPUw89EY0eDhzoXY16110xH1Mp2ruXwbNmsSojg/kZGWS0aVP9TpIw1BRIWCRTDLEm\nSkcVO3SI3fcATT30BKKHnZt3jlz08I479sdUTj4Zjjsu/OcIg6wpUzT1MImpKZCwSLYYYk3EelRR\nUw89paOHczLnhBY9rAkzr/vq0cO7mjMGBaYeTtbUw6Slawqk1pIxhlgTsRxVjEj0MA6FJXoYjLQ0\nGD48Ju+JXXrq4Sg1BElLTYHUWrLGEGsiFqOKEYseljZ/PvzySwQOHF5hiR4miLotWnDVt98yduhQ\nv0sRH6kpkFpJ9hhiTcRSVDGi0cOAbdvg/PO9CxZiWFijhwngwm7dmDJsGKl19KlyMlNTILWiGGLN\nxEJUMWrRw8DS0TXXVP9cn2jqociBqSmQWlEMseYCUcWpU+GBB6J//rBOPaxMHCwdaeqhSOXUFEjI\nAjHEOLqlve8GDoQHH4T77vOag2gJRA8nTIhw9DDGl46iEj0Mxa5dMHQozJvndyWS5GLkb4TEI8UQ\nQzNmjHcB+ogR0XkPiGr0MIaXjqIWPQxFSgp8+60XU1mxwu9qJImpKZCQKIYYumhGFaMaPYzhpaOo\nRw+DVTqmEuGpim++9hqrFiyI2PElvqkpkJAohlg70YgqRiV6WFoMLx3FRfSwdEwlQlMVFy9axADn\n+MuSJWE/tiQGNQUStDi4liwuRPI9ICrRw/K6d4c//Snmlo7iKnpYOqYyfHhYYyrr166l77p1dN66\nlfFXXhm240piUVMgQYvxa8niSuA9YOHC8EUVfZt6OHSo90YWQ+IyenjGGTBlCkyfHrapitt/+om+\n8+eT6hyze/WiYaNGYTmuJB41BRK0GL6WLC6FO6oYlehhHIjr6OGgQfCXv3izErZtq9WhCvfuZfBL\nL7G6aVNyOnQg4/DDw1SkJCLdukqCommIkVF6sm5tpipq6qEnZqOHwbjjDm/56JBDanWYrOee4/Uj\nj/SmHnbpEqbiJFGpKZCgxPC1ZHGv9GTdI46Anj2D219TDz0xHT0MhlmtL9p54Y03eLJDB009lBqL\nw/ZZ/KIYYmQFJut27+7NSAgmqqiph56Yjx5GWf9zzyW7sFBTD6XG1BRIjSmGGHmhRBWjHj2MYXER\nPYyieqmpDD7vPL/LkDiSxP98SDAUQ4yeYKKKvkQPA/bu9WeyUyXiKnooEqOCagrM7A4zW2hmBWb2\nvZn9x8yOrsF+55hZrpntMrMVZqaB3XFGMcToqklU0bfoYcDjj8Npp3nLRz6Ly+hhbWze7HcFkqCC\nXSnoCYwHfgWcB9QF3jCzBpXtYGZHAnOBt4GTgCeAp83s/BDqFZ8ohhh91UUVfY0eBpaOjjsOUlOj\nfPKy4jp6GIqVK72ISjQnaknSCCp94JzrXfp7MxsGbAa6AR9UsttoYLVz7vcl339lZj2ALODNoKoV\nXyiG6J/Kooq+Rw9jZOkoIaKHwerYEa64wouptG0LZ53lbXfOu1pVpBZqG0k8GHBAVXfXOB14q9y2\n14HHanluiRLFEP1VPqqYnx8D0cMYWDpKmOhhsAIxlW++8S46+egjxi9cyIbvv+ehm27C6ihpLqEL\n+U+PmRnwOPCBc25ZFU9tCXxfbtv3QGMzq+ecC//UDwmbQAzxzjsVQ/RL4D1g7Vovqrh7t8/RwyVL\nfF862ltUyG9mZ/FVYQOeu/TlaqOH8/Pz2VtcXOnjR6Wn07pevUof/3HvXj77+ecqz9G9SRPqVhH9\nWLFzJxuruGr04Dp16FLN7YfL/BzPPgs33sjK++/nlmuuIWvXLjUEUmu1+RM0ETgW6B6mWirIysqi\nSZMmZbZlZmaSmZkZqVNKOYohxoZAVLFHDzjoIJ+jh08+6fvS0SnvPsuS5ldBc5iwDS5vW/XzL1m6\nlB8KCyt9fMJRR3FD69aVPv7pzz/z688+q/IcW848k2ZpaZU+/vj69UzauLHSx885+GDereaOgxV+\njltvBeDSr79m7LBhVe4riSk7O5vs7Owy2/Lz80M+nrkQIkVm9iRwMdDTObeumue+D+Q6524ttW0Y\n8Jhzrmkl+3QFcnNzc+natWvQ9Ul4FBZ6n2P36uX9UiL+273bawZ8W7X54Qdo08a7yOGuu3wp4bL5\n03hpb2suT9vMQ1360iA1tcrf8gFW//ILxVX8W9c8LY0mVfyWvbOoqMrf8gHaNWhAahWf6W/es4eC\nKhqTUH8O27WL9gcfjCXzDSqkjLy8PLp5H+11c87lBbNv0CsFJQ3BJcDZ1TUEJT4ELiq37YKS7RLD\nYuRaMimlmveMyPv8c+9GCj4tHd2z5FVe2tOSU4pXM+PMmk9kbN+g0oBUjaSnptKxljeBaJGWRosq\nVhJq4oA/R1RvTiGJLtj7FEwErgSGADvM7LCSr/qlnvOgmU0ptdtkoL2ZjTWzTmZ2A3A58Lcw1C8R\nFAPXkkmsOftsr1Ns3jzqp85es4g/bYGM3Wv4sJdudSISCcGuFFyPlzZ4r9z2a4DnSv47A2gTeMA5\nt9bM+uClDW4C1gMjnHPlEwkSQxRDlEr5cF+Cb/O/ZeiyT0lPrceScwZRJ8XfeyOIJKpg71NQ7cqC\nc+6aA2ybh3cvA4kTiiFKrNi+ezv9/t2P5nuLefXKHJrVr/oKfREJnfIrUkEghnjXXYohir9KTz1c\nMGIBxzc93O+SRBKamgKpIBBDvO46vyuRZBeYepgzJEdTD0WiQE2BlKFpiBIrAlMPJ/eZrKmHIlGi\nYKuUoRiixIKkm3ooEiPUFEgZiiFKGYWF3uzmpUujdsqFG/OSa+qhSAxRUyD7BGKIN93kdyUSM+bM\n8S4y2bMnKqf7ZOtazlyyglZHXJI8Uw9FYoj+xsk+iiFKBVFcOtq08yfOXvQBpNRlxkVjk2fqoUgM\n0YWGAiiGKAcQxTtY7S7aS5d5s/ilTgazjm7FSYcoeijiBzUFAiiGKAcQxaWj096dwuZ67fhz8yIu\nbVv1pEARiRx9fCCKIUpFgaWj0aMjvnR02fxpLKnTkSH1t3DX8RdE9FwiUjU1BaIYolQUWDqK8DTE\nfVMPi1Yz7YzBET2XiFRPTYEohihlOQdPPRXxpaPFmxbz8OIXydi9hgWaeigSE3RNQZLTNESpwAze\nfReKiyN2ivUF6+mb3ZeTGrXinV8/SV1NPRSJCWoKkpxiiHJAbdpU/5wQbd+9nYuzL6ZOSh3mZM6h\nUVrDiJ1LRIKjjw+SWOBashtuUAxRoqP01MOcITm0PKil3yWJSClaKUhiiiFKtGnqoUhsU1OQpBRD\nlGjT1EOR2KePD5KUYogSTX/7/BVueeP3mnooEuO0UpCkFEOUaMles4jbvjc6nfwAY8+/3e9yRKQK\nagqSkGKIUsFnn8FBB0GHDmE97Cdb13LVynWkF+/gv+eN1tRDkRinpiAJKYYoFdxyC+zdCx98ELZD\nbiyZemgpDVhwWi+a128UtmOLSGSoKUgymoYoFURg6WhX0V5O1tRDkbijpiDJKIYoFURg6ehXmnoo\nEpf0AV8SUQxRKojANMRB86dq6qFInFJTkEQUQ5QKwjwNce6Kucz44PecU7hMUw9F4pCagiSiGKKU\nEealo8WbFjN45mD6H3k6b597fRgKFJFo0zUFSUIxRKkgjEtHgamHnZt3ZuqlUxU9FIlT+pubJBRD\nlArat4e776710lH5qYfpddPDVKCIRFvQTYGZ9TSz2Wa2wcyKzaxfNc8/u+R5pb+KzEyXukWJpiHK\nAZ10EjzwQK0OoamHIokllJWChsCnwA2Aq+E+DjgKaFnyleGc2xzCuSUEiiFKpASmHs64YoamHook\ngKCvKXDOvQa8BmBmFsSuW5xzBcGeT2pHMUSJlIHzpzLj60+Y3HuCph6KJIhoXVNgwKdmttHM3jCz\nM6N03qSnGKJEwj1LXmXGngxOOfZaTT0USSDRaAo2AaOAy4BLgW+B98xMtzmLAsUQJdyy1yziT1sg\nY/daFvQa5nc5IhJGEY8kOudWACtKbfrIzDoAWcDQqvbNysqiSZMmZbZlZmaSmZkZ9joTkWKIEm6l\npx5+ds5A6qak+l2SSFLLzs4mOzu7zLb8/PyQj2fO1fRawQPsbFYM9HfOzQ5yv0eA7s657pU83hXI\nzc3NpWvXriHXl+xGjoScHFi7VqkDKbFzJ6SHFhnc9Es+Hd6fw56UBuSe8isNORKJUXl5eXTzloe7\nOefygtnXr/sUdMH7WEEiRDFEqWDpUmjZEhYvDnrX3UWFnPT+LH6pcwgvHtNBDYFIggr64wMzawh0\nxLt4EKC9mZ0EbHPOfWtmDwGtnHNDS55/M7AG+AKoD1wH9ALOD0P9UgnFEKWC8eOhcWM4Pvjo4KVv\n/S9b6p2qqYciCS6UlYJTgMVALt79Bx4F8oD7Sx5vCbQp9fy0kucsAd4DTgDOdc69F1LFUi3FEKWC\nWkxDHPfxOF756A7uabhOUw9FElwo9yl4nyqaCefcNeW+/yvw1+BLk1AphigVPPMMFBcHPQ1x7oq5\nZL2exe1n3M79p11T/Q4iEtc0+yABjRsH3bsrhiglAktHgwdD8+Y13i0w9fCSTpcw9vyxESxQRGKF\npiQmGMUQpYIQlo4CUw+PbX6sph6KJBH9TU8wmoYoFQS5dPTznp/3TT2cnTlbUw9FkoiaggSiGKJU\nsGOHd2+CGq4S7CkqZODMIZp6KJKk9PFBAlEMUSpo2BA+/hhqeJOyU999liX1z+KVy2/U1EORJKSm\nIEEohihVqsFA08vmT2NJnY4MyfiOi47S1EORZKSPDxKEYohSG/cseZWX9rTklKLVTDtjsN/liIhP\n1BQkCMUQJVRlpx5WOaNMRBKcPj5IAIohSqg09VBEStNKQQJQDFFC8d0v+Zy96AMMx4LTetG8fiO/\nSxIRn6kpiHOKIUooioqLGDJ7NLsLd2nqoYjso6YgzimGKBU88ABMmVLlU7Jez2LeVy+S06mNph6K\nyD5qCuKYYohSwbZt8PDD8O23lT5l3MfjGL9wPBN6T+A3ih6KSClqCuKYYohSQWDpqJJpiKWnHo46\nZVSUixORWKemII4phihlVLN0pKmHIlIdRRLjlGKIUkEVS0eaeigiNaF/GeKUYohSwfjxB1w62vRL\nPr3+7yZNPRSRammlIA4FYoh33aUYopRYuhTefbfC0tHuokK6vD+TLRnD+PD8+zT1UESqpKYgDimG\nKBX8618HXDo67d1n2VyvHX9uXsSvMk70qTgRiRf6+CDOKIYoB/Tww/Dmm2WWjvZNPay/hbuOv8DH\n4kQkXqgpiDOKIcoBpaXBscfu+1ZTD0UkFGoK4oxiiFIdTT0UkVDpmoI4ohiiVGfptm809VBEQqaV\ngjiiGKJU5ec9P3P1jP40+X6Oph6KSEjUFMQJTUOUqhQVFzF45mBWb1vFvAtv09RDEQmJPj6IE4oh\nSlWyXs/ita9fI2dIDse3ON7vckQkTmmlIA4ohigVbNvmRVGKispMPbywo6Yeikjo1BTEAcUQpYJn\nnoHLL+eNhdmaeigiYRN0U2BmPc1stpltMLNiM+tXg33OMbNcM9tlZivMTDmpICiGKGWULB390P9C\nLn33ek09FJGwCWWloCHwKXAD4Kp7spkdCcwF3gZOAp4Anjaz80M4d9IJxBC1SiD7zJnD+p07Oan3\nSXRq0UVTD0UkbIK+0NA59xrwGoCZWQ12GQ2sds79vuT7r8ysB5AFvBns+ZONYohS3o+TJnDuo2P5\nrmVjXjl1lKYeikjYROPXi9OBt8ptex04IwrnjmuKIUp5uz9dzNW9zmVNy5bMOKYjJzZV9FBEwica\nkcSWwPfltn0PNDazes653VGoIS6NH68YIkDu5x9z63vz2JHWYN+249auJOulKQD8v9tOZG/div3t\n0hZDKEpJ47AfdtFoR2GFx4e+8R96fp7LFx0a848r2ld4vCCtNasP8T7l6vDtz9gBPiwbN+HPpO/e\nxbQ+bfnkhEMqPP5t4zP5If1oGv5SSMaWXRUeD/bnqLO3Ll+ddhoPHLqXAW27VCxIRKQWYvo+BcNu\nGMWRLTLKbMvMzCQzM9OniqJn7lx44AH4wx+SO4b43bbvuDr3c37MaMeJ61fu255uhew4rCkArRq3\npqhuxdv5rk8ppsgKOWTvThr/coDes2EaOw5rijVryuGNK/7G/WPqIfxoXjNx6C/bD9gU/NK8CW5v\nAxo3bcnhjSv+j9pVrwHFVkjD4t0c+sv2Co8H/XPk7+ZqK2DMuVkVixGRpJOdnU12dnaZbfn5+SEf\nz5yr9lrBync2Kwb6O+dmV/Gc94Fc59ytpbYNAx5zzjWtZJ+uQG6d8Q+x7OprOKrJYSHXGI8WL4ae\nPeGCC2DmTEhJ0mvI9hQV0ubNKfxkLZlZby8Xn9Pf75JERGJeXl4e3by4WjfnXF4w+0bj7eZD4Nxy\n2y4o2V6lwpQ0TvnwVQr2/BKRwmLR+vXQty907uxdT5CsDQHAeW9PYHO9I7mndV01BCIiURDKfQoa\nmtlJZhb4QLN9yfdtSh5/yMymlNplcslzxppZJzO7Abgc+Ft157qr1cEUpLXipHenUVxcHGypcWf7\ndrj4YqhTx7thUXoSX1Q+7uNx/Pej/8dNDTZy1/EX+F2OiEhSCOX30FOAxUAu3n0KHgXygPtLHm8J\ntAk82Tm3FugDnId3f4MsYIRzrnwioYJL23bhlia/sLZeR34977kQSo0fhYXebYxXrYKcHGjZ0u+K\n/DN3xVyyXs/ittNv4onTr/a7HBGRpBHKfQrep4pmwjl3zQG2zQNCuh/fY90uYdn7U3jDHclNi2Yx\n7pTLQjlMzLv1VnjtNa8hOD6J59ks3rSYwTMH069TP8aep7v0iYhEU1x8Yv1qz6s59ZdcJr05innf\nzPO7nLAbN86LH06YABcm8Tyb9QXr6Zvdl87NOzN1wFRSUypeiS8iIpETF01BSkoKH1zwO3q2OpEB\nLwxg5Q8rq98pTsydC1lZcNttMCqJ59n8vO073rq8GwftMWYPnk3DtIZ+lyQiknTioikASEtNY9bA\nWbRo2ILe03vzw84f/C6p1hYvhsGD4ZJL4JFH/K7GP4V7drPs3JO4dN5m5nSfQEajjOp3EhGRsIub\npgCgaYOm5AzJIX9XPv1f6M/uwvi9GaKih/sNmvAwOSf9hhWT/szRPS/xuxwRkaQVd29F7Zu25+XB\nL/PJhk8YMXsEtbn5kl8UPdzvpvH389LJZ7PjtI6ccs1dfpcjIpLU4q4pADijzRk8N+A5pi2dxv3v\n31/9DjFE0cP9/jb1CSYc24OBi9/jf2+42+9yRESSXkzPPqjKwOMGsmrbKu5c9l++bziLSafGR1RR\n0UPPS2/O4J5Dj6bnslym3niH3+WIiAhxulIQMKbHGI5qdymTtzdi/Jfv+11OtRQ99HyyZQ3X/2wc\nuelbZgz5H+qm1fO7JBERIc6bAjMjr9dVHLxnAzevL+DNjcv9LqlSih56vvsln7Nz55PfAKafeQrN\nD03iz09ERGJMXDcFAAfVrU9e977ULfqZPkuXsDL/e79LqkDRQ09RcREDcv7ArpSDmN65Ayce09Xv\nkkREpJS4bwoA2jVqzusndaMwpX7MTVVU9HC/rNez+GTZ0/znyAZcdsTJfpcjIiLlJMxb1Dktj2by\nkc1iaqqioof7jft4HOMXjmdC7wlccnQSX1AhIhLDEqYpABh5VHdubvILa9Pacd28x32tRdHD/fZN\nPTzjNkadksQXVIiIxLiEagoAHu92CbemLuGf79/G1CVTfasjED2cMSO5o4eL1y3U1EMRkTiRcE0B\nwP/2vIXhXYYz/OXhvkxVVPTQs2n5Jxzc7UyGbW6lqYciInEgIZsCM2NS30n0aNsj6lMVFT30bNq8\ngZWDBlC3CO7+7QxNPRQRiQMJ2RSAP1MVFT307N69iytm/JtRt/2Z7S+9yGEdT/K7JBERqYGEbQog\nulMVFT3cb8jEsXx0TBduqLeDzudc6nc5IiJSQwn/1lV6quLQ2SMjElVU9HC/wNTDm7/6kN8NvtHv\nckREJAgJ3xSAN1Xxmf7P8ULd7vx63nNhPbaih/s9um/q4fs8esMf/S5HRESClBRNAcCVxw/kgsb1\neJ8juf6Tl8J2XEUPPdlrFnFni6PpsTyP53+rqYciIvEoaZoCgFd7Xs1Re1byVJimKip66Plk61qu\nWrmOOsVbmTHsWtLqpvldkoiIhCCpmoKUlBTyel3NwbvXc/P6At7YuCzkYyl66Pl5z8+MeGkQDQo+\nY/5p59DioKZ+lyQiIiFKqqYASqYq9vCmKvZdupQVIUxVVPTQU1RcxOCZg1n7w3I+7Hk5XQ5p43dJ\nIiJSC0nXFEDZqYqnBjlVUdHD/bJez+K1r19jxhUzOOGwE/wuR0REailp39ICUxW3797O8Jzf4Zyr\ndp9A9DA1FWbPTu7oYWDq4ZO9n+TCjkl8QYWISAJJ2qYAvKmK2R0OY9aSZ7j//furfG756GFGRpSK\njEH/nfUYY+bewm1n3Mb1p1zvdzkiIhImITUFZnajma0xs1/M7CMzO7WK555tZsXlvorMrEXoZYfP\noOMH8pdf/4X737+/yqmKpaOHJyTxSvmXb2Rz8pBbeXbZ0Zp6KCKSYOoEu4OZDQIeBUYCC4Es4HUz\nO9o5t7WS3RxwNLB93wbnNgdfbmTc0eMOvt72NcNfHk7bJm0564izyjweiB5Onpzc0cPczz/m1Wdn\nMiAjnT5Pz9PUQxGRBBPKSkEW8JRz7jnn3JfA9cBOYHg1+21xzm0OfIVw3ogxMyb3nXzAqYqKHnq+\n27yBq3M/Z9KAQdR9/b80bBoTCz0iIhJGQTUFZlYX6Aa8HdjmvCv03gLOqGpX4FMz22hmb5jZmaEU\nG0kHmqqo6KFnz+5dXD7j36xv1pLJzdM4ulNXv0sSEZEICHaloBmQCpQP938PVHbX/03AKOAy4FLg\nW+A9M+sS5LkjrvRUxTNn3U7vS39R9BDILJl6+Jfta7n4nP5+lyMiIhES9DUFwXLOrQBWlNr0kZl1\nwPsYYmikzx+s9k3bM77P/zF403ZS75rKootGkJ6evB3B7yb+iZdOPptbv/gvv7vxbr/LERGRCAq2\nKdgKFAGHldt+GPBdEMdZCHSv7klZWVk0adKkzLbMzEwyMzODOFVwCgvh+d+fSd3DXmbv1Udx5cop\nvNf6moidL5bds+RVnux8Jv2XL+BRNQQiIjEnOzub7OzsMtvy8/NDPp7V5KY9ZXYw+wj42Dl3c8n3\nBqwDxjnn/lrDY7wBFDjnLq/k8a5Abm5uLl27Rvfz65tugokTvXsRPFp/Cm+6I7i+4TYmnXppVOvw\n23/X53HW8g1kFH3P2vOHkZYa8UUlEREJg7y8PLp16wbQzTmXF8y+oayL/w24zsz+x8yOASYD6cCz\nAGb2kJlNCTzZzG42s35m1sHMjjOzx4FewJMhnDuiyk89fK3n1Ry152smh2mqYrxYX7CewS9ezDHf\nTeHTsweqIRARSRJBNwXOuReB24EHgMXAicCFzrktJU9pCZSejJOGd1+DJcB7wAnAuc6590KuOgIO\nFD30pipexcF7aj9VMV78vOdnLs6+mFRL5Z3+42nRoLHfJYmISJSE9Cugc24iMLGSx64p9/1fgRp9\nrOCXqqKHB9WtT173vhyz4C36Ll3KmsbNaX1Qc38KjbDA1MNV21Yxf/h8Mhol8b2cRUSSUPJeVl+i\nJlMPA1MV66//N4NnXMruwt3RLzQKNPVQRCS5JXVTEMzUw3NaHs3rF/yeTzZ8wojZI2o0VTGevPOH\ngaycpqmHIiLJLGmbglCmHp7R5gym9J/CtKXTqp2qGE8+mnwPZ/91Br/fc4qmHoqIJLGkvaw8MPUw\nJye4qYeDjh/Eqh9Xcdc7d9HxkI5cdeJVkSsyCl56cwb31W3L/eefQL/pC/wuR0REfJSUTUFtpx4G\npiqOmD3igFMV40Xu5x9zY4FxqBXTffqbpNap63dJIiLio6T7+CAcUw8DUxW7t+leYapivNi0eQNX\n5X5OSnEx07ufSotDy9+kUkREkk1SNQXhnHoYmKp4SOMOdP3vS6zILz8jKnbt3r2LK2b8mw3NWjK5\neT1O7HSy3yWJiEgMSJqmoCbRw2A1bdCUf/Sfyo6Gx3Dqh69SsOeX2h80Cnq/MKHU1MNL/C5HRERi\nRFI0BcFED4N1TsujmXxkMwrSWnHSu9MoLi4O38Ej4LL503inbTeG7VjJ7wbf6Hc5IiISQxK+KQgl\nehiskUd15+YmO1lbryO/nvdc+E8QJrO/mstL32+kW9Eqnu6n6KGIiJSV8E1BIHo4Y0Zw0cNgPd6t\nP+fbN7zPkYz+5KXInShEizctZsiswfTb+xELzhnqdzkiIhKDEropKD/1MNL2T1U8KKamKq4vWE/f\n7L50bt6Z6QOe09RDERE5oIRtCsIRPQzW/qmKG7h1xVJWbF0RnRNXofTUw9mDZ9MwraHfJYmISIxK\nyKYgED3s1w/Gjo3uuQ+qW5/F3S+m3YZn6JPdhx92/hDdAkop2ruHK/89kFXbVpEzJEdTD0VEpEoJ\n1xSUjx6mpka/hiMbNeO1wbPI35VP/xf6+zZV8b+Xn8boP73KjMtf1NRDERGpVkI1BeWjhw19XClv\n37Q9Lw9+2bepiv+6dyTnzP6M9MszufCo30T13CIiEp8S5oqz0tHD+fMjEz0MVmCq4uBZg+l4SEfu\nO+e+qJz3b1Of4P+dNYgNt+/mjw9Nico5RUQk/iVMUxDq1MNIi/ZUxZfenME9hx5Nj+V5/P4vf4/o\nuVnbGV4AAAkWSURBVEREJLEkRFNQ26mHkVZ6quIhjdrQu93ZETlPYOrhkfnfMmPw1aSl1YvIeURE\nJDHF/TUFfkQPgxWYqnj00SPo+/VPvLlxedjP8d3mDVy9qGTq4Zmn0KJZy7CfQ0REEltcNwV+Rg+D\nlZaaxuyL7qdu8U76LF3CyjBOVdxTVMhpH+awrkVLJjdP48Rjuobt2CIikjzitimIhehhsNo1as7r\nJ3WjMKU+p4RxquJp7z7Lt407cEebulx8Tv+wHFNERJJPXDYFsRQ9DFa4pyqO+3gcny1/mqvqb+au\nEy4IU5UiIpKM4q4piMbUw0gbeVR3bgrDVMW5K+aS9XoWt3XqwfNnZIaxQhERSUZx1xREa+phpD1R\ny6mKizctZvDMwfTr1I+x58X4BRUiIhIX4qopiPbUw0h7refVHL1nBf/49DnmfTOvxvuVnno4dcBU\nUlPi4IIKERGJeXHTFMRD9DBYKSkpLD13GGelFTDghQGs/GFltfv8/MMm/vjnX2vqoYiIhF1cNAXx\nFD0MVlpqGrMGzqJFwxb0nt67wlTF7Ozsff9dtGc3y87rwqMTvubVS2Yk3dTD0q9FMtPr4NHrsJ9e\nC49eh9oLqSkwsxvNbI2Z/WJmH5nZqdU8/xwzyzWzXWa2wsyG1vRc8Rg9DFbTBk3JGZJzwKmKpf+Q\n3zD2DppuTmHNk3/iuPa/8qNUX+kvvEevg0evw356LTx6HWov6KbAzAYBjwL3AicDnwGvm1mzSp5/\nJDAXeBs4CXgCeNrMzq/uXDt2xG/0MFjVTVW8afz9/L1nP/5x52hOueYun6oUEZFEFspKQRbwlHPu\nOefcl8D1wE5geCXPHw2sds793jn3lXNuAjCz5DhVuvPO+I4eBiswVXHa0mnc//79+7b/beoTTDi2\nBwMXv8cjN97jY4UiIpLIghqIZGZ1gW7Ag4FtzjlnZm8BZ1Sy2+nAW+W2vQ48Vt35FiyAV16J7+hh\nsPZNVZz3MDsPOo5NWzbum3r4/I13+F2eiIgksGCnJDYDUoHyN+7/HuhUyT4tK3l+YzOr55zbfYB9\n6gPccsFLNP9mEXkHmgDcrRuYVV7p6tWwbVvljx98MHTsWPnjALm5UG4Zv4x27eDQQyt//McfvaWO\nqhzg57gw/UKebfIjf120mToFv9D24/nc3fMsPv/8i6qPleDy8/PJy8vzuwzf6XXw6HXYT6+FR6+D\nZ/nyfUP36ge7r5X/7LrKJ5tlABuAM5xzH5faPhY4yzlXYbXAzL4C/umcG1tq20V41xmkH6gpMLMh\nwLRgfhAREREp40rn3PRgdgh2pWArUAQcVm77YcB3lezzXSXPL6hklQC8jxeuBNYCu4KsUUREJJnV\nB47Eey8NSlBNgXNur5nlAucCswHMzEq+H1fJbh8CF5XbdkHJ9srO8wMQVHcjIiIi+ywIZadQ0gd/\nA64zs/8xs2OAyUA68CyAmT1kZlNKPX8y0N7MxppZJzO7Abi85DgiIiISI4L9+ADn3Isl9yR4AO9j\ngE+BC51zW0qe0hJoU+r5a82sD17a4CZgPTDCOVc+kSAiIiI+CupCQxEREUlccTH7QERERCJPTYGI\niIgAMdgUBDtsKRGZWU8zm21mG8ys2Mz6+V2TH8zsDjNbaGYFZva9mf3HzI72uy4/mNn1ZvaZmeWX\nfC0ws9/4XZffzGxMyd+RpLpw2czuLfm5S38t87suv5hZKzN73sy2mtnOkr8rXf2uK5pK3jfL/5ko\nNrPxwRwnppqCYIctJbCGeBdw3gAk80UfPYHxwK+A84C6wBtm1sDXqvzxLfAHoCvercbfAV42s86+\nVuWjkl8YRuL9O5GMPse72LtlyVcPf8vxh5kdDMwHdgMXAp2B24Af/azLB6ew/89CS+B8vPePF4M5\nSExdaGhmHwEfO+duLvne8P4xHOece8TX4nxiZsVAf+fcbL9r8VtJc7gZ7+6ZH/hdj9/M7Afgdufc\nv/yuJdrM7CAgF2/g2t3AYufcrf5WFT1mdi9wiXMuqX4bPhAzexjvLrtn+11LLDGzx4HezrmgVldj\nZqWg1LCltwPbnNexVDVsSZLLwXidbxVDLRKfmaWY2WC8+4NUehOwBDcBmOOce8fvQnx0VMlHjKvM\nbKqZtal+l4R0MbDIzF4s+Zgxz8yu9bsoP5W8n14JPBPsvjHTFFD1sKWW0S9HYknJqtHjwAfOuaT8\n7NTMjjez7XjLpBOBASXjy5NKSUPUBUjmsaEfAcPwlsuvB9oB88ysoZ9F+aQ93orRV3h3y50EjDOz\nq32tyl8DgCbAlOqeWF7QNy8S8clE4Figu9+F+OhL4CS8v+yXA8+Z2VnJ1BiY2eF4zeF5zrm9ftfj\nF+dc6Xvaf25mC4FvgIFAsn2clAIsdM7dXfL9Z2Z2PF6z9Lx/ZflqOPCqc66ymUSViqWVglCGLUkS\nMLMngd7AOc65TX7X4xfnXKFzbrVzbrFz7i68C+xu9ruuKOsGNAfyzGyvme0FzgZuNrM9JStKScc5\nlw+sAKqZB5+QNgHLy21bDrT1oRbfmVlbvAuz/xHK/jHTFJR0/YFhS0CZYUshDXaQ+FfSEFwC9HLO\nrfO7nhiTAtTzu4goews4Ae/jg5NKvhYBU4GTXCxdOR1FJRdedsR7g0w284FO5bZ1wls5SUbD8T52\nfyWUnWPt44O/Ac+WTGJcCGRRathSsij5XLAjEPitp72ZnQRsc859619l0WVmE4FMoB+ww8wCq0j5\nzrmkGqltZg8CrwLrgEZ4FxGdjfcZatJwzu0AylxTYmY7gB+cc+V/W0xYZvZXYA7eG19r4H5gL5Dt\nZ10+eQyYb2Z34MXvfgVcC1zna1U+KPlFehjwrHOuOJRjxFRTUINhS8niFOBdvCvtHd69G8C7aGS4\nX0X54Hq8n/+9ctuvAZ6LejX+aoH3/z8DyAeWABck+dX3Acm4OnA43nj5Q4EtwAfA6SVj55OKc27R\n/2/vjgkQiIEACG68YQAl2EAKrpASij8FNPliRsGVW9wla61n9e46T/1Wr7335+xkRzy6PiT8e6/k\nVu8UAADn3GanAAA4SxQAAJUoAACGKAAAKlEAAAxRAABUogAAGKIAAKhEAQAwRAEAUIkCAGD8AHit\n1mA1gCvCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11bd31510>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ca = [a[x] for x in pthb[0]]\n",
    "cb = [b[x] for x in pthb[1]]\n",
    "plot(a, label='a')\n",
    "plot(b, label='b')\n",
    "plot(ca, '--', label='ca')\n",
    "plot(cb, '--', label='cb')\n",
    "legend(loc='best')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x119608650>"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgUAAAFkCAYAAACw3EhvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3Xl4VPXZ//H3nUCAICAKSEBQFkVcEdSqgErdKiCCCxDU\nBwQF0VaN+mtR69pWxT5WBVlstRUFUgXsIxD3lQoqkqCgoCCLyKKAaIIgS5Lv74+TgSxkmcnMnFk+\nr+vKdZkzc865MwJz5zvnc25zziEiIiKS4ncBIiIiEhvUFIiIiAigpkBERERKqCkQERERQE2BiIiI\nlFBTICIiIoCaAhERESmhpkBEREQANQUiIiJSQk2BiIiIALVsCsxsjJkVm9nfqnneOWaWa2a7zGyF\nmQ2tzXlFREQk/EJuCszsVGAk8Fk1zzsSmAu8DZwEPAE8bWbnh3puERERCb+QmgIzOwiYClwL/FTN\n00cDq51zv3fOfeWcmwDMBLJCObeIiIhERqgrBROAOc65d2rw3NOBt8ptex04I8Rzi4iISATUCXYH\nMxsMdAFOqeEuLYHvy237HmhsZvWcc7sPcI5DgQuBtcCuYGsUERFJYvWBI4HXnXM/BLNjUE2BmR0O\nPA6c55zbG8y+QboQmBbB44uIiCS6K4HpwewQ7EpBN6A5kGdmVrItFTjLzH4L1HPOuXL7fAccVm7b\nYUDBgVYJSqwFmDp1Kp07dw6yxMSSlZXFY4895ncZMUGvhUevgyfZX4fvv4f/+R9o0QKaNs1i3Ljk\nfS0Ckv3PBF9+CS+8wPIBA7jqmmug5L00GME2BW8BJ5Tb9iywHHj4AA0BwIfAReW2XVCyvTK7ADp3\n7kzXrl2DLDGxNGnSJOlfgwC9Fh69Dp5kfh22b4cRIyA9Hd5+G0aOTN7XorRk/jPB+vVw8cXQqhUc\ndVRga9AfvwfVFDjndgDLSm8zsx3AD8655SXfPwi0ds4F7kUwGbjRzMYC/wTOBS4HegdbrIhIsiss\nhMxMWL0a5s+Hli39rkh8t3079O0LqakwezZs2hTyocJxR8PyqwMZQJt9Dzq3FugDnAd8ihdFHOGc\nK59IEBGRatx6K7z2GsyYAccf73c14rvSXWJODmRk1OpwQacPynPO/brc99cc4Dnz8K5HEBGREI0b\nB+PHw+TJcMEFflcjMSHQJebkwAnlP90PXq2bAomszMxMv0uIGXotPHodPMn2OsydC1lZcPvtMGpU\n2ceS7bWoTNK9DoEucdIkuPDCsBzSDnxtoL/MrCuQm5ubm7wXjYiIlFi8GHr29FYHZs6EFI2yk+3b\noVMnGDIE/vd/yzyUl5dHt27dALo55/KCOWzcrhSsW7eOrVu3+l1G3GrWrBlt27b1uwwRqcb69d41\nZMceC1OnqiGQEo0awaJFcFj5xH/txGVTsG7dOjp37szOnTv9LiVupaens3z5cjUGIjFs+3YvZVan\njndReXq63xVJTGnVKuyHjMumYOvWrezcuVM3NwrR8uXLueqqq9i6dauaApEYpeih+CEum4IA3dxI\nRBJV4KLyV15R9FCiJ66bAhGRRKToofhFl6yIiMSQqqKHkqR+/DFqp1JTICISIxYvhsGD4ZJLYOxY\nv6uRmDBnDrRrB198EZXTqSkQEYkBih5KBYsXe1eb/vrXEKWL6vXHTkTEZ4oeSgWBLrFz56h2ibrQ\nUETER4oeSgWlu8Q5c6LaJaopEBHxkaKHUkagS1y1ChYsiHqXqKZARMQnih5KBaWnHvrQJeqaghiz\nbt06brjhBo455hjS09Np1qwZAwcO5JtvvvG7NBEJI0UPpQLnIDUVnnwybFMPg6WVghjzySef8NFH\nH5GZmcnhhx/O2rVrmThxIr169WLZsmXUr1/f7xJFpJYUPZQDMoPHHvO1BDUFMaZv375cdtllZbZd\nfPHFnH766cyaNYsrr7zSp8pEJBwUPZRYlhRNwc6d8OWXkT3HMceE5wLRevXq7fvvwsJCCgoKaN++\nPQcffDB5eXlqCkTimKKHEuuSoin48kvo1i2y58jNhXDMZtq1axcPPvggzz77LBs2bMA5B4CZkZ+f\nX/sTiIgvFD2UeJAUTcExx3hv2pE+Rzj89re/ZcqUKWRlZXH66afTpEkTzIxBgwZRXFwcnpOISNQp\neijxICmagvT08PwWHw2zZs1i2LBhPPLII/u27d69m59++snHqkSkNhQ9lAr+/W+47DKoW9fvSsrQ\nJS4xJjU1tcKKwLhx4ygqKvKpIhGpDUUPpYJx47zPknJy/K6kgqRYKYgnffv25fnnn6dx48Yce+yx\nfPjhh7z99ts0a9bM79JEJEiKHkoFc+Z4XeJtt0H//n5XU4Gaghgzbtw46tSpw/Tp09m1axc9evTg\nrbfe4sILL8TM/C5PRGpI0UOpIDD1sF+/mO0S1RTEmMaNG/P0009X2L569WofqhGRUCh6KBWUn3qY\nmup3RQekpkBEJIx8nmcjsWj7dq8hSE31usSGDf2uqFJqCkREwsjneTYSi0aP3n+DiowMv6upkj7l\nEhEJk0D0cMIE3+bZSCz64x/h//4PTjjB70qqFVRTYGbXm9lnZpZf8rXAzH5TxfPPNrPicl9FZtai\n9qWLiMQORQ+lUsccA7/+td9V1EiwHx98C/wBWAkYMAx42cy6OOeWV7KPA44Gtu/b4Nzm4EsVEYlN\nih5KogiqKXDOlb/Twh/NbDRwOlBZUwCwxTlXEGxxIiKxTtFDSSQh//E1sxQzGwykAx9W9VTgUzPb\naGZvmNmZoZ5TRCSWKHooiSbo9IGZHY/XBNTH+0hggHOussHEm4BRwCKgHnAd8J6Zneac+zS0kkVE\n/KfooSSiUCKJXwInAU2Ay4HnzOysAzUGzrkVwIpSmz4ysw5AFjC0uhNlZWXRpEmTMtsyMzPp1KlT\nCGWLiISPoodSRlERDBsGw4dDr15RO212djbZ2dlltuXn54d8vKCbAudcIRC4vd5iMzsNuBkYXcND\nLAS61+SJjz32GF0PMN4wLy+vhqcSEQm/0lMPFT0UwIueZGfDVVdF9bSZmZlkZmaW2ZaXl0e3bt1C\nOl44bl6UgvfRQE11wftYQUQk7ih6KBUkUJcY7H0KHjSznmZ2hJkdb2YPAWcDU0sef8jMppR6/s1m\n1s/MOpjZcWb2ONALeDKcP0Siue+++0hJSWHbtm1+lyIipSh6KBUEusTbbkuILjHYlYIWwBQgA8gH\nlgAXOOfeKXm8JdCm1PPTgEeBVsDOkuef65ybV5uiE52ZaSKiSIxR9FAqCHSJMTz1MFjB3qfg2moe\nv6bc938F/hpCXSIiMUPRQ6kgTqYeBksDkUREqqDooRzQ88/HxdTDYGkBLIZt2bKFgQMH0qRJE5o1\na8Ytt9zC7t27/S5LJKkEooczZih6KKWMGQOLFsX81MNgqSmIUc45Bg4cyJ49e3j44Yfp06cP48aN\nY1QCXMgiEi809VAqZQYtEm+2nz4+iGEdOnTgpZdeAmD06NE0atSISZMmcfvtt3O8fmURiShFDyUZ\nJUVTsHPvTr7cWtmdmMPjmGbHkF43fFcfmRk33nhjmW2/+93vmDhxIq+88oqaApEIUvRQklVSNAVf\nbv2Sbn8P7e5ONZU7MpeuGRXvvlgbHTt2LPN9hw4dSElJYe3atWE9j4jsp+ihJLOkaAqOaXYMuSNz\nI36OSNO9C0QiS9FDqWDrVmjaNGEih9VJiqYgvW562H+Lj4aVK1dyxBFH7Pv+66+/pri4mCOPPNK/\nokQSlKKHUsH27XDeeXD66d4tjJOAFsZilHOOCRMmlNk2btw4zIyLLrrIp6pEEpeih1JGYaF3Ycnq\n1VDu+q5ElhQrBfFqzZo1XHLJJfzmN79hwYIFTJs2jauuuooTTjjB79JEEkoCzbORcMnKgtdf92Zj\nJ9G/uVopiFEpKSm88MIL1KtXjzvuuINXX32Vm266iaefftrv0kQSiqKHUsG4cfDkk95XknWJWimI\nQffeey/33nsvAC+++KLP1YgkLkUPpYI5c/ZPPbz+er+riTqtFIhIUio/z0bRQ2HxYu9q0wSaehgs\n/TUQkaRTOno4Z46ih1IiNRXOPz+hph4GSx8fiEhSUfRQKnXiifCf//hdha/UFIhIUglED3NyFD0U\nKU9NgYgkDUUPRaqmawpEJCkoeihSPTUFIpLwFD0UqRk1BSKS0BQ9lAo+/RQ+/NDvKmKSrikQkYSl\n6KFUsH499OkDHTrA+++Dps+WoaZARBKSoodSwc8/e11iaiq88IIaggNQUyAiCUnRQymjqMi7sGTV\nKpg/HzIy/K4oJqkpEJGEo+ihVJCVtb9LTKKph8FSUyAiCUXRQ6lAXWKN6TpcEUkYih5KBYsW7Z96\nqC6xWlopEJGEoOihHFC3bvD88zBokN+VxAU1BSIS9xQ9lEqZwZAhflcRN4Lqpc3sejP7zMzyS74W\nmNlvqtnnHDPLNbNdZrbCzIbWruTksHHjRkaMGEHr1q2pX78+7du354YbbqCwsJAff/yR22+/nRNP\nPJFGjRrRpEkTevfuzZIlS/wuWyTqSkcPc3IUPRSpjWBXCr4F/gCsBAwYBrxsZl2cc8vLP9nMjgTm\nAhOBIcB5wNNmttE592boZSe2TZs2ceqpp1JQUMCoUaPo1KkTGzZsYObMmezcuZPVq1cze/Zsrrji\nCtq1a8f333/PU089xTnnnMOyZctoqX8VJYkoeigSPkE1Bc65nHKb/mhmo4HTgQpNATAaWO2c+33J\n91+ZWQ8gC4huU7Bpk/dVmfr14dhjqz7GsmWwa1fF7RkZYc28jhkzhs2bN7Nw4UJOPvnkfdvvu+8+\nAE488URWrFhRZp+rr76aTp068cwzz3DXXXeFrRaRWKaLykXCK+RrCswsBRgIpAOV3UT6dOCtctte\nBx4L9bwhe+opuP/+yh8/9lj44ouqj3HFFV5jUN6990LJG3ZtOed4+eWX6devX5mGoLS6devu++/i\n4mJ++ukn0tPT6dSpE3l5eWGpQyTWKXrocc5hujOfhEnQTYGZHY/XBNQHtgMDnHNfVvL0lsD35bZ9\nDzQ2s3rOud3Bnj9ko0ZBv36VP16/fvXHmDGj8pWCMNmyZQsFBQUcd9xxlT7HOcfjjz/OpEmTWLNm\nDUVFRQCYGc2aNQtbLbHip5+8NNEf/gBHH+13NRILvpi7hm0D/sRVv3mEsWMT7898TRUVF3H5jMu5\npNMlDOsyzO9yfPXmK6+Qs2oVj48cCfXqVfq8yz7/nB/27q308d+2bs3lLVpU+vjSn3/mdytXVlnL\nzOOOo1laWqWPj1+/nllbtlT6+AkHHcT4o46q8hxV/RwdN2yoct+qhLJS8CVwEtAEuBx4zszOqqIx\nCFlWVhZNmjQpsy0zM5NOnToFf7BwLPFX9/FClPzlL3/hnnvu4dprr+XPf/4zhxxyCCkpKdx8880U\nFxf7XV7Y/eMf8M9/wrx53mCzBOx7JAjr18PiQQ/xP4X/4sqfVpKy960q3wQSWdbrWcz5ag6jTxnt\ndym+WrxoEQPMuKBBAyi1knogGWlpNKgir3pQamqV+6elpHB4NX/eUqtZuTm4Tp0qj9G8mp8B9v8c\na3NyWPvKK2Ue+3bnzmr3r5RzrlZfeNcGTKrksfeBv5XbNgz4sZpjdgVcbm6uO5Dc3FxX1ePxrLi4\n2DVp0sQNGDCg0ud06dLFnXvuuRW2H3744a5Xr17VniOeXr+9e51r29a5Cy90rnlz53r0cG7XLr+r\nEr8UFDh31vE/uJ3WwO28+Arn6tVz7qqrnCsu9ru0qHvioycc9+EmfzLZ71J89e2aNa7VrFnu1ClT\n3I6CAr/LiQmBf+OBri7I9/Rw3N4jBais5fkQOLfctguo/BqEpGdm9O/fnzlz5lR6fUBqamqgedpn\nxowZbKjFklGsmjMH1q2Dv/wFXn4ZPvkEhg+Hcj++JIFA9LDHyn9Sr04RDZ5+Ep57zrtT0QMP+F1e\nVM1dMZes17O4/YzbGXVK8l5Q8fNPP3HxBx9Qp7iY2b16kd6okd8lxb2gPj4wsweBV4F1QCPgSuBs\nvDd6zOwhoJVzLnAvgsnAjWY2FvgnXoNwOdA7LNUnqAcffJA333yTs846i5EjR9K5c2c2btzIzJkz\nmT9/Pn379uWBBx5g+PDhnHnmmSxdupRp06bRoUMHv0sPu3Hj4MwzvZuSgfceMGgQHHVU2K7tlDgR\niB5mvXAtKQ2PhxYtYOBA+Ppr2LLF6xST4IK7xZsWM3jmYC7pdAljz0/eezkX7d3L4FmzWJWRwYKM\nDFq2aeN3SQkh2GsKWgBTgAwgH1gCXOCce6fk8ZbAvv8zzrm1ZtYHL21wE7AeGOGcK59IkFJatWrF\nxx9/zN1338306dMpKCigdevW9O7dm/T0dO6880527tzJ9OnTefHFF+nWrRuvvPIKY8aMSairkJcu\nhffeg3//e/+2gQO9m9TceSd06ABXX+1beRJFpaOH5152MFDqnml33JEUzQDA+oL19M3uy7HNj2Xq\npVNJseS9l3PWlCm81q4dOUVFHF9JUkuCF+x9Cq6t5vFrDrBtHtAtyLqS3uGHH86//vWvSh9/5JFH\neOSRR8pse+eddyp5dnwaPx5atYJLLy27fcwY75fDESPgiCPgrLP8qU+io9roYZI0BD/v+ZmLsy+m\nTkodZmfOJr1u8t7LedW33/J0mzZM2LKFCwcP9ruchKLZBxKTtm3zPiq+886KFxObwaRJsHYtDBjg\nJRIUVUxMmnroKSouYvDMwaz+cTXzh8+n5UHJfdfSDm3a8FVaGm0OO8zvUhJO8q49SUx75hkoKoKR\nIw/8eFoazJzpfazcpw9s3Rrd+iTyNPVwv6zXs3jt69eYccUMjm+hezkDaggiJIn/mkmsKiyEJ5/0\nrjSv4h4iNG3q3e8+P99bMdgdvVthSYRp6uF+4z4ex/iF45nQewIXdLjA73IkwakpkJgTiCH+7nfV\nP7d9e0UVE03Yph7u3ev9YYpjih5KtKkpkJhTPoZYnTPO8KKK06dXPd5C4kMgejhjRsnUw6IiWLQo\n+AO9+KJ3a/OpU8NeYzQoeih+UFMgMSUQQ7zppuD2GzgQHnzQawri9D1A2B89nDCh1NTDOXPg1FOr\nH1hW3pAhcM013hLSvHlhrzWSFD3c76dvvvG7hKSSvH/SJCZVFkOsiTFjvH//4/A9QKgiehhYOqpi\nSNgBmXk3NujRw7vopNy48Vil6OF+46ZN47hFi/hhzRq/S0kaagokZgRiiKNHVzvT5IACUcWePePq\nPUCoInq4dCm8+27wS0cBaWkwa1bcxFRKRw9zhuQkdfRwzquvktWyJZk//cSh7dr5XU7SUFMgMaO6\nGGJNKKoYf6qMHtZm6SggjmIqih56Fi9aRCbQb+1axg4b5nc5SUVNgcSEmsYQayKO3gOSXpXRwx9+\n8LqEG24IbemotNIxlVtuqd2xIkTRQ8/6tWvpu24dnbduZerAgaRWM8pYwktNgcSEYGKINaGoYuyr\nNnoYWDq67rrwnPCMM7xEws03h+d4YaTooWd7fj59588n1Tlmn3MODTX1MOrUFMShKVOmkJKSUulo\n5XgUbAyxJhRVjG0VooelFRZ6EYRwLB2V1q8fHHNM+I4XBooeegoLC8mcOZPVTZuS0749GZp66AvN\nPohTiT4NMVxKT1Xs2BGuuir855DglZ56uC96WNrOnd5Vh9dUmLGWUBQ93G9vcTENGjRgRp06nKCp\nh75RUyC+C8e1ZFUJTFUcPhzattVURb9VO/UQoHFjr3NIYIoeltUgLY0ZQ4b4XUbSS962VGJCbWOI\nNaGoYuzQ1EOPoocSq9QUxKiNGzcyYsQIWrduTf369Wnfvj033HADhYWF+56zY8cORo0aRbNmzWjS\npAlDhw7lp59+8rHq4IUjhlgTiir6T1MP91P0UGKVPj6IQZs2beLUU0+loKCAUaNG0alTJzZs2MDM\nmTPZuXMnAM45fvvb39K0aVPuv/9+vvrqKyZOnMi6det49913ff4JaiacMcSaCEQVTz/dWzF46y2o\nVy/y55U4mHr47rveTY7Gj/eWliIoED2c3GdyUkcPJTapKYhBY8aMYfPmzSxcuJCTS11wc99995V5\nXv369Xn77bf35Xjbtm3LH/7wB+bOnUvfvn2jWXJIwh1DrIlAVLFXLxgxAp5/PuLvAUmvdPRwwYJa\nTD2MpM2bvbRD8+Zw770RO42ihxLrkqYp2LR7N5v27Kn08fopKRzbsGGVx1i2Ywe7iosrbM9ISyMj\nTL9yOud4+eWX6devX5mG4EBGjhxZ5sYeo0eP5s477+SVV16Ji6YgEjHEmghEFQcN8hIJ5XotCbNA\n9DAn5wDRw1gxaJDXtdx1F3ToEJGYiqKHnqLCQmZMn86gQYMwLdXFnKRpCp7auJH7q5i2dWx6Ol+c\ndlqVx7jiiy9YVrJ8X9q9RxzBfWG6N/eWLVsoKCjguGqGv5gZHTt2LLOtYcOGZGRksHbt2rDUEkmR\njCHWhKKK0VFt9LC04mJ/LzS44w4vpjJiRNhjKooe7pf17LNMbNeOYxct4sTu3f0uR8pJmqZgVKtW\n9GvWrNLH69fgH6MZxx1X6UqBBCfSMcSaUFQxsmoUPSztoou8iMgf/xjx2g4oMFVx7VrvopOPPoKj\njqr1YRU93G/ctGmM79iRyd99x4mDB/tdjhxA0jQFGfXq1XqJv7qPF8KhefPmNG7cmM8//7zK5znn\nWLlyJWefffa+bTt27GDTpk306dMn0mXWSiCGeOedkYsh1kQgqhh4D/jwQzj6aP/qSSRBRw+XLoU3\n3vA6ND8FpiqeeSb07u01BoceGvLhAtHDVdtWsWDEgqSOHs4tmXp426pVjBoxwu9ypBLJu4YVo8yM\n/v37M2fOnGpvY/z3v/+9TERx4sSJFBUV0bt370iXWSvRiiHWhKKK4RdS9DAWlo4CSk/UevbZWh1K\n0UPP4kWLGAz0W7OGsUOH+l2OVCFpVgriyYMPPsibb77JWWedxciRI+ncuTMbN25k5syZzJ8/f9/z\n9uzZw7nnnsvAgQP58ssvmTRpEj179ozpiwyjHUOsCUUVwyek6GFg6eiuu/xdOiqtfXvIy4PWrUM+\nROno4YUdq7ugInHtm3q4Y4c39bCO3nZimf7vxKBWrVrx8ccfc/fddzN9+nQKCgpo3bo1vXv3Jr3k\nX1kz48knn2TatGnce++97N27lyuvvJInnnjC5+qr5kcMsSYUVay9kKOHzzzjXWQYC0tHpR1+eMi7\nKnrocc5x+QcfkNqggTf1sHFjv0uSaqgpiFGHH344//rXvw742NChQxlasgTXo0cPJk2aFM3SasWv\nGGJNKKpYOyFFDwNLR4MHe/cISACKHu5nZjxy4ok0LS7W1MM4oaZAosbvGGJNKKoYmqCih6XF6tJR\niBQ9rOisE0/0uwQJQlB/Ys3sDjNbaGYFZva9mf3HzKq8XtvMzjaz4nJfRWYWI58oS7TE0rVkVRkz\nxrsIfvhwmDfP72piX9DRw9Ly8qBHj9hcOgqSooeSCIJdKegJjAcWlez7EPCGmXV2zv1SxX4OOBrY\nvm+Dc5uDPLfEsViJIdZE+ahimOLqCanWUw//9CfYvTvsdUVUURHs2OGNdw5sUvRQEkRQKwXOud7O\nueedc8udc0uBYUBboCZt/hbn3ObAVwi1ShyLpRhiTZSOKvbuDT/84HdFsSdsUw/jLepx3XXeD16q\nmVH0UBJFbT/wOhhvFWBbNc8z4FMz22hmb5jZmbU8r8SRWIwh1kTpuHr//vH3C20kxfzUw0i67jpY\nuBCuvRac2xc9nNB7QlJHD3HO7wokDEJuCszMgMeBD5xzy6p46iZgFHAZcCnwLfCemXUJ9dwSX+L5\nWrJAVPGTT7yoov7dKxs9zMmJ0amHkXTGGTBlCkydylc3X6XoIbA9P58BTz3F8vfe87sUqaXapA8m\nAscCVU60cM6tAFaU2vSRmXUAsgDd2ioJxHIMsSYUVSwrLqYeRtqgQWz89L90engCj97YjZuSOHpY\nWFhI5syZzGvblsImTfwuR2oppKbAzJ4EegM9nXObQjjEQqppJgCysrJoUu4PWWZmJp06dQrhlOKH\neIgh1oSiip6Qo4cJZn3Ben7V7CUmndmMm/+xFBv4QdJO1Lr12Wd5rV07coqKOKGace8SftnZ2WRn\nZ5fZlp+fH/Lxgm4KShqCS4CznXPrQjxvF7yPFar02GOP0bVr1wrbAzMBli9fHuLpk1s0X7d4iSHW\nRGCqYgQm68aFWkUPE8j23du96GFqXU6b8xF2+bCkjakEph5O+u47LtTUQ19kZmaSmZlZZlteXh7d\nQlyaDaopMLOJQCbQD9hhZoeVPJTvnNtV8pwHgdbOuaEl398MrAG+AOoD1wG9gPNDqhho1qwZ6enp\nXJWsv66FQXp6Os2qGCUdDvEUQ6yJZI4q1jp6GPD5596y0ZgxcNBBYasvWgqLC8mclbk/enhIW2+q\n4qBBsGuX3+VFVemph9dr6mHCCHal4Hq8tMF75bZfAzxX8t8ZQOn7WaYBjwKtgJ3AEuBc51zIt4Vp\n27Yty5cvZ6tG2oWsWbNmtG3bNqLniLcYYk0EoophmqwbFwLRw2OPrWX0ELzPH3Jy4N57w1ZfNN36\n+q289vVr5AzJ2R89bNrUG/ucRPZNPVy7lrHDhvldjoRRUE2Bc67afw6cc9eU+/6vwF+DrKtabdu2\njfibmoQuXmOINVF6qmL//ok9VbF09HD27FpGD2NxGmIQNPVwvydWrOCYoiJv6mFqqt/lSBhp9oFE\nRDzHEGsiGaYqhjz1sDKBpaPrrgtLfdGkqYdl/WPQIAq2b6dho0Z+lyJhpmkdEhHxHkOsiUBUcdo0\nuP9+v6sJv0D0cMaMMEQP43jpSFMPK6qbmsqhBx/sdxkSAVopkLBLlBhiTSRqVDHs0cM4XTrS1ENJ\nNmoKJOwSKYZYE4kWVYxI9HD8+LhbOtoXPazN1MOPPoJDDoGjqxwmKxIz1PZKWAWuJRs9Oi6vJQtJ\nIKrYvbsXVVy50u+KQhe26GFpy5bBu+/CTTeF6YCRVzp6mDMkJ7Sph8XFcOON0KcPKCklcUJNgYRV\nIsYQayIJ70OXAAAgAElEQVQtzYurx/NUxbBGD0vr3NmLaMTR0lEgelirqYcpKd4FGfn5XrcYhxO1\ntqxahdu71+8yJIrUFEjYxPG1ZGERz1MVwxo9LM8Mzj03bpaOwjr1MI4naq1fu5Yun33GY08/7Xcp\nEkVqCiRs4vRasrAq/R5QMlk35iX91MNSIhI9jMOYys8//cTFH3xAqnNk9u3rdzkSRbrQUMImGWKI\nNVF+qmKs37xPUw89EY0eDhzoXY16110xH1Mp2ruXwbNmsSojg/kZGWS0aVP9TpIw1BRIWCRTDLEm\nSkcVO3SI3fcATT30BKKHnZt3jlz08I479sdUTj4Zjjsu/OcIg6wpUzT1MImpKZCwSLYYYk3EelRR\nUw89paOHczLnhBY9rAkzr/vq0cO7mjMGBaYeTtbUw6Slawqk1pIxhlgTsRxVjEj0MA6FJXoYjLQ0\nGD48Ju+JXXrq4Sg1BElLTYHUWrLGEGsiFqOKEYseljZ/PvzySwQOHF5hiR4miLotWnDVt98yduhQ\nv0sRH6kpkFpJ9hhiTcRSVDGi0cOAbdvg/PO9CxZiWFijhwngwm7dmDJsGKl19KlyMlNTILWiGGLN\nxEJUMWrRw8DS0TXXVP9cn2jqociBqSmQWlEMseYCUcWpU+GBB6J//rBOPaxMHCwdaeqhSOXUFEjI\nAjHEOLqlve8GDoQHH4T77vOag2gJRA8nTIhw9DDGl46iEj0Mxa5dMHQozJvndyWS5GLkb4TEI8UQ\nQzNmjHcB+ogR0XkPiGr0MIaXjqIWPQxFSgp8+60XU1mxwu9qJImpKZCQKIYYumhGFaMaPYzhpaOo\nRw+DVTqmEuGpim++9hqrFiyI2PElvqkpkJAohlg70YgqRiV6WFoMLx3FRfSwdEwlQlMVFy9axADn\n+MuSJWE/tiQGNQUStDi4liwuRPI9ICrRw/K6d4c//Snmlo7iKnpYOqYyfHhYYyrr166l77p1dN66\nlfFXXhm240piUVMgQYvxa8niSuA9YOHC8EUVfZt6OHSo90YWQ+IyenjGGTBlCkyfHrapitt/+om+\n8+eT6hyze/WiYaNGYTmuJB41BRK0GL6WLC6FO6oYlehhHIjr6OGgQfCXv3izErZtq9WhCvfuZfBL\nL7G6aVNyOnQg4/DDw1SkJCLdukqCommIkVF6sm5tpipq6qEnZqOHwbjjDm/56JBDanWYrOee4/Uj\nj/SmHnbpEqbiJFGpKZCgxPC1ZHGv9GTdI46Anj2D219TDz0xHT0MhlmtL9p54Y03eLJDB009lBqL\nw/ZZ/KIYYmQFJut27+7NSAgmqqiph56Yjx5GWf9zzyW7sFBTD6XG1BRIjSmGGHmhRBWjHj2MYXER\nPYyieqmpDD7vPL/LkDiSxP98SDAUQ4yeYKKKvkQPA/bu9WeyUyXiKnooEqOCagrM7A4zW2hmBWb2\nvZn9x8yOrsF+55hZrpntMrMVZqaB3XFGMcToqklU0bfoYcDjj8Npp3nLRz6Ly+hhbWze7HcFkqCC\nXSnoCYwHfgWcB9QF3jCzBpXtYGZHAnOBt4GTgCeAp83s/BDqFZ8ohhh91UUVfY0eBpaOjjsOUlOj\nfPKy4jp6GIqVK72ISjQnaknSCCp94JzrXfp7MxsGbAa6AR9UsttoYLVz7vcl339lZj2ALODNoKoV\nXyiG6J/Kooq+Rw9jZOkoIaKHwerYEa64wouptG0LZ53lbXfOu1pVpBZqG0k8GHBAVXfXOB14q9y2\n14HHanluiRLFEP1VPqqYnx8D0cMYWDpKmOhhsAIxlW++8S46+egjxi9cyIbvv+ehm27C6ihpLqEL\n+U+PmRnwOPCBc25ZFU9tCXxfbtv3QGMzq+ecC//UDwmbQAzxzjsVQ/RL4D1g7Vovqrh7t8/RwyVL\nfF862ltUyG9mZ/FVYQOeu/TlaqOH8/Pz2VtcXOnjR6Wn07pevUof/3HvXj77+ecqz9G9SRPqVhH9\nWLFzJxuruGr04Dp16FLN7YfL/BzPPgs33sjK++/nlmuuIWvXLjUEUmu1+RM0ETgW6B6mWirIysqi\nSZMmZbZlZmaSmZkZqVNKOYohxoZAVLFHDzjoIJ+jh08+6fvS0SnvPsuS5ldBc5iwDS5vW/XzL1m6\nlB8KCyt9fMJRR3FD69aVPv7pzz/z688+q/IcW848k2ZpaZU+/vj69UzauLHSx885+GDereaOgxV+\njltvBeDSr79m7LBhVe4riSk7O5vs7Owy2/Lz80M+nrkQIkVm9iRwMdDTObeumue+D+Q6524ttW0Y\n8Jhzrmkl+3QFcnNzc+natWvQ9Ul4FBZ6n2P36uX9UiL+273bawZ8W7X54Qdo08a7yOGuu3wp4bL5\n03hpb2suT9vMQ1360iA1tcrf8gFW//ILxVX8W9c8LY0mVfyWvbOoqMrf8gHaNWhAahWf6W/es4eC\nKhqTUH8O27WL9gcfjCXzDSqkjLy8PLp5H+11c87lBbNv0CsFJQ3BJcDZ1TUEJT4ELiq37YKS7RLD\nYuRaMimlmveMyPv8c+9GCj4tHd2z5FVe2tOSU4pXM+PMmk9kbN+g0oBUjaSnptKxljeBaJGWRosq\nVhJq4oA/R1RvTiGJLtj7FEwErgSGADvM7LCSr/qlnvOgmU0ptdtkoL2ZjTWzTmZ2A3A58Lcw1C8R\nFAPXkkmsOftsr1Ns3jzqp85es4g/bYGM3Wv4sJdudSISCcGuFFyPlzZ4r9z2a4DnSv47A2gTeMA5\nt9bM+uClDW4C1gMjnHPlEwkSQxRDlEr5cF+Cb/O/ZeiyT0lPrceScwZRJ8XfeyOIJKpg71NQ7cqC\nc+6aA2ybh3cvA4kTiiFKrNi+ezv9/t2P5nuLefXKHJrVr/oKfREJnfIrUkEghnjXXYohir9KTz1c\nMGIBxzc93O+SRBKamgKpIBBDvO46vyuRZBeYepgzJEdTD0WiQE2BlKFpiBIrAlMPJ/eZrKmHIlGi\nYKuUoRiixIKkm3ooEiPUFEgZiiFKGYWF3uzmpUujdsqFG/OSa+qhSAxRUyD7BGKIN93kdyUSM+bM\n8S4y2bMnKqf7ZOtazlyyglZHXJI8Uw9FYoj+xsk+iiFKBVFcOtq08yfOXvQBpNRlxkVjk2fqoUgM\n0YWGAiiGKAcQxTtY7S7aS5d5s/ilTgazjm7FSYcoeijiBzUFAiiGKAcQxaWj096dwuZ67fhz8yIu\nbVv1pEARiRx9fCCKIUpFgaWj0aMjvnR02fxpLKnTkSH1t3DX8RdE9FwiUjU1BaIYolQUWDqK8DTE\nfVMPi1Yz7YzBET2XiFRPTYEohihlOQdPPRXxpaPFmxbz8OIXydi9hgWaeigSE3RNQZLTNESpwAze\nfReKiyN2ivUF6+mb3ZeTGrXinV8/SV1NPRSJCWoKkpxiiHJAbdpU/5wQbd+9nYuzL6ZOSh3mZM6h\nUVrDiJ1LRIKjjw+SWOBashtuUAxRoqP01MOcITm0PKil3yWJSClaKUhiiiFKtGnqoUhsU1OQpBRD\nlGjT1EOR2KePD5KUYogSTX/7/BVueeP3mnooEuO0UpCkFEOUaMles4jbvjc6nfwAY8+/3e9yRKQK\nagqSkGKIUsFnn8FBB0GHDmE97Cdb13LVynWkF+/gv+eN1tRDkRinpiAJKYYoFdxyC+zdCx98ELZD\nbiyZemgpDVhwWi+a128UtmOLSGSoKUgymoYoFURg6WhX0V5O1tRDkbijpiDJKIYoFURg6ehXmnoo\nEpf0AV8SUQxRKojANMRB86dq6qFInFJTkEQUQ5QKwjwNce6Kucz44PecU7hMUw9F4pCagiSiGKKU\nEealo8WbFjN45mD6H3k6b597fRgKFJFo0zUFSUIxRKkgjEtHgamHnZt3ZuqlUxU9FIlT+pubJBRD\nlArat4e776710lH5qYfpddPDVKCIRFvQTYGZ9TSz2Wa2wcyKzaxfNc8/u+R5pb+KzEyXukWJpiHK\nAZ10EjzwQK0OoamHIokllJWChsCnwA2Aq+E+DjgKaFnyleGc2xzCuSUEiiFKpASmHs64YoamHook\ngKCvKXDOvQa8BmBmFsSuW5xzBcGeT2pHMUSJlIHzpzLj60+Y3HuCph6KJIhoXVNgwKdmttHM3jCz\nM6N03qSnGKJEwj1LXmXGngxOOfZaTT0USSDRaAo2AaOAy4BLgW+B98xMtzmLAsUQJdyy1yziT1sg\nY/daFvQa5nc5IhJGEY8kOudWACtKbfrIzDoAWcDQqvbNysqiSZMmZbZlZmaSmZkZ9joTkWKIEm6l\npx5+ds5A6qak+l2SSFLLzs4mOzu7zLb8/PyQj2fO1fRawQPsbFYM9HfOzQ5yv0eA7s657pU83hXI\nzc3NpWvXriHXl+xGjoScHFi7VqkDKbFzJ6SHFhnc9Es+Hd6fw56UBuSe8isNORKJUXl5eXTzloe7\nOefygtnXr/sUdMH7WEEiRDFEqWDpUmjZEhYvDnrX3UWFnPT+LH6pcwgvHtNBDYFIggr64wMzawh0\nxLt4EKC9mZ0EbHPOfWtmDwGtnHNDS55/M7AG+AKoD1wH9ALOD0P9UgnFEKWC8eOhcWM4Pvjo4KVv\n/S9b6p2qqYciCS6UlYJTgMVALt79Bx4F8oD7Sx5vCbQp9fy0kucsAd4DTgDOdc69F1LFUi3FEKWC\nWkxDHPfxOF756A7uabhOUw9FElwo9yl4nyqaCefcNeW+/yvw1+BLk1AphigVPPMMFBcHPQ1x7oq5\nZL2exe1n3M79p11T/Q4iEtc0+yABjRsH3bsrhiglAktHgwdD8+Y13i0w9fCSTpcw9vyxESxQRGKF\npiQmGMUQpYIQlo4CUw+PbX6sph6KJBH9TU8wmoYoFQS5dPTznp/3TT2cnTlbUw9FkoiaggSiGKJU\nsGOHd2+CGq4S7CkqZODMIZp6KJKk9PFBAlEMUSpo2BA+/hhqeJOyU999liX1z+KVy2/U1EORJKSm\nIEEohihVqsFA08vmT2NJnY4MyfiOi47S1EORZKSPDxKEYohSG/cseZWX9rTklKLVTDtjsN/liIhP\n1BQkCMUQJVRlpx5WOaNMRBKcPj5IAIohSqg09VBEStNKQQJQDFFC8d0v+Zy96AMMx4LTetG8fiO/\nSxIRn6kpiHOKIUooioqLGDJ7NLsLd2nqoYjso6YgzimGKBU88ABMmVLlU7Jez2LeVy+S06mNph6K\nyD5qCuKYYohSwbZt8PDD8O23lT5l3MfjGL9wPBN6T+A3ih6KSClqCuKYYohSQWDpqJJpiKWnHo46\nZVSUixORWKemII4phihlVLN0pKmHIlIdRRLjlGKIUkEVS0eaeigiNaF/GeKUYohSwfjxB1w62vRL\nPr3+7yZNPRSRammlIA4FYoh33aUYopRYuhTefbfC0tHuokK6vD+TLRnD+PD8+zT1UESqpKYgDimG\nKBX8618HXDo67d1n2VyvHX9uXsSvMk70qTgRiRf6+CDOKIYoB/Tww/Dmm2WWjvZNPay/hbuOv8DH\n4kQkXqgpiDOKIcoBpaXBscfu+1ZTD0UkFGoK4oxiiFIdTT0UkVDpmoI4ohiiVGfptm809VBEQqaV\ngjiiGKJU5ec9P3P1jP40+X6Oph6KSEjUFMQJTUOUqhQVFzF45mBWb1vFvAtv09RDEQmJPj6IE4oh\nSlWyXs/ita9fI2dIDse3ON7vckQkTmmlIA4ohigVbNvmRVGKispMPbywo6Yeikjo1BTEAcUQpYJn\nnoHLL+eNhdmaeigiYRN0U2BmPc1stpltMLNiM+tXg33OMbNcM9tlZivMTDmpICiGKGWULB390P9C\nLn33ek09FJGwCWWloCHwKXAD4Kp7spkdCcwF3gZOAp4Anjaz80M4d9IJxBC1SiD7zJnD+p07Oan3\nSXRq0UVTD0UkbIK+0NA59xrwGoCZWQ12GQ2sds79vuT7r8ysB5AFvBns+ZONYohS3o+TJnDuo2P5\nrmVjXjl1lKYeikjYROPXi9OBt8ptex04IwrnjmuKIUp5uz9dzNW9zmVNy5bMOKYjJzZV9FBEwica\nkcSWwPfltn0PNDazes653VGoIS6NH68YIkDu5x9z63vz2JHWYN+249auJOulKQD8v9tOZG/div3t\n0hZDKEpJ47AfdtFoR2GFx4e+8R96fp7LFx0a848r2ld4vCCtNasP8T7l6vDtz9gBPiwbN+HPpO/e\nxbQ+bfnkhEMqPP5t4zP5If1oGv5SSMaWXRUeD/bnqLO3Ll+ddhoPHLqXAW27VCxIRKQWYvo+BcNu\nGMWRLTLKbMvMzCQzM9OniqJn7lx44AH4wx+SO4b43bbvuDr3c37MaMeJ61fu255uhew4rCkArRq3\npqhuxdv5rk8ppsgKOWTvThr/coDes2EaOw5rijVryuGNK/7G/WPqIfxoXjNx6C/bD9gU/NK8CW5v\nAxo3bcnhjSv+j9pVrwHFVkjD4t0c+sv2Co8H/XPk7+ZqK2DMuVkVixGRpJOdnU12dnaZbfn5+SEf\nz5yr9lrBync2Kwb6O+dmV/Gc94Fc59ytpbYNAx5zzjWtZJ+uQG6d8Q+x7OprOKrJYSHXGI8WL4ae\nPeGCC2DmTEhJ0mvI9hQV0ubNKfxkLZlZby8Xn9Pf75JERGJeXl4e3by4WjfnXF4w+0bj7eZD4Nxy\n2y4o2V6lwpQ0TvnwVQr2/BKRwmLR+vXQty907uxdT5CsDQHAeW9PYHO9I7mndV01BCIiURDKfQoa\nmtlJZhb4QLN9yfdtSh5/yMymlNplcslzxppZJzO7Abgc+Ft157qr1cEUpLXipHenUVxcHGypcWf7\ndrj4YqhTx7thUXoSX1Q+7uNx/Pej/8dNDTZy1/EX+F2OiEhSCOX30FOAxUAu3n0KHgXygPtLHm8J\ntAk82Tm3FugDnId3f4MsYIRzrnwioYJL23bhlia/sLZeR34977kQSo0fhYXebYxXrYKcHGjZ0u+K\n/DN3xVyyXs/ittNv4onTr/a7HBGRpBHKfQrep4pmwjl3zQG2zQNCuh/fY90uYdn7U3jDHclNi2Yx\n7pTLQjlMzLv1VnjtNa8hOD6J59ks3rSYwTMH069TP8aep7v0iYhEU1x8Yv1qz6s59ZdcJr05innf\nzPO7nLAbN86LH06YABcm8Tyb9QXr6Zvdl87NOzN1wFRSUypeiS8iIpETF01BSkoKH1zwO3q2OpEB\nLwxg5Q8rq98pTsydC1lZcNttMCqJ59n8vO073rq8GwftMWYPnk3DtIZ+lyQiknTioikASEtNY9bA\nWbRo2ILe03vzw84f/C6p1hYvhsGD4ZJL4JFH/K7GP4V7drPs3JO4dN5m5nSfQEajjOp3EhGRsIub\npgCgaYOm5AzJIX9XPv1f6M/uwvi9GaKih/sNmvAwOSf9hhWT/szRPS/xuxwRkaQVd29F7Zu25+XB\nL/PJhk8YMXsEtbn5kl8UPdzvpvH389LJZ7PjtI6ccs1dfpcjIpLU4q4pADijzRk8N+A5pi2dxv3v\n31/9DjFE0cP9/jb1CSYc24OBi9/jf2+42+9yRESSXkzPPqjKwOMGsmrbKu5c9l++bziLSafGR1RR\n0UPPS2/O4J5Dj6bnslym3niH3+WIiAhxulIQMKbHGI5qdymTtzdi/Jfv+11OtRQ99HyyZQ3X/2wc\nuelbZgz5H+qm1fO7JBERIc6bAjMjr9dVHLxnAzevL+DNjcv9LqlSih56vvsln7Nz55PfAKafeQrN\nD03iz09ERGJMXDcFAAfVrU9e977ULfqZPkuXsDL/e79LqkDRQ09RcREDcv7ArpSDmN65Ayce09Xv\nkkREpJS4bwoA2jVqzusndaMwpX7MTVVU9HC/rNez+GTZ0/znyAZcdsTJfpcjIiLlJMxb1Dktj2by\nkc1iaqqioof7jft4HOMXjmdC7wlccnQSX1AhIhLDEqYpABh5VHdubvILa9Pacd28x32tRdHD/fZN\nPTzjNkadksQXVIiIxLiEagoAHu92CbemLuGf79/G1CVTfasjED2cMSO5o4eL1y3U1EMRkTiRcE0B\nwP/2vIXhXYYz/OXhvkxVVPTQs2n5Jxzc7UyGbW6lqYciInEgIZsCM2NS30n0aNsj6lMVFT30bNq8\ngZWDBlC3CO7+7QxNPRQRiQMJ2RSAP1MVFT307N69iytm/JtRt/2Z7S+9yGEdT/K7JBERqYGEbQog\nulMVFT3cb8jEsXx0TBduqLeDzudc6nc5IiJSQwn/1lV6quLQ2SMjElVU9HC/wNTDm7/6kN8NvtHv\nckREJAgJ3xSAN1Xxmf7P8ULd7vx63nNhPbaih/s9um/q4fs8esMf/S5HRESClBRNAcCVxw/kgsb1\neJ8juf6Tl8J2XEUPPdlrFnFni6PpsTyP53+rqYciIvEoaZoCgFd7Xs1Re1byVJimKip66Plk61qu\nWrmOOsVbmTHsWtLqpvldkoiIhCCpmoKUlBTyel3NwbvXc/P6At7YuCzkYyl66Pl5z8+MeGkQDQo+\nY/5p59DioKZ+lyQiIiFKqqYASqYq9vCmKvZdupQVIUxVVPTQU1RcxOCZg1n7w3I+7Hk5XQ5p43dJ\nIiJSC0nXFEDZqYqnBjlVUdHD/bJez+K1r19jxhUzOOGwE/wuR0REailp39ICUxW3797O8Jzf4Zyr\ndp9A9DA1FWbPTu7oYWDq4ZO9n+TCjkl8QYWISAJJ2qYAvKmK2R0OY9aSZ7j//furfG756GFGRpSK\njEH/nfUYY+bewm1n3Mb1p1zvdzkiIhImITUFZnajma0xs1/M7CMzO7WK555tZsXlvorMrEXoZYfP\noOMH8pdf/4X737+/yqmKpaOHJyTxSvmXb2Rz8pBbeXbZ0Zp6KCKSYOoEu4OZDQIeBUYCC4Es4HUz\nO9o5t7WS3RxwNLB93wbnNgdfbmTc0eMOvt72NcNfHk7bJm0564izyjweiB5Onpzc0cPczz/m1Wdn\nMiAjnT5Pz9PUQxGRBBPKSkEW8JRz7jnn3JfA9cBOYHg1+21xzm0OfIVw3ogxMyb3nXzAqYqKHnq+\n27yBq3M/Z9KAQdR9/b80bBoTCz0iIhJGQTUFZlYX6Aa8HdjmvCv03gLOqGpX4FMz22hmb5jZmaEU\nG0kHmqqo6KFnz+5dXD7j36xv1pLJzdM4ulNXv0sSEZEICHaloBmQCpQP938PVHbX/03AKOAy4FLg\nW+A9M+sS5LkjrvRUxTNn3U7vS39R9BDILJl6+Jfta7n4nP5+lyMiIhES9DUFwXLOrQBWlNr0kZl1\nwPsYYmikzx+s9k3bM77P/zF403ZS75rKootGkJ6evB3B7yb+iZdOPptbv/gvv7vxbr/LERGRCAq2\nKdgKFAGHldt+GPBdEMdZCHSv7klZWVk0adKkzLbMzEwyMzODOFVwCgvh+d+fSd3DXmbv1Udx5cop\nvNf6moidL5bds+RVnux8Jv2XL+BRNQQiIjEnOzub7OzsMtvy8/NDPp7V5KY9ZXYw+wj42Dl3c8n3\nBqwDxjnn/lrDY7wBFDjnLq/k8a5Abm5uLl27Rvfz65tugokTvXsRPFp/Cm+6I7i+4TYmnXppVOvw\n23/X53HW8g1kFH3P2vOHkZYa8UUlEREJg7y8PLp16wbQzTmXF8y+oayL/w24zsz+x8yOASYD6cCz\nAGb2kJlNCTzZzG42s35m1sHMjjOzx4FewJMhnDuiyk89fK3n1Ry152smh2mqYrxYX7CewS9ezDHf\nTeHTsweqIRARSRJBNwXOuReB24EHgMXAicCFzrktJU9pCZSejJOGd1+DJcB7wAnAuc6590KuOgIO\nFD30pipexcF7aj9VMV78vOdnLs6+mFRL5Z3+42nRoLHfJYmISJSE9Cugc24iMLGSx64p9/1fgRp9\nrOCXqqKHB9WtT173vhyz4C36Ll3KmsbNaX1Qc38KjbDA1MNV21Yxf/h8Mhol8b2cRUSSUPJeVl+i\nJlMPA1MV66//N4NnXMruwt3RLzQKNPVQRCS5JXVTEMzUw3NaHs3rF/yeTzZ8wojZI2o0VTGevPOH\ngaycpqmHIiLJLGmbglCmHp7R5gym9J/CtKXTqp2qGE8+mnwPZ/91Br/fc4qmHoqIJLGkvaw8MPUw\nJye4qYeDjh/Eqh9Xcdc7d9HxkI5cdeJVkSsyCl56cwb31W3L/eefQL/pC/wuR0REfJSUTUFtpx4G\npiqOmD3igFMV40Xu5x9zY4FxqBXTffqbpNap63dJIiLio6T7+CAcUw8DUxW7t+leYapivNi0eQNX\n5X5OSnEx07ufSotDy9+kUkREkk1SNQXhnHoYmKp4SOMOdP3vS6zILz8jKnbt3r2LK2b8mw3NWjK5\neT1O7HSy3yWJiEgMSJqmoCbRw2A1bdCUf/Sfyo6Gx3Dqh69SsOeX2h80Cnq/MKHU1MNL/C5HRERi\nRFI0BcFED4N1TsujmXxkMwrSWnHSu9MoLi4O38Ej4LL503inbTeG7VjJ7wbf6Hc5IiISQxK+KQgl\nehiskUd15+YmO1lbryO/nvdc+E8QJrO/mstL32+kW9Eqnu6n6KGIiJSV8E1BIHo4Y0Zw0cNgPd6t\nP+fbN7zPkYz+5KXInShEizctZsiswfTb+xELzhnqdzkiIhKDEropKD/1MNL2T1U8KKamKq4vWE/f\n7L50bt6Z6QOe09RDERE5oIRtCsIRPQzW/qmKG7h1xVJWbF0RnRNXofTUw9mDZ9MwraHfJYmISIxK\nyKYgED3s1w/Gjo3uuQ+qW5/F3S+m3YZn6JPdhx92/hDdAkop2ruHK/89kFXbVpEzJEdTD0VEpEoJ\n1xSUjx6mpka/hiMbNeO1wbPI35VP/xf6+zZV8b+Xn8boP73KjMtf1NRDERGpVkI1BeWjhw19XClv\n37Q9Lw9+2bepiv+6dyTnzP6M9MszufCo30T13CIiEp8S5oqz0tHD+fMjEz0MVmCq4uBZg+l4SEfu\nO+e+qJz3b1Of4P+dNYgNt+/mjw9Nico5RUQk/iVMUxDq1MNIi/ZUxZfenME9hx5Nj+V5/P4vf4/o\nuVnbGV4AAAkWSURBVEREJLEkRFNQ26mHkVZ6quIhjdrQu93ZETlPYOrhkfnfMmPw1aSl1YvIeURE\nJDHF/TUFfkQPgxWYqnj00SPo+/VPvLlxedjP8d3mDVy9qGTq4Zmn0KJZy7CfQ0REEltcNwV+Rg+D\nlZaaxuyL7qdu8U76LF3CyjBOVdxTVMhpH+awrkVLJjdP48Rjuobt2CIikjzitimIhehhsNo1as7r\nJ3WjMKU+p4RxquJp7z7Lt407cEebulx8Tv+wHFNERJJPXDYFsRQ9DFa4pyqO+3gcny1/mqvqb+au\nEy4IU5UiIpKM4q4piMbUw0gbeVR3bgrDVMW5K+aS9XoWt3XqwfNnZIaxQhERSUZx1xREa+phpD1R\ny6mKizctZvDMwfTr1I+x58X4BRUiIhIX4qopiPbUw0h7refVHL1nBf/49DnmfTOvxvuVnno4dcBU\nUlPi4IIKERGJeXHTFMRD9DBYKSkpLD13GGelFTDghQGs/GFltfv8/MMm/vjnX2vqoYiIhF1cNAXx\nFD0MVlpqGrMGzqJFwxb0nt67wlTF7Ozsff9dtGc3y87rwqMTvubVS2Yk3dTD0q9FMtPr4NHrsJ9e\nC49eh9oLqSkwsxvNbI2Z/WJmH5nZqdU8/xwzyzWzXWa2wsyG1vRc8Rg9DFbTBk3JGZJzwKmKpf+Q\n3zD2DppuTmHNk3/iuPa/8qNUX+kvvEevg0evw356LTx6HWov6KbAzAYBjwL3AicDnwGvm1mzSp5/\nJDAXeBs4CXgCeNrMzq/uXDt2xG/0MFjVTVW8afz9/L1nP/5x52hOueYun6oUEZFEFspKQRbwlHPu\nOefcl8D1wE5geCXPHw2sds793jn3lXNuAjCz5DhVuvPO+I4eBiswVXHa0mnc//79+7b/beoTTDi2\nBwMXv8cjN97jY4UiIpLIghqIZGZ1gW7Ag4FtzjlnZm8BZ1Sy2+nAW+W2vQ48Vt35FiyAV16J7+hh\nsPZNVZz3MDsPOo5NWzbum3r4/I13+F2eiIgksGCnJDYDUoHyN+7/HuhUyT4tK3l+YzOr55zbfYB9\n6gPccsFLNP9mEXkHmgDcrRuYVV7p6tWwbVvljx98MHTsWPnjALm5UG4Zv4x27eDQQyt//McfvaWO\nqhzg57gw/UKebfIjf120mToFv9D24/nc3fMsPv/8i6qPleDy8/PJy8vzuwzf6XXw6HXYT6+FR6+D\nZ/nyfUP36ge7r5X/7LrKJ5tlABuAM5xzH5faPhY4yzlXYbXAzL4C/umcG1tq20V41xmkH6gpMLMh\nwLRgfhAREREp40rn3PRgdgh2pWArUAQcVm77YcB3lezzXSXPL6hklQC8jxeuBNYCu4KsUUREJJnV\nB47Eey8NSlBNgXNur5nlAucCswHMzEq+H1fJbh8CF5XbdkHJ9srO8wMQVHcjIiIi+ywIZadQ0gd/\nA64zs/8xs2OAyUA68CyAmT1kZlNKPX8y0N7MxppZJzO7Abi85DgiIiISI4L9+ADn3Isl9yR4AO9j\ngE+BC51zW0qe0hJoU+r5a82sD17a4CZgPTDCOVc+kSAiIiI+CupCQxEREUlccTH7QERERCJPTYGI\niIgAMdgUBDtsKRGZWU8zm21mG8ys2Mz6+V2TH8zsDjNbaGYFZva9mf3HzI72uy4/mNn1ZvaZmeWX\nfC0ws9/4XZffzGxMyd+RpLpw2czuLfm5S38t87suv5hZKzN73sy2mtnOkr8rXf2uK5pK3jfL/5ko\nNrPxwRwnppqCYIctJbCGeBdw3gAk80UfPYHxwK+A84C6wBtm1sDXqvzxLfAHoCvercbfAV42s86+\nVuWjkl8YRuL9O5GMPse72LtlyVcPf8vxh5kdDMwHdgMXAp2B24Af/azLB6ew/89CS+B8vPePF4M5\nSExdaGhmHwEfO+duLvne8P4xHOece8TX4nxiZsVAf+fcbL9r8VtJc7gZ7+6ZH/hdj9/M7Afgdufc\nv/yuJdrM7CAgF2/g2t3AYufcrf5WFT1mdi9wiXMuqX4bPhAzexjvLrtn+11LLDGzx4HezrmgVldj\nZqWg1LCltwPbnNexVDVsSZLLwXidbxVDLRKfmaWY2WC8+4NUehOwBDcBmOOce8fvQnx0VMlHjKvM\nbKqZtal+l4R0MbDIzF4s+Zgxz8yu9bsoP5W8n14JPBPsvjHTFFD1sKWW0S9HYknJqtHjwAfOuaT8\n7NTMjjez7XjLpBOBASXjy5NKSUPUBUjmsaEfAcPwlsuvB9oB88ysoZ9F+aQ93orRV3h3y50EjDOz\nq32tyl8DgCbAlOqeWF7QNy8S8clE4Figu9+F+OhL4CS8v+yXA8+Z2VnJ1BiY2eF4zeF5zrm9ftfj\nF+dc6Xvaf25mC4FvgIFAsn2clAIsdM7dXfL9Z2Z2PF6z9Lx/ZflqOPCqc66ymUSViqWVglCGLUkS\nMLMngd7AOc65TX7X4xfnXKFzbrVzbrFz7i68C+xu9ruuKOsGNAfyzGyvme0FzgZuNrM9JStKScc5\nlw+sAKqZB5+QNgHLy21bDrT1oRbfmVlbvAuz/xHK/jHTFJR0/YFhS0CZYUshDXaQ+FfSEFwC9HLO\nrfO7nhiTAtTzu4goews4Ae/jg5NKvhYBU4GTXCxdOR1FJRdedsR7g0w284FO5bZ1wls5SUbD8T52\nfyWUnWPt44O/Ac+WTGJcCGRRathSsij5XLAjEPitp72ZnQRsc859619l0WVmE4FMoB+ww8wCq0j5\nzrmkGqltZg8CrwLrgEZ4FxGdjfcZatJwzu0AylxTYmY7gB+cc+V/W0xYZvZXYA7eG19r4H5gL5Dt\nZ10+eQyYb2Z34MXvfgVcC1zna1U+KPlFehjwrHOuOJRjxFRTUINhS8niFOBdvCvtHd69G8C7aGS4\nX0X54Hq8n/+9ctuvAZ6LejX+aoH3/z8DyAeWABck+dX3Acm4OnA43nj5Q4EtwAfA6SVj55OKc27R\n/2/vjgkQiIEACG68YQAl2EAKrpASij8FNPliRsGVW9wla61n9e46T/1Wr7335+xkRzy6PiT8e6/k\nVu8UAADn3GanAAA4SxQAAJUoAACGKAAAKlEAAAxRAABUogAAGKIAAKhEAQAwRAEAUIkCAGD8AHit\n1mA1gCvCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1194096d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cst, pth = fastdtw(a, b)\n",
    "ca = [a[x[0]] for x in pth]\n",
    "cb = [b[x[1]] for x in pth]\n",
    "plot(a, label='a')\n",
    "plot(b, label='b')\n",
    "plot(ca, '--', label='ca')\n",
    "plot(cb, '--', label='cb')\n",
    "legend(loc='best')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Original TS: a = [array([0]), array([1]), array([3]), array([2]), array([3]), array([4])]\n",
      "             b = [array([1]), array([0]), array([1]), array([1]), array([2]), array([3])]\n",
      "Warped TS:   a = [array([0]), array([0]), array([1]), array([1]), array([3]), array([2]), array([3]), array([4])]\n",
      "             b = [array([1]), array([0]), array([1]), array([1]), array([2]), array([2]), array([3]), array([3])]\n",
      "DTW is (0.25, array([[ 1.,  0.,  1.,  1.,  2.,  3.],\n",
      "       [ 0.,  1.,  0.,  0.,  1.,  2.],\n",
      "       [ 2.,  3.,  2.,  2.,  1.,  0.],\n",
      "       [ 1.,  2.,  1.,  1.,  0.,  1.],\n",
      "       [ 2.,  3.,  2.,  2.,  1.,  0.],\n",
      "       [ 3.,  4.,  3.,  3.,  2.,  1.]]), array([[  1.,   1.,   2.,   3.,   5.,   8.],\n",
      "       [  1.,   2.,   1.,   1.,   2.,   4.],\n",
      "       [  3.,   4.,   3.,   3.,   2.,   2.],\n",
      "       [  4.,   5.,   4.,   4.,   2.,   3.],\n",
      "       [  6.,   7.,   6.,   6.,   3.,   2.],\n",
      "       [  9.,  10.,   9.,   9.,   5.,   3.]]), (array([0, 0, 1, 1, 2, 3, 4, 5]), array([0, 1, 2, 3, 4, 4, 5, 5])))\n"
     ]
    }
   ],
   "source": [
    "p = [[0, 0], [0, 1], [1, 2], [1, 3], [2, 4], [3, 4], [4, 5], [5, 5]]\n",
    "aw = [a[x[0]] for x in p]\n",
    "bw = [b[x[1]] for x in p]\n",
    "print('Original TS: a = {}\\n             b = {}'.format(list(a), list(b)))\n",
    "print('Warped TS:   a = {}\\n             b = {}'.format(aw, bw))\n",
    "# dtw = np.sum([m[x[0], x[1]] for x in p])\n",
    "print('DTW is {}'.format(dtw(a, b, dist=lambda x, y: norm(x - y, ord=1))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def fdtw(a, b):\n",
    "    return fastdtw(a, b)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def sdtw(a, b):\n",
    "    d, _, _, _ = dtw(a, b, dist=squared_norm)\n",
    "    return d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def ldtw(a, b):\n",
    "    d, _, _, _ = dtw(a, b, dist=l1_norm)\n",
    "    return d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30,\n",
       "           metric=<function fdtw at 0x116f391b8>, metric_params=None,\n",
       "           n_jobs=-1, n_neighbors=1, p=2, weights='uniform')"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "one_nn = KNeighborsClassifier(n_neighbors=1, metric=fdtw, n_jobs=-1)\n",
    "one_nn.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "predicted = one_nn.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "onn_accu = accuracy_score(y_test, predicted)\n",
    "onn_conf = confusion_matrix(y_test, predicted)\n",
    "# onn_log = log_loss(y_test, predicted)  # needs predict_proba\n",
    "onn_report = classification_report(y_test, predicted)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1-NN with DTW accuracy: 0.627934120088\n"
     ]
    }
   ],
   "source": [
    "print('1-NN with DTW accuracy: {}'.format(onn_accu))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1-NN with DTW confusion matrix:\n",
      " [[ 182    0   28  147  193   96   21]\n",
      " [   0 1387    1   25  541    0    2]\n",
      " [  11    0  476   27   27    7  207]\n",
      " [  60    0  172  604  214   63   52]\n",
      " [  10   76   23  108 1608    9   35]\n",
      " [  40    0   13  138   55  493    4]\n",
      " [   3    9   11   21  420    0   92]]\n"
     ]
    }
   ],
   "source": [
    "print('1-NN with DTW confusion matrix:\\n {}'.format(onn_conf))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1-NN with DTW report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.59      0.27      0.37       667\n",
      "          2       0.94      0.71      0.81      1956\n",
      "          3       0.66      0.63      0.64       755\n",
      "          4       0.56      0.52      0.54      1165\n",
      "          5       0.53      0.86      0.65      1869\n",
      "          6       0.74      0.66      0.70       743\n",
      "          7       0.22      0.17      0.19       556\n",
      "\n",
      "avg / total       0.65      0.63      0.62      7711\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print('1-NN with DTW report: {}'.format(onn_report))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Weighted DTW\n",
    "* DTW with penalty applied to warping distances $$M_{ij}=w(|i-j|)(a_i-b_j)^2$$\n",
    "* usually warping penalty is some logistic function of warping distance and sequence length $$w(d)=\\frac{w_{max}}{1+\\exp(-g(d-m/2))}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Time Warped Edit TWE\n",
    "* modifies DTW\n",
    "* adds some characteristics of Longest Commoon SubSequence LCSS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def twe(a, b, nu=1, lam=0):\n",
    "    assert a.shape[0] == b.shape[0]\n",
    "    d = np.zeros((a.shape[0] + 1, b.shape[0] + 1))\n",
    "    d[0, 0] = 0\n",
    "    d[1, 0] = a[0] ** 2\n",
    "    d[0, 1] = b[0] ** 2\n",
    "    for i in range(1, d.shape[0]):\n",
    "        d[i, 0] = d[i - 1, 0] + (a[i - 2] - a[i - 1]) ** 2\n",
    "    for j in range(1, d.shape[0]):\n",
    "        d[0, j] = d[0, j - 1] + (b[j - 2] - b[j - 1]) ** 2\n",
    "    for i in range(1, d.shape[0]):\n",
    "        for j in range(1, d.shape[0]):\n",
    "            if i > 1 and j > 1:\n",
    "                dst1 = d[i - 1, j - 1] + nu * np.abs(i - j) * 2 + \\\n",
    "                (a[i - 1] - b[j - 1]) ** 2 + (a[i - 2]- b[j - 2]) ** 2\n",
    "            else:\n",
    "                dst1 = d[i - 1, j - 1] + nu * np.abs(i - j) + (a[i - 1] - b[j - 1]) ** 2\n",
    "            if i > 1:\n",
    "                dst2 = d[i - 1, j] + (a[i - 1] - a[i - 2]) ** 2 + lam + nu\n",
    "            else:\n",
    "                dst2 = d[i - 1, j] + a[i - 1] ** 2 + lam\n",
    "            if j > 1:\n",
    "                dst3 = d[i, j - 1] + (b[j - 1] - b[j - 2]) ** 2 + lam + nu\n",
    "            else:\n",
    "                dst3 = d[i, j - 1] + b[j - 1] ** 2 + lam\n",
    "            d[i, j] = np.min([dst1, dst2, dst3])\n",
    "    return d[d.shape[0] - 1, d.shape[0] - 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "twe(a, b, nu=0.1, lam=0.5) = 8.2\n"
     ]
    }
   ],
   "source": [
    "nu = 0.1; lam = 0.5\n",
    "print('twe(a, b, nu={}, lam={}) = {}'.format(nu, lam, twe(a, b, nu=nu, lam=lam)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Move-Split_Merge\n",
    "* a typical edit distance based\n",
    "* context is added to insertions and deletions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def c_penalty(a, ap, b, c=1):\n",
    "    if (ap <= a and a <= b) or (ap >= a and a >= b):\n",
    "        return c\n",
    "    else:\n",
    "        return c + np.min([np.abs(a - ap), np.abs(a - b)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def msm(a, b, c=1):\n",
    "    d = np.zeros((a.shape[0], a.shape[0]))\n",
    "    d[0, 0] = np.abs(a[0] - b[0])\n",
    "    for i in range(1, a.shape[0]):\n",
    "        d[i, 0] = d[i - 1, 0] + c_penalty(a[i], a[i - 1], b[0])    \n",
    "    for i in range(1, a.shape[0]):\n",
    "        d[0, i] = d[0, i - 1] + c_penalty(b[i], b[i - 1], a[0])\n",
    "    for i in range(1, a.shape[0]):\n",
    "        for j in range(1, a.shape[0]):\n",
    "            d[i, j] = np.min([d[i - 1, j - 1] + np.abs(a[i] - b[j]),\n",
    "                              d[i - 1, j] + c_penalty(a[i], a[i - 1], b[j]),\n",
    "                              d[i, j - 1] + c_penalty(b[j], b[j - 1], a[i])])\n",
    "    return d[d.shape[0] - 1, d.shape[0] - 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.0"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "msm(a, b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "msm_one_nn = KNeighborsClassifier(n_neighbors=1, metric=msm, n_jobs=-1)\n",
    "msm_one_nn.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* multiple tests using 1-NN on 72 problems have shown no substantial differences between algorithms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Differential Distance Based Classifiers\n",
    "* based on the first order difference of the series $$diff=a_i-a_{i-1}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Complexity Invariant Based CID\n",
    "* weighs distance measures to compensate for differences in complexity in series compared"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def cid(a, b, dist=fdtw):\n",
    "    d = dist(a, b)\n",
    "    ca = np.sum([(a[i] - a[i - 1]) ** 2 for i in range(1, a.shape[0])])\n",
    "    cb = np.sum([(b[i] - b[i - 1]) ** 2 for i in range(1, b.shape[0])])\n",
    "    return d * np.max([ca, cb]) / np.min([ca, cb])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.0"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cid(a, b, dist=fdtw)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Derivative Distance DD-DTW\n",
    "* weighted combination of raw series and first order differences\n",
    "* parameter $\\alpha$ needs to be found using cross-validation\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def dd_dtw(a, b, dist = fdtw, alpha=0.5, toplot=False):\n",
    "    c = np.array([(a[i] - a[i - 1]) for i in range(1, a.shape[0])])\n",
    "    d = np.array([(b[i] - b[i - 1]) for i in range(1, b.shape[0])])\n",
    "    # TODO replace with a real DTW\n",
    "    x = dist(a, b)\n",
    "    y = dist(c, d)\n",
    "    if toplot:\n",
    "        plot(a, label='a')\n",
    "        plot(b, label='b')\n",
    "        plot(c, label='diff(a)')\n",
    "        plot(d, label='diff(b)')\n",
    "    legend(loc='best')\n",
    "    return alpha * x + (1. - alpha) * y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dd_dtw(a, b, alpha=0.0) = 4.0\n",
      "dd_dtw(a, b, alpha=0.1) = 3.9\n",
      "dd_dtw(a, b, alpha=0.2) = 3.8\n",
      "dd_dtw(a, b, alpha=0.3) = 3.7\n",
      "dd_dtw(a, b, alpha=0.4) = 3.6\n",
      "dd_dtw(a, b, alpha=0.5) = 3.5\n",
      "dd_dtw(a, b, alpha=0.6) = 3.4\n",
      "dd_dtw(a, b, alpha=0.7) = 3.3\n",
      "dd_dtw(a, b, alpha=0.8) = 3.2\n",
      "dd_dtw(a, b, alpha=0.9) = 3.1\n",
      "dd_dtw(a, b, alpha=1.0) = 3.0\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.42499999999999999"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAFkCAYAAAC9wjgoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3Xt8VOW9LvDnnRCREOViANlUQEQTLt6S0MoRKGIxeOMi\nCkQuSrxsiiB6trstPQe5dEup53gjBbutUFRIqlxa8VRapEqtFYpOtIgkCAIiyi2IgAQQyO/8MRlc\nmUySNTPrXdfn+/nk8zGTmVmvEckvz3rWu5SIgIiIiIIr5PQCiIiIyFkcBoiIiAKOwwAREVHAcRgg\nIiIKOA4DREREAcdhgIiIKOA4DBAREQUchwEiIqKA4zBAREQUcBwGiIiIAs62YUAp9TOlVLVS6km7\njklERESNs2UYUEr1AnA/gH/ZcTwiIiIyT/swoJTKBLAYwL0AvtZ9PCIiIkqMHcnAPACvicibNhyL\niIiIEtRE55srpUYBuApAvsnnXwCgAMBOACf0rYyIiMh3zgXQGcBfRORgIi/UNgwopb4H4GkAPxKR\nUyZfVgBgia41ERERBcBoACWJvEBnMpAHoA2AMqWUqnksDUA/pdQkAE1FRGJesxMAFi9ejG7dumlc\nGhk9/PDDeOqpp5xeRqDwe24/fs/tx++5fgcOALNnA2+/DfTuXY5168YANT9LE6FzGFgD4PKYxxYB\nKAcwJ84gANScGujWrRtyc3M1Lo2MWrRowe+3zfg9tx+/5/bj91wfEaCkBJg8GUhPB1asADp1AvLy\nACRxml1bgVBEjonIZuMHgGMADopIua7jEhER+dmePcDQocCYMcCgQcDmzcCwYam9p9YCYRzx0gAi\nIiJqRLw0INUhIMrWYUBEBth5PCIiIj/YsweYMAFYuRIoLASKi4ELLrDu/e1OBsiFCgsLnV5C4PB7\nbj9+z+3H73nqdKYBRip+j88ZSqlcAOFwOMzSCRERBVqiaUBZWRnyIg3CPBEpS+RYnksGdu3ahcrK\nSqeXQUnIyspCx44dnV4GEZGr2ZUGGHlqGNi1axe6deuGqqoqp5dCScjIyEB5eTkHAiKieujuBtTH\nU8NAZWUlqqqquCmRB5WXl2PMmDGorKzkMEBEFMOJNMDIU8NAFDclIiIiv3AqDTDy5DBARETkdU6n\nAUZ23MKYiIiIDHTsIpgKJgNEREQ2cVMaYMRkgIiIyAZuSwOMmAwQERFp5NY0wIjJABERkSZuTgOM\nmAwQERFZzAtpgBGTASIiIgt5JQ0w4jDgIrt27cLEiRORk5ODjIwMZGVlYcSIEfjss8+cXhoRETVC\nBFiyBOjRA1i/PpIGlJTYv4FQMniawEXee+89rF+/HoWFhfje976HnTt3Yv78+bjuuuuwefNmnHvu\nuU4vkYiI4nDDLoKp4DDgIrfccguGDx9e67Fbb70V11xzDZYvX47Ro0c7tDIiIorHa92A+vh6GKiq\nAioq9B4jJwfIyLDmvZo2bXr2n0+fPo0jR46gS5cuaNmyJcrKyjgMEBG5iNfTACNfDwMVFUBent5j\nhMOAVfdMOnHiBGbPno1Fixbhiy++gIgAAJRSOHz4sDUHISKilPglDTDy9TCQkxP5Ya37GFaZNGkS\nXnjhBTz88MO45ppr0KJFCyilMHLkSFRXV1t3ICIiSoqf0gAjXw8DGRnW/dZuh+XLl+Puu+/G448/\nfvaxkydP4uuvv3ZwVURE5Mc0wIiXFrpIWlpanQRg7ty5OHPmjEMrIiIiL+4bkChfJwNec8stt+Cl\nl17C+eefj+7du2PdunX461//iqysLKeXRkQUOH5PA4w4DLjI3Llz0aRJE5SUlODEiRPo06cP1qxZ\ng4KCAiilnF4eEVFg+LUbUB8OAy5y/vnn4/nnn6/z+Pbt2x1YDRFR8AQpDTBiZ4CIiAjB6AbUh8kA\nEREFWlDTACMmA0REFFhBTgOMmAwQEVHgMA2ojckAEREFCtOAupgMEBFRIDANqB+TASIi8j2mAQ1j\nMkBERL7FNMAcJgNERORLTAPMYzJARES+wjQgcUwGiIjIN5gGJIfDgIvMmDEDoVAIX331ldNLISLy\nFBFgyRKgRw9g/fpIGlBS4u+bC1mJw4CLKKV4d0IiogQxDUgdOwNERORJ7AZYh8kAERF5DtMAa3EY\ncKEDBw5gxIgRaNGiBbKysvDQQw/h5MmTTi+LiMhx7AbowWHAZUQEI0aMwLfffos5c+bg5ptvxty5\nc/Hv//7vTi+NiMhRTAP0YWfAhS655BKsWLECAPDjH/8Y5513Hp599lk88sgj6Nmzp8OrIyKyF7sB\n+vl6GKg6VYWKygqtx8jJykFGeoZl76eUwgMPPFDrscmTJ2P+/Pl4/fXXOQwQUaDs2QNMmACsXAkU\nFgLFxTwloIOvh4GKygrkPZen9Rjh+8PIbZ9r6Xt27dq11ueXXHIJQqEQdu7caelxiIjcimmAvXw9\nDORk5SB8f1j7MXTj3gNEFCRMA+zn62EgIz3D8t/a7bB161Z06tTp7Ofbtm1DdXU1Onfu7NyiiIg0\nYxrgHF5N4DIignnz5tV6bO7cuVBK4cYbb3RoVUREevFKAWf5Ohnwqh07dmDIkCEYNGgQ3n33XSxZ\nsgRjxozB5Zdf7vTSiIgsxTTAHZgMuEwoFMLLL7+Mpk2bYurUqVi1ahUefPBBPP/8804vjYjIUkwD\n3IPJgItMnz4d06dPBwC88sorDq+GiEgPpgHuozUZUEpNUEr9Syl1uObjXaXUIJ3HJCIi92Ia4E66\nk4HPAfwUwFYACsDdAF5VSl0lIuWaj01ERC7BNMDdtCYDIvInEfmziHwqIttE5H8D+AbANTqPS0RE\n7sE0wP1s6wwopUIARgDIALDOruMSEZEzmAZ4h/arCZRSPZVSRwGcBDAfwDAR0XvDACIX+/prYO9e\np1dBpNfevUwD7FZZVZn0a+1IBioAXAmgBYDbAbyolOrX0EDw8MMPo0WLFrUeKywsRHZ2ttaFEulW\nVQX06wds2wbMmQNMmgSEeIEv+QjTAHuUlpaitLT07Oe7j+zGxl0bk34/7cOAiJwGsL3m0w+UUt8H\nMAXAj+t7zVNPPYXc3LrbCJeVlWlZI5FdpkyJDAIjRkT+edkyYOFCIObeVESeZLynwKhRkXsKZGU5\nvSp/KiwsRGFhIfZ+sxcT/t8EfLDlAxRkFOAvP/lLUu/nxO8kIQBNHTgukaMWLwaefx749a+BRYuA\nt94Cdu8GrrgCmDsXqK52eoVEyREBliwBevQA1q8Hli8HSks5COgkIliycQm6z+uOdbvXYcWIFZh9\n/eyk30/3PgOzlVJ9lVKdaroDvwTwQwCLdR6XyG0qKiK/MY0dC4wfH3msf39g40bgnnsiKUH//pHU\ngMhLjFcKFBQAH38M3Hab06vyt73f7MWwl4dhzB/GYFDXQdg8cTOGdUvtXIzuZKAtgBcQ6Q2sAZAH\n4AYReVPzcYlco6oKuOMO4KKLgPnzAeMdqTMzI1EqUwLyGqYB9ouXBpQML8EFGanf31n3PgP3ikgX\nEWkmIheKCAcBCpwpU4BPPwWWLo388I+HKQF5CdMA++lIA4zYYybSyNgT6Nmz4ecyJSC3YxpgP51p\ngBGHAY+YMWMGQoZr0Dp37oyioqJaz9m2bRtuuOEGtGzZEmlpaVi5ciUA4L333sO1116LzMxMpKWl\nYePG7y4/mThxIgoKChJez1dffYXMzEz8+c9/TvLfyP/i9QTMYEpAbsQ0wH660wAj3rXQI5RSUIaT\nzaFQqNbnADBu3Dh89tlnmD17Nlq2bIn8/HycPn0ad9xxBzIyMvD0008jIyMDnTp1AgDs2LEDCxYs\nwBtvvJHwelq3bo17770X06ZNw6BBvPdUrIZ6AmZEU4Lhw4GiokhKwH0JyAmx+wYsX84hQDcRQclH\nJZi8ajLS09KxYsQKbUNArYO65QNALgAJh8MSTzgcloa+7mczZsyQUCh09vNvv/1WTp8+ffbz48eP\ni1JKHn300Vqvq6ioEKWULFy4sM57TpkyRXJycpJeU3l5uSil5K233mr0uUH7b3fvvSLNmol89FHq\n73X0qMikSSKASN++Ilu3pv6eRGZ8+aXI4MGRP3ujRokcOOD0ivxvz9E9MqR0iGAGpHBZoVQeqzT9\n2ujfswByJcGfv/wdw6PS09ORlpZ29vP9+/cDQJ2dG/ft2xf38dOnT6OkpAQjR45Meg05OTno2bMn\nFi1alPR7+FEiPQEz2CUgu7EbYD+xqRtQHw4DLvTOO++gV69eaNasGS699FI899xzdZ5j7AzMnDkT\nnTt3hlIKjzzyCEKhELp06YLx48ejf//+UErh9ttvRygUwoABAwAAf//733Hw4EFcf/31td731KlT\nePTRR5Gfn4+WLVsiMzMT/fr1w9q1a+OudeDAgXjttdes/QZ4WLI9ATPYJSA7sBtgPzu7AfVhZ8Bl\nNm3ahIKCArRt2xazZs3CqVOnMGPGDLRt27bW84x9geHDh6NVq1Z46KGHcOedd+Kmm25CZmYm2rVr\nhw4dOmD27NmYMmUKevXqhXbt2gEA3n33XSilcPXVV9d63yNHjmDhwoUoLCzE/fffj6NHj2LBggUY\nNGgQNmzYgCuuuKLW8/Py8vD0009j8+bN6N69u6bvijek2hMwg10C0oXdAPuJE92Aevh7GKiqivyq\nplNODpCRYdnbTZs2DUAkHejQoQOAyA/7ng3kzT179sR5552Hhx56CLm5ubjzzjvPfu3EiROYPXs2\n+vbti9sM/2dv2bIFrVu3RmbMhe+tW7fGzp070aTJd3807rvvPmRnZ6O4uBi//e1vaz2/S5cuEBEO\nA/huP4ENG+rfT8Aq0ZRg6lTe44BSx3sK2C96T4FXt7yKwp6FKL6x2LZTAvH4exioqADy8vQeIxwG\n4txUKRnV1dVYvXo1hg0bdnYQAIDs7GwUFBRg1apVlhwHAA4ePIhWrVrVeVwpdXYQEBF8/fXXOHPm\nDPLz8+PeKCr6HpWVyd860w+iPYEFC6zpCZjBlIBSxTTAfm5KA4z8PQzk5ER+WOs+hkUOHDiA48eP\no2ucX++ys7MtHQYARK/gqOOFF17Ak08+iYqKCpw6ders4126dKn3PWIvcwwSnT0BM5gSUDKYBtjP\nbWmAkb+HgYwMy35r95sLLrgAGzZsqPP44sWLMX78eNx22234yU9+grZt2yItLQ2zZ8/G9u3b6zz/\n0KFDAICsgP4tYkdPwAymBGQW0wD7uTUNMOJfEy7Spk0bNGvWDFu3bq3ztQqLuw85OTk4dOgQjh49\nWuvx5cuX45JLLsGyZcswevRoDBw4EAMGDMCJEyfivs+OHTuglEK3bt0sXZ9XmLnvgJ14xQE1hFcK\n2M8NVwqYwWHARUKhEAoKCvDHP/4Ru3fvPvt4eXk5Vq9ebemxevfuDRFBOOY0inHvgqh//vOfWLdu\nXdz3CYfDaNGiRSDLg1bvJ2AV7ktAsbhvgP2c3jcgURwGXGbmzJkQEfTp0wePP/44HnvsMQwYMKDB\nqwmS0adPH7Ru3Rpr1qyp9fgtt9yCTz/9FEOHDsVvf/tbTJ06FTfeeCN69OgR933eeOMN3HrrrZau\nzQuc7gmYwZSAAKYBTvBKGmDEYcBlLr/8cqxevRpt27bF9OnTsWjRIsyaNQtDhw6t9bzYexXU91j0\n8Vjp6ekYPXo0li5dWuvxu+++G7/85S+xceNGTJkyBW+88QaWLFmCvLy8Ou9TUVGBTZs2Ybxbfxpq\n4paegBlMCYKLaYD9vJYG1JLo/sU6P8B7E9hq+/bt0rRpU3nzzTeTev2UKVMkLy/P1HP99N/OyvsO\n2In3OAgO3lPAfqncU8AqvDcBJeXiiy/GPffcgzlz5iT82q+++goLFy7EY489pmFl7uXWnoAZTAn8\nj2mA/cTLaYCBvy8tpEbNmzcvqde1bt0aR44csXg17uaFnoAZ3JfAn7hvgP3cvG9AopgMEJngpZ6A\nGUwJ/INpgP38kgYYcRggMsFt+wlYhVcceBuvFLCfF68UMIPDAFEjvNwTMIMpgfcwDbCfH9MAIw4D\nRA2I9gTGjfN2T8AMpgTewDTAfn5NA4w4DBDVI9oT6NjRHz0BM5gSuBfTAPv5PQ0w4jBAVI9oT+CV\nV4DmzZ1ejb2YErgL0wD7BSENMOIwQBRHtCcwb54/ewJmMCVwHtMA+wUpDTDiMEAUw9gTuPtup1fj\nPKYEzmAaYL+gpQFGHAaIDILYEzCDKYF9mAbYL6hpgBGHAY+YMWMGQqHv/nN17twZRUVFtZ6zbds2\n3HDDDWjZsiXS0tKwcuVKAMB7772Ha6+9FpmZmUhLS8PGjRvPvmbixIkoKCg4+/nf/vY3hEIhrFix\notE1FRYWYuTIkan+q7lKkHsCZjAl0ItpgP2CnAYYcTtij4i9I2EoFKpzF8Fx48bhs88+w+zZs9Gy\nZUvk5+fj9OnTuOOOO5CRkYGnn34aGRkZ6NSpEwBgx44dWLBgAd544406xzLjpz/9KfLz8/HRRx/h\n8ssvT/Hf0HnRnsDChcHtCZgRTQmGDweKiiIpwZw5wKRJQIi/XiRFBCgpASZPBtLTI2kAhwC9RAQl\nH5Vg8qrJSE9Lx4oRKwI5BERxGPCoLVu21EoKTpw4gfXr12PatGmYOHFireft2rULCxYsqHOr4Wee\neQZdunRBv379aj0ukTtINuqqq65Cfn4+nnjiCSxatCj5fxkXYE8gcbzHgTV4TwH7+emeAlbhHO9R\n6enpSEtLO/v5/v37AQAtWrSo9bx9+/bFffz06dMoKSlJOeYfMWIEVqxYgaqqqpTex0nsCSSPXYLk\nsRtgP3YD6sdhwIXeeecd9OrVC82aNcOll16K5557rs5zjJ2BmTNnonPnzlBK4ZFHHkEoFEKXLl0w\nfvx49O/fH0op3H777QiFQhgwYAAA4O9//zsOHjyI66+/vs57K6Vw5swZ/PznP0f79u2RmZmJIUOG\nYPfu3XWeO3DgQHzzzTd1TjV4CXsCqWOXIDHsBtiP3YCG8TSBy2zatAkFBQVo27YtZs2ahVOnTmHG\njBlo27ZtrecZz+sPHz4crVq1wkMPPYQ777wTN910EzIzM9GuXTt06NABs2fPxpQpU9CrVy+0a9cO\nAPDuu+9CKYWrr766zhpEBP/1X/+FUCiEn/3sZ9i/fz+eeuopDBw4EB9++CGaNm169rndu3dHs2bN\n8I9//ANDhgzR9F3Rhz0B67BL0Dh2A+zHboBJIuKaDwC5ACQcDks84XBYGvq6HwwdOlQyMjJk9+7d\nZx+rqKiQJk2aSCgUOvtY586dZfz48Wc/37lzpyil5Iknnqj1fmvXrhWllCxfvrzW42PHjpU2bdrU\nOX70+RdddJEcO3bs7ONLly4VpZQUFxfXeU12drbcfPPNDf57ufG/XXm5SPPmIuPGiVRXO70afzl6\nVGTSJBFApG9fka1bnV6R8778UmTw4Mj3ZNQokQMHnF6R/+05ukeGlA4RzIAULiuUymOVTi9Jq+jf\nswByJcGfv75OBqrOnEGF5nPZORkZyDCcu09FdXU1Vq9ejWHDhqFDhw5nH8/OzkZBQQFWrVplyXEA\n4ODBg2jVqlW9X7/rrruQkZFx9vPbb78d7du3x+uvv45JkybVem6rVq1QWVlp2drswJ6AXkwJvsM0\nwH7CNCBhvh4GKqqqkBcOaz1GOC8PueedZ8l7HThwAMePH0fXOHXs7OxsS4cBoOGrBuKtoWvXrti5\nc2fc9zF7OaJbRHsCGzawJ6BT0K844JUC9uOVAsnx9TCQk5GBcF6e9mN40QUXXIANGzZY8l6HDh3C\nZZddZsl72YE9AXsFMSVgGmA/pgGp8fUwkJGWZtlv7XZo06YNmjVrhq1bt9b5WkVFhaXHysnJQUlJ\nCY4ePYrz4nyP4q1h27ZtuPLKK2s9dubMGXz++eeeKQ9yPwHnBCUlYBpgP6YBqfPpXO5NoVAIBQUF\n+OMf/1jrMr7y8nKsXr3a0mP17t0bIoJwPadRXnzxRXzzzTdnP1+6dCn27NmDm266qdbzNm/ejBMn\nTuDaa6+1dH06sCfgPD/vS8B9A+wn3DfAMhwGXGbmzJkQEfTp0wePP/44HnvsMQwYMAA9Lc6z+/Tp\ng9atW2PNmjVxv966dWv06dMHzzzzDKZOnYq77roLl112Ge69995az1u9ejWaN2+OH/3oR5auTwfu\nJ+AeftuXgPsG2I/7BliLw4DLXH755Vi9ejXatm2L6dOnY9GiRZg1axaGDh1a63mx9yqo77Ho47HS\n09MxevRoLF26NO7zf/7zn+OWW27BnDlzUFxcjIEDB2LNmjU499xzaz132bJlGD58OJq7/KdrtCcw\nbx57Am7hh5SAaYD9mAZokui1iDo/wH0GbLV9+3Zp2rSpvPnmm0m9/oMPPpC0tDTZuHFjo8918r8d\n9xNwPy/uS8B9A+wXtH0DEpXKPgNMBgLs4osvxj333IM5c+Yk9fpf/epXuOOOO1x9x0L2BLzBSykB\n0wD7CdMA7Xx9NQE1bt68eUm/trS01MKV6MH9BLzF7Vcc8EoB+/FKAXswGSDfYk/Am9yYEjANsB/T\nAHtxGCBf4n4C3ueWKw54pYD9eKWA/TgMkO+wJ+AfTqYETAPsxzTAORwGyHe4n4D/2J0SMA2wH9MA\nZ3EYIF9hT8C/7EgJmAbYj2mAO3AYIN9gTyAYdKUETAPsxzTAPTx5aWF5ebnTS6AE6f5vxp5AsFh5\nJ0TeYdB+wjsMuo7WYUApNRXAMAA5AI4DeBfAT0Xkk2TeLysrCxkZGRgzZoyFqyS7ZGRkIEtT3sr9\nBIIp1X0JuG+A/bhvgDvpTgb6AigG8H7NsX4JYLVSqpuIHE/0zTp27Ijy8nJUVlZavEyyQ1ZWFjp2\n7Gj5+0Z7AgsXsicQRMmkBEwD7Mc0wN20DgMiUut+t0qpuwHsB5AH4J1k3rNjx45afqCQN7EnQFFm\nUwKmAfZjGuB+dhcIWyJyE4WvbD4u+RB7AhSroSsOeKWA/XilgHfYViBUkfvoPg3gHRHZbNdxqRGH\nDgGnTwNt2ji9koSxJ0D1iZcStGrFNMBOTAO8xc6rCeYD6A7g2sae+PDDD6NFixa1HissLERhYaGm\npQWQ8aRp27bApk1AE+9cXMKeADUmtkuwZQu7AXZgN8AepaWldW4Wd/jw4aTfT4lIqmtq/CBK/RrA\nrQD6isiuBp6XCyAcDoeRm5urfV2BtXdv5KTpq68CN94IrFoFvPRS5AJrD6ioAPLzI3/JL1rE0wPU\nuJMnIwEYEyS99hzdgwl/moCVW1ZiVM9RKL6xGFkZjGDsUlZWhry8PADIE5GyRF6rvTNQMwgMAXBd\nQ4MA2SB60rR798hJ0z/8AXj9deDWW4Ff/CLyt6XLsSdAyWjalIOATtFuQI/5PbB+93osH7EcpcNL\nOQh4iNZhQCk1H8BoAHcCOKaUalfzca7O41Ice/cCw4ZFfvu/8cbI9mpDh0a+Nn068MknwO9/7+wa\nTeB9B4jcZc/RPRj68lCM+cMYFHQtwMcTP8Zt3Xguxmt0JwMTAJwPYC2ALw0fIzQfl6LipQFLlgAX\nGIo8eXmeSAd43wEi92Aa4C9ahwERCYlIWpyPF3Uel2o0lAbEcnk6wP0EiNyDaYD/8EZFfmQmDYjl\n4nSAPQEid2Aa4F8cBvwmkTQglkvTAfYEiJzHNMDfvHNhOTXMuG/AOedE0gCzQ0CUMR0YNcoV+w5w\nPwEiZ8XuG7B8xHIOAT7EZMAPUkkDYrkoHWBPgMhZTAOCg8OAlyXTDWiMS7oD7AkQOYfdgODhMOBV\nVqYBsVyQDrAnQOQMpgHB5PxJYUqMFd2AxjjcHWBPgMh+7AYEG5MBL9GZBsRyKB1gT4DIfkwDiMOA\nF+joBjTGge4AewJE9mI3gKI4DLidnWlALJvTAfYEiOzDNICM2BlwKzu6AY2xsTvAngCRPdgNoHiY\nDLiRk2lALBvSAfYEiOzBNIDqw2HATZzoBjRGc3eAPQEi/dgNoMZwGHALN6UBsTSmA+wJEOnFNIDM\nYGfAaW7oBjRGU3eAPQEifdgNoEQwGXCSm9OAWBanA+wJEOnDNIASxWHACW7sBjTGwu4AewJEerAb\nQMniMGA3L6UBsSxKB9gTILIe0wBKBTsDdvFCN6AxFnQH2BMgsha7AWQFJgN28HIaECuFdIA9ASJr\nMQ0gq3AY0MmL3YDGJNkdYE+AyDrsBpDVOAzo4qc0IFYS6QB7AkTWYBpAOrAzYDU/dAMak2B3gD0B\notSxG0A6MRmwkp/TgFgm0wH2BIhSxzSAdOMwYAU/dgMaY6I7wJ4AUWrYDSC7cBhIVZDSgFiNpAPs\nCRAlj2kA2YmdgWQZuwHp6f7sBjSmge4AewJEyWE3gJzAZCAZxjRg0CBg8+bgDQJRcdIB9gSIksM0\ngJzCYSARxm7AunXAihWRdMDP3YDGxHQH2BMgShy7AeQ0DgNmxUsDhg1zelXuYEgH2BMgSgzTAHID\ndgYaE9sNWLGCQ0CsmnTgyH/+Ar/bOwq/XdiEPQGiRrAbQG7CZKAhTANM23HXdJy/9xPM6/N79gSI\nGsE0gNyGyUA8TAMSUlUFDJ6Rh2cyb8V9+34BdSa5OxoS+R3TAHIrJgOxmAYkLNoT6Pj8dIS2JndH\nQyK/YxpAbsZhIIpXCiQlup/AvHlA15HJ3dGQyM94pQB5AYcBgGlAkuLuJ5DEHQ2J/IppAHlFsIcB\npgFJq3c/ARP3LCDyO6YB5DXBHQaYBqSkwf0EmA5QgDENIC8KXuWbVwqkrNH7DjRwzwIiv+KVAuRl\nwUoGmAakzPR9B5gOUIAwDSCvC8YwwG6AJRK67wC7AxQA7AaQX/h/GGAaYJmE7zvAdIB8jGkA+Yl/\nT+ayG2CpRnsC8bA7QD7EbgD5kT+TAaYBljLdE4iH6QD5CNMA8it/DQPsBlguoZ5APOwOkA+wG0B+\n559hgGmAFgn3BOJhOkAexjSAgsD7J3HZDdAmqZ5APOwOkAexG0BB4u1kgGmANin1BOJhOkAewjSA\ngsabwwC7AVql3BOIh90B8gB2AyiovDcMMA3QzpKeQDxMB8jFmAZQkHnn5C27AbawrCcQD7sD5ELs\nBhBpTgZ8mV7WAAAYwUlEQVSUUn2VUiuVUl8opaqVUoOTeiOmAbawvCcQD9MBchGmAUQRuk8TNAfw\nIYCJACThV7MbYBstPYF42B0gF2A3gKg2rcOAiPxZRB4VkVcBJPbjhWmArbT1BOJhOkAOYhpAVJc7\nC4Svv840wEbRnsC8eRp6AvEwHSAHMA0gqp87G1zTpgGFhUBxMYcAzTZsPoX7f1aNceOa6usJxDN9\nOpCfH0kHxoyx8cDOO1h1EB/t/8jpZQTK4a+PYumbL2PT/k0YcuEQDO8+HJn7MrF231qnl+ZrLTMz\ncVV+vtPLIBOUSOKn8pM6kFLVAIaKyMoGnpMLINzj4gtxcc98KMOZhcLCQhQWFtqw0mAQAX5Xegb3\nf1uG6n87jl917YL/uLgDQtrKAnEMHgxs2QJ8/HEgriwwttYPnTjk9HICI731j9Ds4sk4knm+00sJ\nnB/u3Im1tv6WERylpaUoLS2t9djhw4fx9ttvA0CeiJQl8n6uHAZwP9Cvdz8sHLwQl7S+xJb1Bcme\nPZGrBlZeugVpg/bh9qw2ePnrfejbogUWZmeja0aGPQsJhyPpwEsv+T4d2HN0Dyb8aQJWblmJUT1H\nYVq/aTgn7Rynl+Vr+0+dwaNfHMJfj5zAyLVr8ZNrr8H5F3V2elmB0qxZM3To1MnpZQRGWVkZ8vLy\ngCSGAVf+Ovbft/435nw6B1f85grMuX4OHvj+Awgpd9YbvMS4VcOZ6/YCt+zBc9nZKGrfHhMOXYii\nLVtwxfvvY06XLpjUwYaUIAD7DvAadvuJCEr278fkrVuRfvIkls+cidvuvhsYOMjppRG5lu59Bpor\npa5USl1V81CXms8vauh1+f+Wj40/3oiiq4rw4J8fxHUvXIdPv/pU51J9b88eYOjQyC/g/2PkMZyZ\n8gnGtmuH8RdeCADo36oVNubn45727TFl2zb0//BDbKuq0r8wH19ZwNa6/facPImhmzZhTHk5Cpo0\nwcdjx+K2zp2B8eOdXhqRq+n+dTsfwAcAwojsM/AEgDIAMxt7YeY5mSi+qRhv3fUWPj/8Oa74zRUo\n/mcxqqVa74p9JrpVQ48ewPr1QOmKM/isaDMuOvdczL/0UijDb/+ZTZqg+NJL8daVV2L3yZO44v33\nMXf3blTrPJXkwysL2Fq3n4hgyb596PHee1h/5AiWd+2K0vvuQ1aLFpo3ziDyB937DPxNREIikhbz\nUWT2Pfp37s+UIEnGNCC6VcNfu2/Dp8ePY2n37sisJ5a3PSXwUTrANMB+tdKA1q3xca9euG3mzMjG\nGUuXApmZTi+RyPU8cSKeKUFiYtOA6FYNq07txfN79uDXl16Kno38BWlrSuCDdIBpgP3qpAE9eqC0\ne3dkvfJKZOOMX//apo0ziHxARFzzASAXgITDYanP0ZNHZdKfJglmQPr9rp9sO7it3ucG0Zdfigwe\nLAKIFBaKVFZGHi//5htp/re/ydjNm6W6ujqh9zx66pRM+uQTwVtvSd+yMtl67Jj1C3///ciiX3rJ\n+vfW7MsjX8rg0sGCGZBRy0bJgWMHnF6S73154oQM3rhR8NZbMurjj+XAyZORL5SXizRvLjJ2rEiC\nf86JvC4cDgsip+RzJcGfv55IBoyYEsRXXxpwwQVA1ZkzuGNz/J6AGbakBB5MB4RpgO2kvjTgnHO+\nu8HGRRexJ0CUIM8NA1HsEnwnXjfAeBuHKdsa7wmYob1L4KHuALsB9ovbDWjT5rsnRG+wwZ4AUcI8\nOwwATAkaSgOiFu813xMwQ2tK4IF0gGmA/RpMA6KiN9hgT4AoOYmeV9D5AROdgfoErUtQXzfAKJWe\ngBlaugQu7g6wG2C/ersBRuwJEIlIwDoD9QlKSmAmDQBS7wmYoSUlcGE6IEwDbCdm0gCAPQEii/hm\nGIjyc5egsW6AkVU9ATMs7xK4qDvAboD9Gu0GGLEnQGQJ3w0DgP9SArNpQJTVPQEzLE0JXJAOMA2w\nn+k0IIo9ASLrJHpeQecHUugM1MfrXQIz3QAj3T0BMyzpEjjYHWA3wH6mugFG7AkQ1cHOQAO8mhIk\nmgYA9vQEzLAkJXAgHRCmAbaTRNMAgD0BIg18PwxEealLkEg3wMjOnoAZKXcJbOwOsBtgv4S6AUbs\nCRBZLjDDAOD+lCCZNCDKiZ6AGSmlBDakA0wD7JdUGhDFngCRHomeV9D5AQ2dgfq4rUuQaDfAyA09\nATOS6hJo7A6wG2C/hLsBRuwJEDWInYEkuCUlSCUNANzTEzAjqZRAQzogTANsJ6mkAQB7AkSaBXYY\niHKyS5BsN8DIbT0BMxLuEljYHWA3wH5JdwOM2BMg0irwwwBgf0qQahoQ5daegBkJpQQWpANMA+yX\nchoQxZ4AkX6JnlfQ+QEbOwP10d0lSKUbYOSVnoAZproEKXQH2A2wX0rdACP2BIhMY2fAQrpSAqvS\nAMBbPQEzTKUESaQDwjTAdmJVGgCwJ0BkIw4D9bCyS2BFN8DIiz0BMxrtEiTQHWA3wH6WdAOM2BMg\nsg2HgQakmhJYmQZEebknYEaDKYGJdIBpgP0sTQOi2BMgslei5xV0fsAFnYH6JNolsKobYOSnnoAZ\ncbsEDXQH2A2wn2XdACP2BIiSws6ADcymBDrSAMB/PQEz4qYE7dqhOiYdEKYBthMdaQDAngCRQzgM\nJKihLoHV3QAjv/YEzKjTJfjP/8S2b74Bfv97dgMcYHk3wIg9ASJHBOunikWiKcHw7sNR9GoRrvjN\nFbi9xRysnPYAzkkPYcUK64YA4LuewILsbF/2BMyIpgTDs7JQtGULrli0CDNeXoLHdz2ItHPSsXzE\ncg4BmokISvbvx+StW5GuFJb36GHdEAB81xNYsIA9ASKbMRlIQf/O/bF62Ea0+6IIL1Y+iPT7rsOf\n1n1q6SBQcewYJnzyCca2a4fxF15o3Rt7VP9WrbA65yJkV5bhp2PuRsvLnsBrRWUcBDTTmgYAQEUF\nMGECMHYsMH68de9LRKZwGEhStBvw/asycWxZMWZ1eQuZ//Y5fvh763YvDGJPoCHRbsD3//tyfPnF\nk1j2wrOolkz03/Sp+TshUkK0dQOM2BMgchyHgSTE6wZMG2v9PQ6C3BOIFa8bMHzST7Bx3Djcc+SI\nuXscUEK0pwFR7AkQOY7DQAIau1LAyt0L/b6fgFkNXimQl4fMgQNR/B//gbd69jR/J0RqkC1pQBT3\nEyByh0SvRdT5ARfvM5DovgGp3OMgaPsJ1MfUvgGGfQdM3eOAGqRl34D6cD8BIktxnwGNkt03INmU\ngD2BBPcNMOxKmAmYvxMi1SJ2pgEAewJELsNhoAFW7BuQ6D0Ogt4TSGrfgJh7FjR6jwOqxbZugBF7\nAkSuwmEgDqt3ETSbEgS5J5BQGhArzj0LTN0JMeBsTwOi2BMgcp9Ezyvo/IALOgM67ilgVF+XIMg9\nAUvuKdDAPQvYJajL1m6AEXsCRNqk0hlQ4qLflJRSuQDC4XAYubm5th5bJPLb/+TJQHo68JvfWLuL\nYKy1O9ei6NUi7Du2D7MG/AqL0r6P0wK8l5sbmNMDIoKSj0owedVkpKel49mbn01t86DBg4EtW4CP\nPwbifA/XHjqEoi1bsPfbbzGnSxdM6tABoYCdq5aYXQSfvewy/acEoqqqgB/8IJLevPceTw8QWays\nrAx5eXkAkCciZYm8lqcJoPeeAvUxdgke2bETm49+jSe/d15gBgEt9xSI6Q7ECnqXwJFugBF7AkSu\nFehhQNcdBs3KPCcTP8j9X0D7W9D6i8W4fVG+ZbsXupXovMNgnO5ArCB2CcSpboARewJErhbYYcCJ\nNCCW8b4D20fMs3T3Qjey5Q6DjaQDUUFJCRxPAwDed4DIAwI3DDidBkTF7idwXtPzLNu90G20pgGx\nTKQDUX5OCVyRBgDcT4DIIwI1DLghDYiqbz+BRPclcDtb0oBYJtOBKL+lBK5IA6LYEyDyhEAMA25J\nA6Ia20/AynscOMXWNCBWAulAlB9SAtekAVHsCRB5R6LXIur8gIZ9BnTvG5CoRPcTSOUeB06xZN+A\nVDWw70BjvLgvgWP7BtSH+wkQ2Y73JojDbWkAkNx9B7yUEoiTaUCsJNKBKC+lBOK2NABgT4DIg3w5\nDLipG2CUyn0H3N4lcKQb0JgEuwOx3N4lcFU3wIg9ASLP8dUw4MY0IMqK+w64MSVwVRoQK4V0IMqN\nKYEr04Ao9gSIvCnR8wo6P5BCZ8Bt3QAjHfcdcEOXwBXdgMak0B2I5YYugeu6AUbsCRA5KtCdATen\nAUByPQEznEwJxM1pQCwL0oEoJ1MCcXMaALAnQORxnh4G3NoNMEqlJ2CG3V0CV3YDGpNidyCW3V0C\n13YDjNgTIPI0Tw4Dbk8DoqzoCZhhR0rgqTQgloXpQJQdKYHr04Ao9gSIvC/R8wo6P2CiM+DmboCR\njp6AGTq6BJ7oBjTGwu5ALB1dAld3A4zYEyByjUB0BrySBgD6egJmWJkSiJfTgFga0oEoK1MC8Uoa\nALAnQOQj2ocBpdQDSqkdSqnjSqn1Sqleib6HF7oBRrp7Amak2iXwZDegMRZ3B2Kl2iXwRDfAiD0B\nIt/QOgwopUYCeALAdABXA/gXgL8opUz9aumlNCDKrp6AGcmkBL5KA2JpTAeikkkJPJUGRLEnQOQv\niZ5XSOQDwHoAzxg+VwB2A/hJPc8/2xnwSjfAyKmegBlmugS+6AY0RmN3IJaZLoFnugFG7AkQuZIr\nOwNKqXQAeQD+ahg8BMAaAL0beu3rr3srDQCc7QmY0VBKIH5OA2LZkA5ENZQSiBfTAIA9ASKf0nlC\nOwtAGoB9MY/vA5Dd0AunTQMKC4HiYvcPAVHRnsCG3FzHegJmRLsEU9dMxYN/fhDLypeh5bktsXLL\nSozqOQrFNxb7cwgwmj4dyM+PdAfGjNF+uGiXYOqOHZiybRuWHTiAVk2aYOXBgxjVti2Ku3Z1/xAQ\nFe0JbNjAngCRjyjRtHuaUqo9gC8A9BaRfxoe/xWAfiJSJx1QSuUCCHfv3g+XXNKi1tcKCwtRWFio\nZa2pWrx3L8ZWVGBBdjaK2rd3ejmmrd25FkWvFuHYqWN49uZnvV8QTMTgwcCWLcDHHwM2Dm9rDx1C\n0ZYtOHbmDJ697DJ3FwRjLV4MjB0LLFgAFBU5vRqiQCstLUVpaWmtxw4fPoy3334bAPJEpCyR99M5\nDKQDqAIwXERWGh5fBKCFiNS5HiA6DITDYeTm5mpZl9Uqjh1DfjiM29q0wQs5Oa47PdCYk6dP4nT1\naTQ/p7nTS7FXOBxJB156yZZ0wOhkdTVOi6B5Wpqtx01JRUXk+3XbbcALL/D0AJELlZWVIS8vD0hi\nGNDWGRCRUwDCAK6PPqYiPymvB/CuruPaye09ATOaNmkavEEAsLU7EKtpKOStQYA9ASLf073PwJMA\n7lNKjVNK5QD4DYAMAIs0H9cWbthPgFKged8B3+B+AkS+p3UYEJFXADwCYBaADwBcAaBARA7oPK4d\n3LSfACXJwXTAM7ifAFEgaN+BUETmi0hnEWkmIr1F5H3dx9St4tgxTPjkE4xt1w7jL7zQ6eVQKpgO\n1K+iApgwIVIaHD/e6dUQkUaeuTeBW/ihJ0AGTAfiY0+AKFA4DCSIPQEfYjpQF3sCRIHCYSAB7An4\nFNOB2tgTIAocDgMmsSfgc0wHItgTIAokDgMmsCcQAEwH2BMgCjAOAyawJxAQQU8H2BMgCiwOA41g\nTyBAgpwOsCdAFGgcBhrAnkAABTEdYE+AKPA4DNSDPYGAClo6wJ4AEYHDQL3YEwiwIKUD7AkQETgM\nxMWeQMAFJR1gT4CIanAYiMGeAAHwfzrAngARGXAYMGBPgM7yczrAngARxeAwYMCeANXi13SAPQEi\nisFhoAZ7AlSHH9MB9gSIKA4OA2BPgBrgp3SAPQEiqkfghwH2BKhBfkkH2BMgogYEfhhgT4Aa5Yd0\ngD0BImpAoIcB9gTIFK+nA+wJEFEjAjsMsCdACfFqOsCeABGZEMhhgD0BSpgX0wH2BIjIpEAOA+wJ\nUFK8lg6wJ0BEJgVuGGBPgJLmpXSAPQEiSkCghgH2BChlXkgH2BMgogQFZhhgT4As4fZ0gD0BIkpC\nYIYB9gTIMm5OB9gTIKIkBGIYYE+ALOXWdIA9ASJKku+HAfYESAu3pQPsCRBRCnw9DLAnQNq4KR1g\nT4CIUuTrYYA9AdLKLekAewJElCLfDgPsCZB2bkgH2BMgIgv4chhgT4Bs42Q6wJ4AEVnEd8MAewJk\nK6fSAfYEiMhCvhsG2BMg2zmRDrAnQEQW8tUwwJ4AOcLudIA9ASKymG+GAfYEyFF2pQPsCRCRBr4Y\nBtgTIMfZkQ6wJ0BEmvhiGGBPgFxBdzrAngARaeL5YYA9AXINnekAewJEpJGnhwH2BMh1dKQD7AkQ\nkWaeHQbYEyBXsjodYE+AiGzg2WGAPQFyLSvTAfYEiMgGnhwG2BMgV7MqHWBPgIhs4rlhgD0B8oRU\n0wH2BIjIRp4aBtgTIM9IJR1gT4CIbOapYYA9AfKUZNMB9gSIyGaeGQbYEyDPSSYdYE+AiBzgiWGA\nPQHyrETSAfYEiMghrh8G2BMgTzObDrAnQEQOcv0wwJ4AeZ6ZdIA9ASJykKuHAfYE7FFaWur0Evwt\nTjpQ63vOnoAt+Ofcfvyee4e2YUAp9XOl1D+UUseUUl8l+nr2BOzD/2FtEJMOnP2esydgG/45tx+/\n596hMxlIB/AKgGcTfeFx9gTIb+J1B9gTICKX0HYSXkRmAoBS6q5EX/t/P/8cn2ZlYUNuLnsC5B/T\npwP5+d91B6I9gQ0b2BMgIke58iftHysrsaB3b/YEyF+M6UBGBvDaa8CCBewJEJHj3DYMnAsA1x4+\njCu//BJle/Y4vZ5AOHz4MMrKypxeRjCMHAmMGYPDAMpuugm48kqA33tb8M+5/fg9t1d5eXn0H89N\n9LVKRMw/WalfAvhpA08RAN1E5BPDa+4C8JSItDbx/ncCWGJ6QURERBRrtIiUJPKCRJOB/wvgd408\nZ3uC72n0FwCjAewEcCKF9yEiIgqacwF0RuRnaUISGgZE5CCAg4keJMH3T2iaISIiorPeTeZF2joD\nSqmLALQG0AlAmlLqypovbRORY7qOS0RERIlJqDOQ0Bsr9TsA4+J86ToReVvLQYmIiChh2oYBIiIi\n8gZX35uAiIiI9OMwQEREFHCuGQaUUg8opXYopY4rpdYrpXo5vSY/U0r1VUqtVEp9oZSqVkoNdnpN\nfqeUmqqU2qCUOqKU2qeU+oNS6jKn1+VnSqkJSql/KaUO13y8q5Qa5PS6gkIp9bOav1+edHotfqaU\nml7zfTZ+bE7kPVwxDCilRgJ4AsB0AFcD+BeAvyilshxdmL81B/AhgImIbBZF+vUFUAzgBwB+hMjN\nvFYrpZo5uip/+xyRjdJyAeQBeBPAq0qpbo6uKgBqfqG7H5G/z0m/TQDaAbiw5qNPIi92RYFQKbUe\nwD9FZErN5wqR/4nnisjjji4uAJRS1QCGishKp9cSJDXD7n4A/UTkHafXExRKqYMAHhGRxjZQoyQp\npTIBhAH8GMA0AB+IyP90dlX+pZSaDmCIiOQm+x6OJwNKqXREJva/Rh+TyISyBkBvp9ZFZIOWiKQy\nXzm9kCBQSoWUUqMAZABY5/R6fG4egNdE5E2nFxIgl9ac9v1UKbW4Zq8f09xwo6IsAGkA9sU8vg9A\ntv3LIdKvJv16GsA7IpLQuT1KjFKqJyI//M8FcBTAMBGpcHZV/lUzcF0FIN/ptQTIegB3A9gCoD2A\nGQDeVkr1NLvJnxuGAaIgmg+gO4BrnV5IAFQAuBJACwC3A3hRKdWPA4H1lFLfQ2TI/ZGInHJ6PUEh\nIsZ7EWxSSm0A8BmAEWj8fkIA3DEMVAI4g0jxwagdgL32L4dIL6XUrwHcBKCviPA+3ZqJyGl8dwO1\nD5RS3wcwBZHz2WStPABtAJTVpF9AJPntp5SaBKCpuKGo5nMiclgp9QmArmZf43hnoGZ6DAO4PvpY\nzR+i65HkDReI3KpmEBiCyLbcu5xeT0CFADR1ehE+tQbA5YicJriy5uN9AIsBXMlBwB41Bc6uAEz/\nsuGGZAAAngSwSCkVBrABwMOIlHwWObkoP1NKNUfkD0t0eu9SczOpr0Tkc+dW5l9KqfkACgEMBnBM\nKRVNww6LCG/ZrYFSajaAVQB2ATgPkVuk/xDADU6uy69qzk/X6sAopY4BOCgi5c6syv+UUv8HwGuI\nnBroAGAmgFMASs2+hyuGARF5peYyq1mInB74EECBiBxwdmW+lg/gLUTa7ILIPg8A8AKAIqcW5XMT\nEPler415fDyAF21fTTC0ReTPdHsAhwFsBHADW+62Yhqg3/cAlAC4AMABAO8AuEZEDpp9A1fsM0BE\nRETOcbwzQERERM7iMEBERBRwHAaIiIgCjsMAERFRwHEYICIiCjgOA0RERAHHYYCIiCjgOAwQEREF\nHIcBIiKigOMwQEREFHAcBoiIiALu/wNIS9UYv2igWgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11aa3d710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for al in np.linspace(0, 1, 11):\n",
    "    print(\"dd_dtw(a, b, alpha={}) = {}\".format(al, dd_dtw(a, b, dist=fdtw, alpha=al)))\n",
    "dd_dtw(a, b, dist=sdtw, toplot=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Derivative Transform Distance\n",
    "* uses DTW with conjuction of sine, cosine, and Hilbert transforms\n",
    "* a cosine version transforms vector $a$ into series using\n",
    "$$c_i=\\sum_{k=1}^ma_k\\cos\\left(\\frac{\\pi}{2}\\left(k-\\frac{1}{2}\\right)\\left(i-1\\right)\\right)$$\n",
    "\n",
    "---\n",
    "\n",
    "All papers claim superiority to DTW\n",
    "  * results for CID_DTW and DTDC are __significantly__ better than DTW\n",
    "  * DTDC is significantly more accurate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def cos_transform(a):\n",
    "    def ci(i):\n",
    "        cs = 0\n",
    "        for j in range(a.shape[0]):\n",
    "            cs += a[i] * np.cos((np.pi / 2) * (j - 1. / 2) * (i - 1))\n",
    "        return cs\n",
    "    return np.array([ci(i) for i in range(a.shape[0])])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def dtdc(a, b, dist=fdtw, alpha=1./3, beta=1./3):\n",
    "    c = np.array([(a[i - 1] - a[i]) for i in range(1, a.shape[0])])\n",
    "    d = np.array([(b[i - 1] - b[i]) for i in range(1, b.shape[0])])\n",
    "    e = cos_transform(a)\n",
    "    f = cos_transform(b)\n",
    "    x = dist(a, b)\n",
    "    y = dist(c, d)\n",
    "    z = dist(e, f)\n",
    "    return alpha * x + beta * y + (1 - alpha - beta) * z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7.747546895706431"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dtdc(a, b, dist=fdtw)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dictionary based classifiers\n",
    "* transform series into representative words\n",
    "* similarity based on comparing distribution of words\n",
    "* move w sliding window over series\n",
    "  * each window produces vector of labels\n",
    "  * labels discretized by assigning symbols from an alphabet of set size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bag Of Patterns BOP\n",
    "* based on Symbolic Aggregate Approximation SAX\n",
    "<img src=\"sax1.png\" width=\"100%\" />\n",
    "  * applies SAX to each window to form a word\n",
    "  * if consecutive windows produce identical words, only the first is returned\n",
    "  * distribution of words over a sieries forms a count histogram"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def BOP(X, y, w, l, alpha):\n",
    "    '''\n",
    "    X: n series of length m each\n",
    "    y:\n",
    "    w: window length\n",
    "    l: word length\n",
    "    alpha: alphabet size\n",
    "    '''\n",
    "    # get n histograms of words over series\n",
    "    H = np.zeros((X.shape[0], alpha))\n",
    "    for i in range(X.shape[1]):\n",
    "        for j in range(X.shape[0]):\n",
    "            q = X[i, j:j + w + 1]\n",
    "            r = SAX(q, l, alpha)\n",
    "            if not trivial_match(r, p):\n",
    "                # determine location of r in count matrix h\n",
    "                pos = index(r, h)\n",
    "                H[i, pos] += 1\n",
    "            p = r\n",
    "    return H"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Symbolic Aggregate Approximation Vector Space Model SAXVSM\n",
    "* combines SAX with vector space used in information retrieval\n",
    "  * SAXVSM forms words over __classes__ rather than series\n",
    "  * words are wighted by the ___term frequency - inverse document frequency__ measure\n",
    "  $$tfidf(tf, df)=\\begin{cases}\n",
    "  \\log(1+tf)*log(c/df)&\\mbox{if } df>0\\\\\n",
    "  0\n",
    "  \\end{cases}$$\n",
    "  where \n",
    "    * $tf$ number of times a word appears in a class\n",
    "    * $df$ number of classes a word appears in\n",
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bag of SFA symbols BOSS\n",
    "* uses a truncated Discrete Fourier Tranform DFT over each window\n",
    "* truncated series is discretized through Multiple Coefficient Binning instead of fixed intervals\n",
    "* segmented and discretized series, with breakpoints found, are then processed much like BOP"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def BOSS(X, y, w, l, alpha):\n",
    "    # histograms\n",
    "    H = np.zeros((X.shape[0], alpha))\n",
    "    # brakpoints\n",
    "    for i in range(X.shape[1]):\n",
    "        for j in range(X.shape[0]):\n",
    "            o = X[i, j:j + w + 1]\n",
    "            q = DFT(o, l, alpha, p)\n",
    "            q_first = q[:l/2]\n",
    "            r = SFA_lookup(q_first, B)\n",
    "            if not trivial_match(r, p):\n",
    "                # determine location of r in count matrix h\n",
    "                pos = index(r, h)\n",
    "                H[i, pos] += 1\n",
    "            p = r\n",
    "    return H, B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DTW features\n",
    "* combines DTW distances and SAX histograms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Comparisons\n",
    "\n",
    "<img src=\"dictionary_ranks.png\" width=\"100%\" />\n",
    "\n",
    "* on 19 data sets using single train/test spli\n",
    "  * BOP significantly worse than BOSS and SAXVSM\n",
    "  * no significant differences between DTWF, BOSS, SAXVSM\n",
    "  * no significant differences between BOSS and DTWF (on larger number o sets)\n",
    "    * published results of BOSS are probably over-optimistic\n",
    "      * may include test set optimization "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Shapelets\n",
    "* TS subsequences discriminatory of class membership\n",
    "  * allow phase-independent similarity of series in the same class\n",
    "  * Keogh and Ye used shapelets as criterions in decision trees"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fast Shapelets FS\n",
    "* an extension of a decision tree\n",
    "* discretizes and approximates shapelets\n",
    "  * form a dictionary of SAX words for each possible shapelet\n",
    "  * mask randomly selected letters to reduce dimensionality (random projection)\n",
    "  * build count histograms for each class\n",
    "  * find scores how each shapelet discriminates between classes\n",
    "  * use k-best SAX words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def build_fs(X, y, l, alpha, w, r, k):\n",
    "    '''\n",
    "    r: number of random projections\n",
    "    k: number of SAX words to convert back\n",
    "    '''\n",
    "    b = empty_shapelet()\n",
    "    for l in range(5, X.shape[1]):\n",
    "        sax_list = create_sax_list(X, y, l, alpha, w)\n",
    "        sax_map = random_projection(sax_list, r)\n",
    "        score_list = score_sax(sax_list, sax_map)\n",
    "        s = find_best_sax(score_list, sax_list, k)\n",
    "        if b < s:\n",
    "            b = s\n",
    "    ((X_left, y_left), (X_right, y_right)) = split_data(X, y)\n",
    "    if not is_leaf(X_left, y_left):\n",
    "        build_fs(X_left, y_left)\n",
    "    if not is_leaf(X_right, y_right):\n",
    "        build_fs(X_right, y_right)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Shapelet Transform ST\n",
    "* find best k shapelets in a single run\n",
    "* use shapelets to transform data\n",
    "* transformed data is used to build a classifier\n",
    "  * use a weighted ensemble of classifiers\n",
    "    * k-NN, Naive Bayes, C4.5, SVM, Random Forest, Rotation Forest, Bayesian NN\n",
    "    * each classifier has its weight"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def build_st(X, y, min_l, max_l, k, c):\n",
    "    '''\n",
    "    min_l, max_l: minimum/maximum shapelet length\n",
    "    k: number of shapelets to find\n",
    "    c: number of classes\n",
    "    '''\n",
    "    s = set() # set of shapelets\n",
    "    p = k / c\n",
    "    for k in range(X.shape[0]):\n",
    "        (x_this, y_this) = (X[k, :], y[k])\n",
    "        r = set()\n",
    "        for l in range(min_l, max_l + 1):\n",
    "            w = candidates(x_this, y_this, l)\n",
    "            for a in w:\n",
    "                d = distances(a, (X, y))\n",
    "                q = assess(a, d) # find quality of candidate shapelet\n",
    "                r += (a, q)\n",
    "        r.sort_by_quality()\n",
    "        r.remove_self_similar()\n",
    "        s.merge(p, r)\n",
    "    return s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Learned Shapelets LS\n",
    "* heuristic gradient descent search to discover most informative shapelets\n",
    "  * shapelets are __not restricted__ to being subseries of training data\n",
    "  * init by k-means clustering of candidates\n",
    "  * logistic loss function (with regularization) based on logistic regression for each class\n",
    "* LS learns regression weights together with shapelets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def ls(X, y, k, min_l, scale, reg, eta, n_iter, alpha):\n",
    "    '''\n",
    "    k: number of shapelets\n",
    "    min_l: minimum shapelet length\n",
    "    scale: scale of shapelet length\n",
    "    reg: regularization parameter\n",
    "    eta: learning parameter\n",
    "    n_iter: number of iterations\n",
    "    alpha: softmax parameter\n",
    "    '''\n",
    "    s = init_shapelets_k_means(X, y, k, scale, min_l)\n",
    "    w = init_weights(X, y, k, scale)\n",
    "    m = init_model(X, y)\n",
    "    loss = init_loss()\n",
    "    for k in range(n_iter):\n",
    "        m.update(X, y, alpha, min_l, scale)\n",
    "        loss.update(X, y, m, w)\n",
    "        w, s = update_w_s(X, y, w, s, eta, scale, min_l, loss, reg, alpha)\n",
    "        # check if the model has diverged: for some problems LS \n",
    "        # may not terminate - a maximum number of restarts is needed\n",
    "        if m.diverged():\n",
    "            k = 0\n",
    "            eta = eta / 3."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Known results\n",
    "* evaluated on 33 common subsets from UCR\n",
    "<img src=\"shapelets_ranks.png\" width=\"100%\" />\n",
    "* FS significantly worse\n",
    "* no significant difference between LS and ST\n",
    "  * better adaptation in LS possible"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Interval Based Classifiers\n",
    "* uses features derived from intervals of each series\n",
    "  * $m(m-1)/2$ contiguous intervals in each $m$-length TS\n",
    "  * huge dimension increase\n",
    "  * how to treat each interval\n",
    "* first propositions\n",
    "  1. use only intervals of the lengths of power of two\n",
    "  2. binary features based on thresholds "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Time Series Forest TSF\n",
    "* features calculated as summary statiistics (mean, deviation, slope, etc.)\n",
    "* each member tree uses only $\\sqrt{m}$ intervals\n",
    "  * the split is random"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def tsf(X, y, r, min_l):\n",
    "    '''\n",
    "    r: number of trees\n",
    "    min_l: minimum subseries length\n",
    "    '''\n",
    "    forest = [tree(k) for k in range(r)]\n",
    "    m = X.shape[1]\n",
    "    for i in range(r):\n",
    "        # select cases with 3 (mean, deviation, slope) x sqrt(m) random features \n",
    "        s_features = np.floor(np.sqrt(m))\n",
    "        (X_s, y_s) = split(X, y, s_features)\n",
    "        for j in range(s_features):\n",
    "            # select intervals\n",
    "            a = np.random.randint(1, m - min_l)\n",
    "            b = np.random.randint(s + min_l, m)\n",
    "            for k in range(X.shape[0]):\n",
    "                X_s[k, 3 * (j - 1) + 0] = np.mean(X[k, :], a, b)\n",
    "                X_s[k, 3 * (j - 1) + 1] = np.sdev(X[k, :], a, b)\n",
    "                X_s[k, 3 * (j - 1) + 2] = slope(X[k, :], a, b)\n",
    "        forest[i] = build_tree_classifier(X_s, y_s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Time Series Bag of Features TSBF\n",
    "* extension of TSF in stages\n",
    "  1. generate subseries classification problem\n",
    "    * select $w$ subseries start- and end-points\n",
    "    * for each subseries take _equal_ width intervals and calculate features (mean, deviation, slope)\n",
    "    * concatenate features computed, add original class label to form a new training example\n",
    "  2. produce class probability estimates\n",
    "    * train a random forest on new subseries dataset\n",
    "      * add trees incrementally until an out-of-bag error stops decreasing\n",
    "    * find random forest estimates of class probabilities for each subseries\n",
    "  3. recombine probabilities and form a bag of patterns\n",
    "    * discretize class probability estimates into $b$ equal width bins\n",
    "    * bag together discretized probabilities\n",
    "  4. build final random forest classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Learned Pattern Similarity LPS\n",
    "* subseries become attributes\n",
    "* an internal predictive model built as in TSBF\n",
    "  * regression model in LPS rather than classification as in TSBF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Known results\n",
    "* tested on 44 common data sets\n",
    "<img src=\"intervals_ranks.png\" width=\"120%\" />\n",
    "* no significant differences between the methods\n",
    "  * pairwise comparisons show no differences either\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ensembles\n",
    "* some of approaches given are ensembles by itself"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Elastic Ensemble EE\n",
    "* combination of nearest neighbour classifiers with elastic distance measures\n",
    "  * __ED__ 1-NN with Euclidean distance\n",
    "  * __DTW__ and __DTWCV__ Dynamic Time Warping and DTW with window size through CV\n",
    "  * __DDTW__ and __DDTWCV__ Derivative Time Warping and DTW with window size through CV\n",
    "  * __WDTW__ and __WDDTW__ weighted DTW and Derivative Weighted DTW\n",
    "  * __LCSS__ Longest Common Subsequence\n",
    "  * __ERP__ Edit Distance with Real Penalty\n",
    "  * __TWE__ Time Warp Edit\n",
    "  * __MSM__ Move Split Merge distance\n",
    "* __none__ of the individual members of EE outperforms DTW_CV (Dynamic Time Warping wit Cross Validation with window size set with CV)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Collective Of Transfometion Ensembles COTE\n",
    " * classifiers in time, autocorrealtion, power spectrum, shapelet domain\n",
    " * components of EE\n",
    " * __35__ classifiers pooled with votes weighted by train set cross validation accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"summary_ensembles.png\" width=\"100%\" />\n",
    "* __NN__ nearest neighbour classification\n",
    "* __time__ time domain distance function\n",
    "* __deri__ derivative based distance function\n",
    "* __shpt__ shapelet based\n",
    "* __int__ interval\n",
    "* __dict__ dictionary\n",
    "* __auto__ auto-correlation based\n",
    "* __ens__ ensemble"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comparisons\n",
    "* 85 datasets from UEA [http://timeseriesclassification.com]\n",
    "  * length of series from 24 to 2709\n",
    "  * train set sizes from 16 to 8926\n",
    "  * number of classes from 2 to 60\n",
    "  * data from: sensor readings, motion capture, spectrographs, ECG measurements, electric devices, etc.\n",
    "  \n",
    "  \n",
    "* 100 resample folds on each problem for each classifier\n",
    "* each classifier evaluated 8500 times\n",
    "  * model selection repeated on every training set fold\n",
    "  * 850 000 models were built for each classifier\n",
    "  \n",
    "* methodology proposed by Demsar used when testing for significant differences between classifiers over various datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Benchmark classifiers tested\n",
    "* these classifiers take series as vectors\n",
    "  * do not include time dependecy/auto-correlation/etc.\n",
    "<img src=\"benchmark_classifiers_ranks.png\" width=\"100%\" />\n",
    "  * used: Logistic regression, C4.5, Naive Bayes, Bayesian Network, SVM with Linear kernel, 1-NN with Euclidean Distance, Multi-Layer Perceptron, SVM with Quadratic kernel, 1-NN with DTW, Random Forest (500 trees), Rotation Forest (50 trees)\n",
    "  * RotF and DTW used as benchmarks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### All time-series classifiers\n",
    "<img src=\"all_tsc_ranks.png\" width=\"100%\" />\n",
    "* COTE __is significantly__ better than others\n",
    "  * only nine best shown\n",
    "  * both EE and ST are elements of COTE\n",
    "  * this shows the benefits of ensembling\n",
    "* high rank of BOSS (Bag of SFA Symbols dictionary based approach)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Comparison by classifier type\n",
    "#### time domain\n",
    "* MSM (Move-Split-Merge) has the highest rank\n",
    "#### difference based\n",
    "* CID_DTW (complexity Invariant Distance using DTW) and DTD_C (Derivative Transform Distance with cosine transform) are best\n",
    "* better than benchmark DTW, equal to benchmark RotF\n",
    "#### dictionary based\n",
    "* SAXVSM and BOP ranked very low\n",
    "* at the same time BOSS is one of the best!\n",
    "  * BOSS uses a Fourier transform\n",
    "    * BOP is based on Piecewise Aggregate Approximation PAA\n",
    "  * BOSS uses data driven dicretization\n",
    "    * BOP uses arbitrary break points\n",
    "  * which of these differences is the most important?\n",
    "#### shapelet based\n",
    "* Fast Shapelets FS is the worse of all classifiers\n",
    "* Shapelet Transform ST ranks very high\n",
    "* ST is significantly beter than benchmarks RotF (Rotation Forest) and DTW\n",
    "#### interval based\n",
    "* all interval based TSF, TSBF, and LPS are significantly better than benchmarks\n",
    "* there must be a value in intervals\n",
    "#### ensemble\n",
    "* top seven classifiers __are ensembles__\n",
    "  * diversity, diveristy, diversity!\n",
    "<img src=\"summary_table.png\" width=\"100%\" />\n",
    "\n",
    "\n",
    "* The accuracy __may depend__ on the type of problem\n",
    "  * which algorithm types work best on which type of data?\n",
    "\n",
    "* the computational complexity is very different"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to choose a model?\n",
    "* start with Dynamic Warping, Rotation Forest, Random Forest\n",
    "  * as benchmarks\n",
    "  * for sanity\n",
    "* check the type of discriminatory features\n",
    "  * if embedded in subseries, then choose shapelet ST or interval based TSF\n",
    "  * if in frequency domain, then choose BOSS"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}