Tennis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Continuous Control\n",
    "\n",
    "---\n",
    "\n",
    "In this notebook, you will learn how to use the Unity ML-Agents environment for the second project of the [Deep Reinforcement Learning Nanodegree](https://www.udacity.com/course/deep-reinforcement-learning-nanodegree--nd893) program.\n",
    "\n",
    "### 1. Start the Environment\n",
    "\n",
    "We begin by importing the necessary packages.  If the code cell below returns an error, please revisit the project instructions to double-check that you have installed [Unity ML-Agents](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Installation.md) and [NumPy](http://www.numpy.org/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# when running in Udacity workspace\n",
    "# !pip -q install ./python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from maddpg_agent import Agent\n",
    "from collections import deque\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import random\n",
    "import time\n",
    "import torch\n",
    "from unityagents import UnityEnvironment\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will start the environment!  **_Before running the code cell below_**, change the `file_name` parameter to match the location of the Unity environment that you downloaded.\n",
    "\n",
    "- **Mac**: `\"path/to/Reacher.app\"`\n",
    "- **Windows** (x86): `\"path/to/Reacher_Windows_x86/Reacher.exe\"`\n",
    "- **Windows** (x86_64): `\"path/to/Reacher_Windows_x86_64/Reacher.exe\"`\n",
    "- **Linux** (x86): `\"path/to/Reacher_Linux/Reacher.x86\"`\n",
    "- **Linux** (x86_64): `\"path/to/Reacher_Linux/Reacher.x86_64\"`\n",
    "- **Linux** (x86, headless): `\"path/to/Reacher_Linux_NoVis/Reacher.x86\"`\n",
    "- **Linux** (x86_64, headless): `\"path/to/Reacher_Linux_NoVis/Reacher.x86_64\"`\n",
    "\n",
    "For instance, if you are using a Mac, then you downloaded `Reacher.app`.  If this file is in the same folder as the notebook, then the line below should appear as follows:\n",
    "```\n",
    "env = UnityEnvironment(file_name=\"Reacher.app\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:unityagents:\n",
      "'Academy' started successfully!\n",
      "Unity Academy name: Academy\n",
      "        Number of Brains: 1\n",
      "        Number of External Brains : 1\n",
      "        Lesson number : 0\n",
      "        Reset Parameters :\n",
      "\t\t\n",
      "Unity brain name: TennisBrain\n",
      "        Number of Visual Observations (per agent): 0\n",
      "        Vector Observation space type: continuous\n",
      "        Vector Observation space size (per agent): 8\n",
      "        Number of stacked Vector Observation: 3\n",
      "        Vector Action space type: continuous\n",
      "        Vector Action space size (per agent): 2\n",
      "        Vector Action descriptions: , \n"
     ]
    }
   ],
   "source": [
    "## when running locally\n",
    "env = UnityEnvironment(file_name='Tennis.app')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Environments contain **_brains_** which are responsible for deciding the actions of their associated agents. Here we check for the first brain available, and set it as the default brain we will be controlling from Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the default brain\n",
    "brain_name = env.brain_names[0]\n",
    "brain = env.brains[brain_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Examine the State and Action Spaces\n",
    "\n",
    "In this environment, two agents control rackets to bounce a ball over a net. If an agent hits the ball over the net, it receives a reward of +0.1.  If an agent lets a ball hit the ground or hits the ball out of bounds, it receives a reward of -0.01.  Thus, the goal of each agent is to keep the ball in play.\n",
    "\n",
    "The observation space consists of 8 variables corresponding to the position and velocity of the ball and racket. Two continuous actions are available, corresponding to movement toward (or away from) the net, and jumping. \n",
    "\n",
    "Run the code cell below to print some information about the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of agents: 2\n",
      "Size of each action: 2\n",
      "There are 2 agents. Each observes a state with length: 24\n",
      "The state for the first agent looks like: [ 0.          0.          0.          0.          0.          0.\n",
      "  0.          0.          0.          0.          0.          0.\n",
      "  0.          0.          0.          0.         -6.65278625 -1.5\n",
      " -0.          0.          6.83172083  6.         -0.          0.        ]\n"
     ]
    }
   ],
   "source": [
    "# reset the environment\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "\n",
    "# number of agents\n",
    "num_agents = len(env_info.agents)\n",
    "print('Number of agents:', num_agents)\n",
    "\n",
    "# size of each action\n",
    "action_size = brain.vector_action_space_size\n",
    "print('Size of each action:', action_size)\n",
    "\n",
    "# examine the state space \n",
    "states = env_info.vector_observations\n",
    "state_size = states.shape[1]\n",
    "print('There are {} agents. Each observes a state with length: {}'.format(states.shape[0], state_size))\n",
    "print('The state for the first agent looks like:', states[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Take Random Actions in the Environment\n",
    "\n",
    "In the next code cell, you will learn how to use the Python API to control the agent and receive feedback from the environment.\n",
    "\n",
    "Once this cell is executed, you will watch the agent's performance, if it selects an action at random with each time step.  A window should pop up that allows you to observe the agent, as it moves through the environment.  \n",
    "\n",
    "Of course, as part of the project, you'll have to change the code so that the agent is able to use its experience to gradually choose better actions when interacting with the environment!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Score (max over agents) from episode 1: 0.0\n",
      "Score (max over agents) from episode 2: 0.0\n",
      "Score (max over agents) from episode 3: 0.0\n",
      "Score (max over agents) from episode 4: 0.0\n",
      "Score (max over agents) from episode 5: 0.0\n"
     ]
    }
   ],
   "source": [
    "for i in range(1, 6):                                      # play game for 5 episodes\n",
    "    env_info = env.reset(train_mode=False)[brain_name]     # reset the environment    \n",
    "    states = env_info.vector_observations                  # get the current state (for each agent)\n",
    "    scores = np.zeros(num_agents)                          # initialize the score (for each agent)\n",
    "    while True:\n",
    "        actions = np.random.randn(num_agents, action_size) # select an action (for each agent)\n",
    "        actions = np.clip(actions, -1, 1)                  # all actions between -1 and 1\n",
    "        env_info = env.step(actions)[brain_name]           # send all actions to tne environment\n",
    "        next_states = env_info.vector_observations         # get next state (for each agent)\n",
    "        rewards = env_info.rewards                         # get reward (for each agent)\n",
    "        dones = env_info.local_done                        # see if episode finished\n",
    "        scores += env_info.rewards                         # update the score (for each agent)\n",
    "        states = next_states                               # roll over states to next time step\n",
    "        if np.any(dones):                                  # exit loop if episode finished\n",
    "            break\n",
    "    print('Score (max over agents) from episode {}: {}'.format(i, np.max(scores)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. It's Your Turn!\n",
    "\n",
    "Now it's your turn to train your own agent to solve the environment!  When training the environment, set `train_mode=True`, so that the line for resetting the environment looks like the following:\n",
    "```python\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "SOLVED_SCORE = 0.5\n",
    "CONSEC_EPISODES = 100\n",
    "PRINT_EVERY = 10\n",
    "ADD_NOISE = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# MADDPG function\n",
    "\n",
    "def maddpg(n_episodes=2000, max_t=1000, train_mode=True):\n",
    "    \"\"\"Multi-Agent Deep Deterministic Policy Gradient (MADDPG)\n",
    "    \n",
    "    Params\n",
    "    ======\n",
    "        n_episodes (int)      : maximum number of training episodes\n",
    "        max_t (int)           : maximum number of timesteps per episode\n",
    "        train_mode (bool)     : if 'True' set environment to training mode\n",
    "\n",
    "    \"\"\"\n",
    "    scores_window = deque(maxlen=CONSEC_EPISODES)\n",
    "    scores_all = []\n",
    "    moving_average = []\n",
    "    best_score = -np.inf\n",
    "    best_episode = 0\n",
    "    already_solved = False    \n",
    "\n",
    "    for i_episode in range(1, n_episodes+1):\n",
    "        env_info = env.reset(train_mode=train_mode)[brain_name]         # reset the environment\n",
    "        states = np.reshape(env_info.vector_observations, (1,48)) # get states and combine them\n",
    "        agent_0.reset()\n",
    "        agent_1.reset()\n",
    "        scores = np.zeros(num_agents)\n",
    "        while True:\n",
    "            actions = get_actions(states, ADD_NOISE)           # choose agent actions and combine them\n",
    "            env_info = env.step(actions)[brain_name]           # send both agents' actions together to the environment\n",
    "            next_states = np.reshape(env_info.vector_observations, (1, 48)) # combine the agent next states\n",
    "            rewards = env_info.rewards                         # get reward\n",
    "            done = env_info.local_done                         # see if episode finished\n",
    "            agent_0.step(states, actions, rewards[0], next_states, done, 0) # agent 1 learns\n",
    "            agent_1.step(states, actions, rewards[1], next_states, done, 1) # agent 2 learns\n",
    "            scores += np.max(rewards)                          # update the score for each agent\n",
    "            states = next_states                               # roll over states to next time step\n",
    "            if np.any(done):                                   # exit loop if episode finished\n",
    "                break\n",
    "\n",
    "        ep_best_score = np.max(scores)\n",
    "        scores_window.append(ep_best_score)\n",
    "        scores_all.append(ep_best_score)\n",
    "        moving_average.append(np.mean(scores_window))\n",
    "\n",
    "        # save best score                        \n",
    "        if ep_best_score > best_score:\n",
    "            best_score = ep_best_score\n",
    "            best_episode = i_episode\n",
    "        \n",
    "        # print results\n",
    "        if i_episode % PRINT_EVERY == 0:\n",
    "            print('Episodes {:0>4d}-{:0>4d}\\tMax Reward: {:.3f}\\tMoving Average: {:.3f}'.format(\n",
    "                i_episode-PRINT_EVERY, i_episode, np.max(scores_all[-PRINT_EVERY:]), moving_average[-1]))\n",
    "\n",
    "        # determine if environment is solved and keep best performing models\n",
    "        if moving_average[-1] >= SOLVED_SCORE:\n",
    "            if not already_solved:\n",
    "                print('<-- Environment solved in {:d} episodes! \\\n",
    "                \\n<-- Moving Average: {:.3f} over past {:d} episodes'.format(\n",
    "                    i_episode-CONSEC_EPISODES, moving_average[-1], CONSEC_EPISODES))\n",
    "                already_solved = True\n",
    "                # save weights\n",
    "                torch.save(agent_0.actor_local.state_dict(), 'models/checkpoint_actor_0.pth')\n",
    "                torch.save(agent_0.critic_local.state_dict(), 'models/checkpoint_critic_0.pth')\n",
    "                torch.save(agent_1.actor_local.state_dict(), 'models/checkpoint_actor_1.pth')\n",
    "                torch.save(agent_1.critic_local.state_dict(), 'models/checkpoint_critic_1.pth')\n",
    "            elif ep_best_score >= best_score:\n",
    "                print('<-- Best episode so far!\\\n",
    "                \\nEpisode {:0>4d}\\tMax Reward: {:.3f}\\tMoving Average: {:.3f}'.format(\n",
    "                i_episode, ep_best_score, moving_average[-1]))\n",
    "                # save weights\n",
    "                torch.save(agent_0.actor_local.state_dict(), 'models/checkpoint_actor_0.pth')\n",
    "                torch.save(agent_0.critic_local.state_dict(), 'models/checkpoint_critic_0.pth')\n",
    "                torch.save(agent_1.actor_local.state_dict(), 'models/checkpoint_actor_1.pth')\n",
    "                torch.save(agent_1.critic_local.state_dict(), 'models/checkpoint_critic_1.pth')\n",
    "            elif (i_episode-best_episode) >= 200:\n",
    "                # stop training if model stops converging\n",
    "                print('<-- Training stopped. Best score not matched or exceeded for 200 episodes')\n",
    "                break\n",
    "            else:\n",
    "                continue\n",
    "            \n",
    "    return scores_all, moving_average"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_actions(states, add_noise):\n",
    "    '''gets actions for each agent and then combines them into one array'''\n",
    "    action_0 = agent_0.act(states, add_noise)    # agent 0 chooses an action\n",
    "    action_1 = agent_1.act(states, add_noise)    # agent 1 chooses an action\n",
    "    return np.concatenate((action_0, action_1), axis=0).flatten()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# initialize agents\n",
    "agent_0 = Agent(state_size, action_size, num_agents=1, random_seed=0)\n",
    "agent_1 = Agent(state_size, action_size, num_agents=1, random_seed=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Best Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episodes 0000-0010\tMax Reward: 0.100\tMoving Average: 0.040\n",
      "Episodes 0010-0020\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 0020-0030\tMax Reward: 0.000\tMoving Average: 0.017\n",
      "Episodes 0030-0040\tMax Reward: 0.100\tMoving Average: 0.018\n",
      "Episodes 0040-0050\tMax Reward: 0.100\tMoving Average: 0.016\n",
      "Episodes 0050-0060\tMax Reward: 0.100\tMoving Average: 0.017\n",
      "Episodes 0060-0070\tMax Reward: 0.100\tMoving Average: 0.019\n",
      "Episodes 0070-0080\tMax Reward: 0.100\tMoving Average: 0.020\n",
      "Episodes 0080-0090\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 0090-0100\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 0100-0110\tMax Reward: 0.100\tMoving Average: 0.023\n",
      "Episodes 0110-0120\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 0120-0130\tMax Reward: 0.100\tMoving Average: 0.029\n",
      "Episodes 0130-0140\tMax Reward: 0.100\tMoving Average: 0.034\n",
      "Episodes 0140-0150\tMax Reward: 0.100\tMoving Average: 0.042\n",
      "Episodes 0150-0160\tMax Reward: 0.200\tMoving Average: 0.046\n",
      "Episodes 0160-0170\tMax Reward: 0.100\tMoving Average: 0.048\n",
      "Episodes 0170-0180\tMax Reward: 0.100\tMoving Average: 0.052\n",
      "Episodes 0180-0190\tMax Reward: 0.100\tMoving Average: 0.051\n",
      "Episodes 0190-0200\tMax Reward: 0.300\tMoving Average: 0.056\n",
      "Episodes 0200-0210\tMax Reward: 0.100\tMoving Average: 0.059\n",
      "Episodes 0210-0220\tMax Reward: 0.100\tMoving Average: 0.060\n",
      "Episodes 0220-0230\tMax Reward: 0.200\tMoving Average: 0.064\n",
      "Episodes 0230-0240\tMax Reward: 0.100\tMoving Average: 0.061\n",
      "Episodes 0240-0250\tMax Reward: 0.200\tMoving Average: 0.057\n",
      "Episodes 0250-0260\tMax Reward: 0.200\tMoving Average: 0.058\n",
      "Episodes 0260-0270\tMax Reward: 0.300\tMoving Average: 0.063\n",
      "Episodes 0270-0280\tMax Reward: 0.200\tMoving Average: 0.067\n",
      "Episodes 0280-0290\tMax Reward: 0.200\tMoving Average: 0.073\n",
      "Episodes 0290-0300\tMax Reward: 0.200\tMoving Average: 0.078\n",
      "Episodes 0300-0310\tMax Reward: 0.200\tMoving Average: 0.084\n",
      "Episodes 0310-0320\tMax Reward: 0.200\tMoving Average: 0.087\n",
      "Episodes 0320-0330\tMax Reward: 0.200\tMoving Average: 0.091\n",
      "Episodes 0330-0340\tMax Reward: 0.300\tMoving Average: 0.097\n",
      "Episodes 0340-0350\tMax Reward: 0.200\tMoving Average: 0.099\n",
      "Episodes 0350-0360\tMax Reward: 0.200\tMoving Average: 0.103\n",
      "Episodes 0360-0370\tMax Reward: 0.300\tMoving Average: 0.105\n",
      "Episodes 0370-0380\tMax Reward: 0.600\tMoving Average: 0.114\n",
      "Episodes 0380-0390\tMax Reward: 0.800\tMoving Average: 0.120\n",
      "Episodes 0390-0400\tMax Reward: 0.500\tMoving Average: 0.121\n",
      "Episodes 0400-0410\tMax Reward: 0.200\tMoving Average: 0.124\n",
      "Episodes 0410-0420\tMax Reward: 0.300\tMoving Average: 0.131\n",
      "Episodes 0420-0430\tMax Reward: 0.400\tMoving Average: 0.136\n",
      "Episodes 0430-0440\tMax Reward: 0.200\tMoving Average: 0.139\n",
      "Episodes 0440-0450\tMax Reward: 0.700\tMoving Average: 0.152\n",
      "Episodes 0450-0460\tMax Reward: 0.400\tMoving Average: 0.160\n",
      "Episodes 0460-0470\tMax Reward: 0.200\tMoving Average: 0.161\n",
      "Episodes 0470-0480\tMax Reward: 0.400\tMoving Average: 0.154\n",
      "Episodes 0480-0490\tMax Reward: 0.500\tMoving Average: 0.160\n",
      "Episodes 0490-0500\tMax Reward: 0.300\tMoving Average: 0.164\n",
      "Episodes 0500-0510\tMax Reward: 0.600\tMoving Average: 0.174\n",
      "Episodes 0510-0520\tMax Reward: 0.200\tMoving Average: 0.172\n",
      "Episodes 0520-0530\tMax Reward: 0.600\tMoving Average: 0.172\n",
      "Episodes 0530-0540\tMax Reward: 0.800\tMoving Average: 0.176\n",
      "Episodes 0540-0550\tMax Reward: 0.800\tMoving Average: 0.179\n",
      "Episodes 0550-0560\tMax Reward: 0.800\tMoving Average: 0.182\n",
      "Episodes 0560-0570\tMax Reward: 0.500\tMoving Average: 0.190\n",
      "Episodes 0570-0580\tMax Reward: 0.400\tMoving Average: 0.194\n",
      "Episodes 0580-0590\tMax Reward: 0.500\tMoving Average: 0.185\n",
      "Episodes 0590-0600\tMax Reward: 0.600\tMoving Average: 0.189\n",
      "Episodes 0600-0610\tMax Reward: 1.700\tMoving Average: 0.199\n",
      "Episodes 0610-0620\tMax Reward: 0.900\tMoving Average: 0.213\n",
      "Episodes 0620-0630\tMax Reward: 0.300\tMoving Average: 0.218\n",
      "Episodes 0630-0640\tMax Reward: 0.400\tMoving Average: 0.225\n",
      "Episodes 0640-0650\tMax Reward: 1.200\tMoving Average: 0.254\n",
      "Episodes 0650-0660\tMax Reward: 2.500\tMoving Average: 0.291\n",
      "Episodes 0660-0670\tMax Reward: 4.200\tMoving Average: 0.346\n",
      "Episodes 0670-0680\tMax Reward: 0.900\tMoving Average: 0.375\n",
      "Episodes 0680-0690\tMax Reward: 2.300\tMoving Average: 0.421\n",
      "Episodes 0690-0700\tMax Reward: 1.800\tMoving Average: 0.440\n",
      "<-- Environment solved in 607 episodes!                 \n",
      "<-- Moving Average: 0.544 over past 100 episodes\n",
      "Episodes 0700-0710\tMax Reward: 5.200\tMoving Average: 0.598\n",
      "Episodes 0710-0720\tMax Reward: 1.200\tMoving Average: 0.604\n",
      "Episodes 0720-0730\tMax Reward: 2.600\tMoving Average: 0.665\n",
      "Episodes 0730-0740\tMax Reward: 0.900\tMoving Average: 0.673\n",
      "Episodes 0740-0750\tMax Reward: 2.700\tMoving Average: 0.688\n",
      "Episodes 0750-0760\tMax Reward: 3.800\tMoving Average: 0.695\n",
      "Episodes 0760-0770\tMax Reward: 2.700\tMoving Average: 0.706\n",
      "Episodes 0770-0780\tMax Reward: 4.800\tMoving Average: 0.727\n",
      "<-- Best episode so far!                \n",
      "Episode 0785\tMax Reward: 5.200\tMoving Average: 0.780\n",
      "Episodes 0780-0790\tMax Reward: 5.200\tMoving Average: 0.759\n",
      "Episodes 0790-0800\tMax Reward: 4.000\tMoving Average: 0.827\n",
      "Episodes 0800-0810\tMax Reward: 1.700\tMoving Average: 0.719\n",
      "<-- Best episode so far!                \n",
      "Episode 0817\tMax Reward: 5.200\tMoving Average: 0.792\n",
      "Episodes 0810-0820\tMax Reward: 5.200\tMoving Average: 0.839\n",
      "Episodes 0820-0830\tMax Reward: 1.400\tMoving Average: 0.814\n",
      "Episodes 0830-0840\tMax Reward: 4.500\tMoving Average: 0.927\n",
      "Episodes 0840-0850\tMax Reward: 1.500\tMoving Average: 0.910\n",
      "Episodes 0850-0860\tMax Reward: 3.200\tMoving Average: 0.921\n",
      "Episodes 0860-0870\tMax Reward: 0.400\tMoving Average: 0.839\n",
      "Episodes 0870-0880\tMax Reward: 0.500\tMoving Average: 0.781\n",
      "<-- Best episode so far!                \n",
      "Episode 0884\tMax Reward: 5.200\tMoving Average: 0.906\n",
      "Episodes 0880-0890\tMax Reward: 5.200\tMoving Average: 0.890\n",
      "Episodes 0890-0900\tMax Reward: 1.500\tMoving Average: 0.815\n",
      "<-- Training stopped. Best score not topped for 200 episodes\n"
     ]
    }
   ],
   "source": [
    "# run the training loop\n",
    "\n",
    "# BUFFER_SIZE = int(1e6)  # replay buffer size\n",
    "# BATCH_SIZE = 128        # minibatch size\n",
    "# LR_ACTOR = 1e-3         # learning rate of the actor\n",
    "# LR_CRITIC = 1e-3        # learning rate of the critic\n",
    "# WEIGHT_DECAY = 0        # L2 weight decay\n",
    "# LEARN_EVERY = 1         # learning timestep interval\n",
    "# LEARN_NUM = 1           # number of learning passes\n",
    "# GAMMA = 0.99            # discount factor\n",
    "# TAU = 7e-2              # for soft update of target parameters\n",
    "# OU_SIGMA = 0.2          # Ornstein-Uhlenbeck noise parameter, volatility\n",
    "# OU_THETA = 0.12         # Ornstein-Uhlenbeck noise parameter, speed of mean reversion\n",
    "# EPS_START = 5.5         # initial value for epsilon in noise decay process in Agent.act()\n",
    "# EPS_EP_END = 250        # episode to end the noise decay process\n",
    "# EPS_FINAL = 0           # final value for epsilon after decay\n",
    "\n",
    "scores, avgs = maddpg()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJztnXd8HMX5h5+5Uy+WbEuucgX3ipEBA6a3UEIJYEoILSEECCWFACGhBAKB9F8gBEgCCWA6hmCasU0HYxsbdxsX2ZZsyypWl67O748rurJXdSeddO/jjz7e252dmV2dvvvuO++8o7TWCIIgCH0fU093QBAEQegeRPAFQRDSBBF8QRCENEEEXxAEIU0QwRcEQUgTRPAFQRDSBBF8QRCENEEEXxAEIU0QwRcEQUgTMnq6A76UlJTo0aNH93Q3BEEQeg0rV66s1VqXRlM2pQR/9OjRrFixoqe7IQiC0GtQSu2Mtqy4dARBENIEEXxBEIQ0QQRfEAQhTUgpH74RNpuNyspKOjo6eroraUNOTg5lZWVkZmb2dFcEQUggKS/4lZWVFBYWMnr0aJRSPd2dPo/Wmrq6OiorKxkzZkxPd0cQhASS8i6djo4OBg4cKGLfTSilGDhwoLxRCUIfJOUFHxCx72bkfgtC36RXCL4gCH2Ld9btpa7FkvR2Pt9Wx7aalqjKLlhVRavFnvA+bN3fzLLtdQmvNx5E8KNAKcV3v/td72e73U5paSlnnnmmX7lzzjmHI444wm/f3XffzfDhw5k5cybjxo3jvPPOY8OGDd7jxx13HBMmTGD69OlMnDiRG264gYaGBu9xs9nMzJkzmTp1KhdccAFtbW0AVFdXc8kllzB27FgOPfRQ5syZw2uvvZaMyxeEhNLQZuXaZ77iqqeTP8ny4ie+4MQ/fBix3Ne7G7j5hdX88rW1Ce/DSX/8iHmPf5HweuNBBD8K8vPzWbduHe3t7QAsWrSI4cOH+5VpaGhg5cqVNDY2sn37dr9jt9xyC6tXr+abb75h3rx5nHDCCdTU1HiPP/vss6xZs4Y1a9aQnZ3N2Wef7T2Wm5vL6tWrWbduHVlZWTz22GNorTnnnHM45phj2L59OytXruT555+nsrIyiXdBEBKDzaEBqKxv6+GedGKxOwGoPNDewz1JLiL4UXL66aezcOFCAObPn8/FF1/sd/zVV1/lrLPO4qKLLuL5558PWc+8efM45ZRTeO6554KOZWVl8dBDD7Fr1y6+/vrroONz585l69atLFmyhKysLK699lrvsVGjRvHjH/843ssThLQmw+wat7I5dQ/3JLmkfFimL/f8bz0b9jQltM7Jw/px11lTIpa76KKLuPfeeznzzDNZs2YNV111FR9//LH3+Pz58/n1r3/N4MGD+c53vsMdd9wRsq5Zs2axadMmw2Nms5kZM2awadMmZsyY4d1vt9t5++23Oe2001i/fj2zZs2K4SoFQQhHltll+9odzh7uSXJJqoWvlKpQSq1VSq1WSvXqrGjTp0+noqKC+fPnc/rpp/sdq66u5ptvvuHoo49m/PjxZGZmsm7dupB1aR3eivA93t7ezsyZMykvL2fkyJFcffXVQeWvv/56ZsyYwezZs2O8KkEQoNPCtzvEwu8qx2utaxNRUTSWeDL59re/zc9+9jM++OAD6uo6R91ffPFFDhw44J2o1NTUxPz587n//vsN61m1ahXl5eWGxxwOB2vXrmXSpElApw/flylTpvDKK694Pz/yyCPU1taGrFMQhPBkmDwuHbHwBTdXXXUVd911F9OmTfPbP3/+fN555x0qKiqoqKjwDqIa8corr/Dee+8FjQGAK43E7bffzogRI5g+fXrIfpxwwgl0dHTw97//3bvPE70jCELsmJRY+IlAA+8ppTTwD63140luL6mUlZVx4403+u2rqKhg586dfuGYY8aMoaioiGXLlgHwpz/9iWeeeYbW1lamTp3KkiVLKC3tXK/g0ksvJTs7G4vFwkknncTrr78eth9KKRYsWMAtt9zCQw89RGlpKfn5+fzud79L4NUKQvrR1334yRb8o7XWVUqpQcAipdQmrfVHvgWUUtcA1wCMHDkyyd2Jj5aW4Ikbxx13HMcddxwAVVVVQce/+uorAA4//HDuvvvukHV/8MEHMbcNMHTo0LDRQIKQqqTyRO6+HqWTVJeO1rrK/f9+4DXgMIMyj2uty7XW5b5WryAIQnfhkXlbH7fwkyb4Sql8pVShZxs4BQgduiIIQlqRira0+PDjZzDwmjsRVwbwnNb6nSS2JwiCEBeeSOi+buEnTfC11tuBGRELCoIgpAh28eELgiD0dVxC7xDBFwRBSH1aLHZ2p1BCtlREBL+HeOyxx/jPf/7T090QhG4nWVGZ33n0M+Y+tDSucyNkO+kz9KrkaX0J30yXgiB0nc3VzT3dhZRHLPwIVFRUMHHiRK644grGjx/PpZdeyvvvv89RRx3FuHHj+PLLLwGor6/nnHPOYfr06RxxxBGsWbMGp9PJ6NGj/RY0GTduHNXV1dx99938/ve/B1yTuH7xi19w2GGHMX78eG8Wzra2Ni688EImT57Mueeey+GHH86KFcE56O69915mz57N1KlTueaaa9Bas2nTJg47rHPaQ0VFhTclxFtvvcXEiRM59NBDufHGG4MWchGEdCNNDPxeZuHffDMEJBLrMjNnwp//HLbI1q1beemll/jXv/7F7Nmzee655/jkk0944403+O1vf8uCBQu46667OOSQQ1iwYAFLlizhe9/7HqtXr+bss8/mtdde48orr2TZsmWMGjWKwYMHB7Vht9v58ssveeutt7jnnnt4//33efTRR+nfvz8bNmxg3bp1zJw507B/N9xwA7/+9a8BuOyyy3jzzTc566yzsFqt7NixgzFjxvDCCy8wb948Ojo6+OEPf8hHH33EmDFjDHP6CILQNxELPwrGjBnDtGnTMJlMTJkyhRNPPBGlFNOmTaOiogKATz75hMsuuwxwJTerq6ujqamJefPm8cILLwDw/PPPM2/ePMM2zjvvPAAOPfRQvzovuugiAKZOnRoyodrSpUs5/PDDmTZtGkuWLGH9+vUAXHjhhd62PYK/adMmxo4d683sKYIvCOLDT00iWOLJIjs727ttMpm8n00mE3Z7+EWP58yZw9atW6mpqWHBggXceeedYdswm80R6/Slo6OD6667jhUrVjBixAjuvvtuOjo6ANfqWhdccAHnnXceSinGjRsXlGpZEHqKSOtCCIlHLPwEMXfuXJ599lnAlRCtpKSEfv36oZTi3HPP5Sc/+QmTJk1i4MCBUdd51FFH8eKLLwKwYcMG1q4NXmDZI+4lJSW0tLTw8ssve48ddNBBmM1mfvOb33jfLCZMmMD27du9bxGeNwBB6C5SUeZ1SvYq8fQuCz+Fufvuu7nqqquYPn06eXl5PP30095j8+bNY/bs2Tz11FMx1Xnddddx+eWXM3nyZCZOnMiUKVMoKiryK1NcXMwPfvADpk6dypAhQ4JWvZo3bx4///nP2bFjB+BaUOXRRx/ltNNOIz8/X1bJEnqMA202HE6N2ZTC6TP7GCqVXqvKy8t1YBTKxo0bvas/pRsOhwObzUZOTg7btm3jpJNOYvPmzWRlZXWp3paWFgoKCtBac/311zNu3DhuueUWvzLpfN+F5FLbYqH8vvcBmP+DI5hzUPRvveEYfdtCACoePCPsPiM27m3iW3/5OKqyiehXIlFKrdRaR7XcnVj4KUxbWxvHH388NpsNrTWPPvpol8Ue4IknnuDpp5/GarVyyCGH8MMf/jABvRWE6PC1MVPFlZJCdm9SEcFPYQoLCw3j7rvKLbfcEmTRC0JPoJI271YwolcM2qaS2ykdkPstJJNUsep9CdenDpuDo3+3hI+21HRjj5JDygt+Tk4OdXV1IkLdhNaauro6cnJyerorgpAS7Kxro/JAO/ct3NDTXekyKe/SKSsro7Kykpqa3v907S3k5ORQVlbW090Q+iopaLuFsyed7oN9wf2U8oKfmZnpnRUqCILQ3XgeBqm8+Hq0pLxLRxCEvkUKGvhh8Vr4fUDxRfAFQegxepOG9oX5YSL4giB0K35x+Cli7kflwxfBFwRB6Nt4ffh9YNBWBF8QhG7FN+Y9VazmcHH4niPi0hEEQejjOPtQmI4IviAIaU84H77nmFj4giCkJbUtFhZvrI7r3FQZqI0W7Z14FTtb97cktjNdRARfEISY+e6Ty7j66RV02Bw93ZWEEO4Z1OnDj13yT/rjh3H1J1mI4AuCEDM7alvjPreXGfg4na4exyP4qYYIviAIPUaqSGi45IxOz6FU6WwXEMEXBKFb6W2Zbz39lUHbKFBKmZVSq5RSbya7LUEQhHiIxocvE6+i4yZgYze0IwhCLyAVDfyowjL7gD8kqZeglCoDzgCeTGY7giD0TnpDBsq+lA8/2c+sPwO3As4ktyMIghCSuhYLh//2fTbtawpRInJqhV7wbIpI0gRfKXUmsF9rvTJCuWuUUiuUUitkVStB6F2konvGiKWba6husvD4R9tjPlfy4UfHUcC3lVIVwPPACUqpZwILaa0f11qXa63LS0tLk9gdQRBSgVR8SITtk6RWiIzW+natdZnWejRwEbBEa/3dZLUnCEL3Ey7LZE9hFPbZlVBQZxdSK6QafWDcWRAEITpCDbyGNfC9yTJ7v+R3yyLmWusPgA+6oy1BELqPeAznZOfD1zqx9Uo+fEEQhD5ENEsc9gWnjgi+IAhxE49nvL7VmpC2qxraabXYg/YnelRB8uELgiAQ32DouY9+lpC2j3pwCec/9nlUZSP1Mtx1aFnEXBAEoefZuDd4IlWik7N1JR9+qiGCLwhC3KReUGZ8hLsOp1j4giAIXScZGhpOvOMRbW9YpgzaCoKQzqTirNl4iCZKRyx8QRDSmxQU/EQ/hDqjdHq/4ovgC4KQ9oRLEeE5JmGZgiCkNV3NpZOUmbZGfepCN53u5O59IbWCCL4gCH2KcC6dkJIdbsUrz7m9X+9F8AVBiJ8+M2gb5piseCUIgpAueLNl9mw3EoEIviAIcdNHDPzwi5jLoK0gCELi0xgkAqMudWVw2SkTrwRBEBJBaohouAeCx4dv6gNq2QcuQRCEniL17PvEL7vo9FaXGg+nriCCLwhC2hBq4DWsD19SKwiCIKRmWGai++R0JmbQNhXGO0TwBUFIe8IuYp6oNnpe70XwBUGIn0T7yxNBspY4lCgdQRDSmy6ra+IfGEauk0jWddglDrvaoQTX0xVE8AVBSBvisdJTwfeeKETwBUGIm65KYTK0NJ4qozmnq1E6qfDgEMEXBKFPkiiBDVVNm9XO22v3JqSN7kIEXxCEuOmqpibD5o2rT1Hk0gk08O94dS0/evYr1u9p7GoT3YYIviAIcZOKUToefIW/K70M9QDZVd8GQLvV0YXauxcRfEEQeoykuLXjqDP8EocuAle88iZVi9K5nwIufBF8QRDiJxVELBTJnjDlGSMwRzkFNxXehpIm+EqpHKXUl0qpr5VS65VS9ySrLUEQeifJiFwJJ6xx5dIJUZ/Hwu9NefIzkli3BThBa92ilMoEPlFKva21/iKJbQqC0I30vM0aHZEnXsV+zJs2uRe5dJIm+Nr16G5xf8x0/6TAJQuCkCokM0rH6O0hkRkvHc7el0UzqT58pZRZKbUa2A8s0lovS2Z7giB0Lz01mSjedkP646NoK1DYtdel03sUP6mCr7V2aK1nAmXAYUqpqYFllFLXKKVWKKVW1NTUJLM7giAkmC7H4Sdxpm3SB22JzaWTCnRLlI7WugFYCpxmcOxxrXW51rq8tLS0O7ojCEIvJ5oFS4wIPWgbRVgmxmGZ0Q7apoIPP5lROqVKqWL3di5wMrApWe0JgtD7SGaoov/Eq/jbCTlo6/Xh98GwTKXU0UqpK93bpUqpMRFOGQosVUqtAZbj8uG/GX9XBUEQXCR6wZLw9YUKy+x9g7ZRRekope4CyoEJwL9xRdw8AxwV6hyt9RrgkAT0URCEFKXLboo4z49m0NZXqLuyeIl3AZSAKjwunWjvQW9y6ZwLfBtoBdBa7wEKk9UpQRB6B6ngpgjESFgj9TP8xCtjnN6TUu8ehCJawbe64+o1gFIqP3ldEgQhXYhXKqM5z1jE47D03RUF1herxZ4Kj4VoBf9FpdQ/gGKl1A+A94EnktctQRB6A6ngpggkvreO2Jc4dIZ4EKQyUfnwtda/V0qdDDTh8uP/Wmu9KKk9EwShzxOvWHanyHpn7gZIv1fww5w7a2QxX+1qcNfT80+GiIKvlDID72utjwdE5AVB8NJTEhbWijfy4Xchl44zxEGHM3ydQW3EVjwpRHTpaK0dgFMpVdQN/REEIY2Ixf2yq66NlTvrg/ZXNbSzbHtdVHWECqFcWxV61Srv0GyQD7+PunRwJUFbq5RahDtSB0BrfWNSeiUIQq+gO90Uxzy8FICKB8/wE9njHl6KzdG5I5RAh9oH8OgH20K2Gyn/TjSLp4SrpzuJVvBfdf8IgiB46XIYfgJE0Ffsk0GkiVepIOTREu2g7dNKqSxgvHvXZq21LXndEgRBiI9Qg6wQ56zYEOmWPemRo+mLbz09SVRhmUqp44BvgEeAR4EtSqljktgvQRB6AV3Olhnw+R8fbuOb6uYutWsk9F3pZqhzO/Pud6HybibaOPw/AKdorY/VWh8DnAr8KXndEgQh3XA6NQ+8vYmzH/k0IfUlSoh1iPDLzrDMKH34KWDiRyv4mVrrzZ4PWustuPLpCIKQ1nRNxHzdJJ6tdpsjilbDiGyCdTXSEoe9ycKPdtB2hVLqSVwJ0wAuBVYkp0uCIKQjiY74MaqtCy78IGGPwoXvd1IqPBiitfB/BGwAbnT/bHDvEwQhTfjZS1/z6Adb/fYl0ofvEdBoRDmeZGfxEmoQONYH1M0vrE5Ul+ImWgs/A/iL1vqP4J19m520XgmCkHK8vLISgOuOO9i7L5HiGouPuzuN5dBhme7jUT58PtzS80u4RmvhLwZyfT7n4kqgJgiCED8+ithpSXexSq9vPTGPhVDRONGEZaYa0Qp+jta6xfPBvZ2XnC4JgtBb6Cm/dMxinsSOducAcleJVvBblVKzPB+UUuVAe3K6JAhCuuArlokLo/TUHUw8E69ChWUGtmd4LAVCMX2J1od/M/CSUmqP+/NQYF5yuiQIQm8hkYLWHT78eB4qkU5JLUkPT1gLXyk1Wyk1RGu9HJgIvADYgHeAHd3QP0EQUpguR+kY+PAT1W7C3xjieVik2NMgkkvnH4DVvT0HuANXeoUDwONJ7JcgCH2AhjYrDW3WyAUJn+UyUcTl0vHJi2l4PNVUPQyRXDpmrbUnAfU84HGt9SvAK0qpng8qFQShR4mkdTPvda2ZVPHgGRHPj0k4w/nNDfS5S7l0Ii2e0oVzu5tIFr5ZKeV5KJwILPE5Fq3/XxAEISKJ0sZED5RGevNINVEPRyTRng98qJSqxRWV8zGAUupgIPQSMYIgpAVdFdd4FwiJpt1ECX9XBD3VngVhBV9rfb9SajGuqJz3dOc7lwn4cbI7JwhCGpGgQdtwx1QXsum8v7E67PHeQES3jNb6C4N9W5LTHUEQehNdj9LxzZaZmsLp6WJti5Wdda2MGphveNz43NS6pmgnXgmCICSVBI3ZJjzax7eeNmtw6ubUkvTwiOALgtBjpNgKgIYkYzZwTyGCLwhCTHy6tda7nUgBjMX9Ea6sUSqELg28+oV3BlfUmx4CSRN8pdQIpdRSpdQGpdR6pdRNyWpLEITu49InlyWsLn8xjeG8ONuLb+JVhONhHz6xt5dMkhlLbwd+qrX+SilVCKxUSi3SWm9IYpuCIHQjCc2lkyifu7e+xFjj8aZ/cPUltRQ/aRa+1nqv1vor93YzsBEYnqz2BEHoPhKZMq1zKxaXTsI6ELmtCP1KLUkPT7f48JVSo4FDgMS9CwqC0OMkSng37m3ilD99FFXZE/7wATXNloh9MupaU4eNk/74IVuqm6PvXKTUCnHOCegJki74SqkC4BXgZq11k8Hxa5RSK5RSK2pqen4JMEEQIuNxhXd9dSrX/394bwsNbbaoztle08rrX1fF2I6roQ+31LB1fwv/t2RrhDN8zvWrx+h4iql6GJIq+EqpTFxi/6zW+lWjMlrrx7XW5Vrr8tLS0mR2RxCEtCDBuXS6MDks1R4FyYzSUcA/gY2exc8FQehbdHUmaTIEsSv56w3r63KB1CGZFv5RwGXACUqp1e6f05PYniAIQlTEEp0ZKUonfHrk1HoaJC0sU2v9CbHdV0EQehmJ8uEnks50+AnKlhli27svtTQ9LDLTVhCEuEmU2MUzISoe4ovD14bbUZ0be3NJRQRfEISks7u+zXC/xwqPWYhjXPGqtiW6ZRbD1heiWYnSEQQhTYhO7OY+tJR9jR1J7kto/rbUFYbZtTVtQ/jwo0ndmSKI4AuC0C3UtwZb2fG6hMJrbHDytK7Qm3z0kRDBFwQhbnqrGMZi6ftfo0F+nnDnRt9MtyCCLwhpwO76NlbuPJDwerfXtHbpfI8gJnLQNpY4/HfW7aPDFryoiV99PrLtNHTpRE7VnCqI4AtCGjD3oaV85++fJbzeW19Zk/A6wT/nfiLx1d8vttdx7TMrefjdzVGf4zRQ/NSS9PCI4AuC0GOEsoCjzatjXGfkMgqobnINIu8Pk4gN/AXdyMKP9txUQARfEISUw5QAF0+kcEmL3QlATkZ4GfSfaRvjVFu6b45BNIjgC4LQ4wTqaCSRDOs3j9Kutrh99zmZ5gglI/jww7SndWqlGxDBFwShTxFu0Nb3QdLuFfzoLXxnHKtoqRQy8UXwBUHoMTxiGayJyRfJDpvLpZMbwcL39+HHmlpB+11JT0ftiOALgpByJMIojpTozBOOmR1J8P1y6YSv0wjfa+npKM1kLmIuCIIQllD+b1+9r6ht5Yf/Xel/XpzC2djuiv5RSnkt/OxIg7Y+279/bzPLdtT7nRM+PTIolLeUBv77xU5W7TrAHy+cGccVdA0RfEEQUg5fv/c/PtrO5hjWoO304Yd/KtgcLsHPNEfvw1+/p4n1e4JWag2Pn4Wv+dWCdQA9Ivji0hEEoceIKmY+Se58u9OTqTN8JyJ1MfxMW/+3lZ6OyxfBFwQh5VAhtqMh2pTLdreF74hQrqsPBPHhC4IgEFoAfUXSyMJPhG46Qlj4Wmsa2mwc8ptFUdUTcdDW55HV07nzxcIXBCEi3R1O6Cf4Mdr40XbV49IxCrXcFWLBlljRWvvNGu5pC18EXxCEiCRCqIys21DV+op8Mnz4ik4L3+3Z6exTzNca/gSZeCUIQq8iEYap1rCtpiW6wspwM7p2oiznidIxsvDjz5cf3Bf/iVfBZbbVtLCioj76BruACL4gCBFJhEunw+bgxD98aFhvoL4mwiaO1OWQPvxY24lUwNelY1D6n5/s4Npnvoqx1fgQwRcEISKJsPA9E51iJVaXSFQPJwUO7fHhx3F+1H2JbOEH+vmTiQi+IAgRSYQG2p3Bgh/Shx9B5CO5UVz/h++03eHx4Ru4dGJ4x4gleZpRUacTTN3k5xfBFwQhJDXNFj7fVhdTOGFTh40PNu8P2h8q8ViHzcGijdV++/zi8OPUwtW7G8Iet7p9+EYunZh8+OHSI6MD4vCDyzrFwhcEIRU4/7HPuPiJL2Ky8H/83Cqu+Pdy9rtXlPIQGA0DgIYH3toYtn4jaztSDnqAm55fHbafVrtn0DZssYhEujemSBa+7r5IHhF8QUgjYvVP76yLPR59635XJI5nRSkPDgOXDhjHvPv2MllaaPPOtA2ceJW4NrSGTLPy+xxcRmPqJiUWwReENCJeazYRImhk4Yey1H3dP7HrfXSd9Vr4zkCXTqw578OT5ZuN06CwQ2vx4QuCkHjijUCJZeEPb6hlgIYZDdqGrqRzMzkTr1SnhR8g+B9vqY2prvDLLfpn4zR6mDh1Hxi0VUr9Sym1Xym1LlltCIIQG/Fa+IFuj3B4Sgb6pWNZHjDWlaWiqTMQj4UfeG3f/8+KuNs2IstX8A361lcGbZ8CTkti/YIgxEi8ybsC3R5h23AXDdQww0HbCHVA8gY0re6wTKNrS5Qf3+XDD79Yiu4LLh2t9UdA98wXFgQhKuIVMqNY9VgxFNYQZf0GbWNsJ5qeKgVWu2uJQ6O3l9hcWOF74+vDNwzLlDh8QRCSQdyCH5NLxzgLpVEd9y/caFiH36CtgRh21QLXGmxhJl7FUn2ktya/KB2/PnTep+7Kr9bjgq+UukYptUIptaKmpqanuyMIfZp4feMxjbdq//892A2EtcViD1sHhMiHH2GVqWiwhhi0hejuk9JOpuzbSlbDgbB9yQzhw/c02ycGbaNFa/241rpca11eWlra090RhD5NvIZxPIO2gaIZ2zhA+LJdnyylQ6ZHNmxfa07d8hnDmvYzvqYCgAvXLGLh0zcz58eXhW0rK0SUjuf+dGccvqx4JQhpRPwWfuw+7UDLOZZxgEg+/PAzbSO3Y3MGC68vgV39/vLXuHPpv7yf/3Xotzlj8ycA9N+wBiZMgJ/9DH7wg4B+Qr+OVgY117G/cKDfhWmvhd8HBm2VUvOBz4EJSqlKpdTVyWpLEITo8NW2tZWNjL5tYcScMxDsjgklqWf89WNqWyxAsGjGIvhOrRl920J+/+5mQ5dOuKqiacXmMwv4tVVVwfX7NFDaUs8vPnwagN1FgwG4auUbDG6p55Upx1N50pnQ2grXXAOLgpdFvP2ey/ny0ctZ9OSPyP3bX3yuwePD777UCkmz8LXWFyerbkEQ4sPX+vUkOFu0YR8zRxSHPS9IrEOo6vo9TYZtQYxuIXfRvy3dyg3HHxzyeLzYIsSI+l7uIXs2k+l0cOElD/LliKkMbaohz9bBsKYaviybwv2XHMaIUTkwfDhceils3w4FBe5+agbt2wXAuLrd8MvbOHLefXw5YmqAhd+164kWcekIQhrhK5Rmd/RINPHxQf74KBQ3UOBDWfhGe33LJnTQVmtmV64nY9isoEOndFRx/lv/JttuZVDexcAkhjbVcOGa9+jIyOLrIeMA2NvPNda4beCIzr6UlMDzz8N550F5OaxaBbm5jK5mWUY2AAAgAElEQVTZ5a3/9Cv+yptv3MNzL9xJVWEpfG8pTJuC7sZBWxF8oddjdzipbbEypCinp7uSEOwOJzUtFoYW5QKwr7GDkoIsMswmmjpsOJ2a4rysuOr2FeoMk0fwIyt+h83h99lIU6sa2v0+e/LNe4jFpWP1eQoZSWGsYxGDmut48bnbsJvMHFxfSc2bA3Ge9CMWjTvCW+bWlx7i4D3bXB/u+oqPigYzpLmOLKedNyfOxZKZbVi3tyfnnAPf/z48+SR88QUcfzyXffYKlpxc5vzgSerziug48WTyXnqe4c016NmHwgMPkMdkLLmFMV1PvPR4lI4gdJXfvLmBIx5YTEObtae7khAeeHsTcx5YQk2zhcY2G0c8sJh739wAwKG/WcTMe4P9xNHiK5Nmd2iIUbgkwKpdneGGPwpYgi9QcHfXt3HUg0vClgkl+EaC7vewMLB+w/vwtfcNYM7Or7lo9Tv8/q0/M7phLwfXV7K3YCDN5myeePU+Tt38GQB51nbG7N3B3w8/n2OueQJL6SBGNlazeth4/n74+dx+2g2hG/Tt5333uba/9S2YMYPzvnqHHdMPpz6vCIDGBx/mW1f+lYsvuh/n2LHwk5/wm99eRUFbc+T6E4BY+EKvZ/Emly+6ucMet+WbSix1+9Yb261kmc0ALNm0n3vP7pwsFC/GFr5xnRV1rd7tQOs90MD2DNT6ElhvLFa5b6K1aC38IU21XLnyDQoP0egxZ4DW/POVe8mzufpmNWVw2bzfsLxsMgfX7ea9f93APxb8loN+/jqT92/HrJ0sL5vMrv5DWfnaYm797zIqi4dE7qxvVwYPhttug6VLwW6nKTufdUedyk9PHs8fFm3B0X8AGweNBaDliZ9S9O6bDLxwHg/dfzlcVwE5yX1LFcEX+gyJzGPeo3ivo1PqEubi9fXhuwU/lIXvedhEg18KYDfRWvhGRHywGRy++dPnuGjNe3DRq+g7D+ZR0yDybBb+N3EuA9sa+fXJ17K1ZCQAW0pHs2DysZyz4UOuWPk/xtZXArDW7ae3lAyKTuwxCBF94AHv5tx73uPcQ4Yz2e1u9PWeaZOC73yHe2/8C5N2beDiJIs9iOALfQCPGMabGMwXu8PJ4k37OWXy4G4LlQvEcxUmlZhr8sVXc70WfghxzTYQ8VCYDcJMAgeDYxF8u68P3111v44WhjTXsqV0NE6tmbJvK/0sbXw+ajpT923lwjWL+GrYBErPPp2yF57m9Pqt1OQV8/Ax32NX/6FBbdx+6o85Z8OH/GrJkwB8OPsUagr6A7GlkQ67vq77oGdQ1ndswnM71o2byfYp5XRHWKMIvtDriWXB6Ug8+sE2/rhoC098r5yTJw9OWL2x4LGMlVI+mScTc42+D5CIFn4Mgh84QOuqN3DFq+hFVDU2ckjVJqbv+4bDqnN46sNPOG7HSgBq8otZfeK5HP+/p8nQTr5/3q+4Z9FjmNA8PetMLvzJbZT+8UGO/vmL1OUVoZXxdbRn5XDzmT/l6uULOJDbjwXfugLcnqlYUkmEw3PFnt+e1e4r+J1x+KZuissUwRcEHyoPuJbbq28N9kn3BIn2UjkNXDqhonRiCRU0zEcTUG2oB4svp27+jL/+7yGyHXauMThem1eE1ZTJyW/827vvsdfuJ0M7ufOU63h9yvFcCGiTmdr8/hHbs118CWdNOR6AI8sGwrY6V9/jSCURCqU631JsjmDB1xKHLwixk4AMvilBZ/KxTns8Ud4lX1dFJAs/lhWqjMoGxuGHE9ECSxtXrXidn3zyLE1Zeay88Pt8sNfC/yYdww9mD+WThZ/x6agZWDKzybF18PCu99lfsZfBzXWcvvlTlowt55mZ33JfY/SC7ZvnxtctFct3KXzcv+s/z+/P16XjOdadydNE8IVexzvr9lGQncHR40r89iciZ3syWbhmL/3zMznyoJKgY5UH2nhzzV5+eMxYv/TCTq9LJ36e+WKnd9tXnELlvPEQm8/dyML33+eJPgJXuoLfvf1XDtmzmWzlJK/dFRHkRHHhpb/j2PNP5PGPtgNwYPholhzcmVWzIzOHt8/5Pm+t3Qe4fPutWbleVd24t4mNeztn/IbDV/AzfAT/wy37jYobEmmcRbn/gX9Kh85smWLhC0JIrn3G5cutePAMoNN66sqyeN3B9c+5Ytk9/fbl6qdWsLm6mW/PGOYjxIl5iN25oHOVUf/0vK4PoS38WMIog8sG9n13fWdo57nrl3LC9hW8Nf5IyrI1uqaWD8fM4plDTmd/4UCO9HmAGCY38zGUm3IK/I7d/5Zxjn0jMnxy1fta2fO/3B11HeEIfEOzGLh0+kQuHUHoblLdwg9Hc4cNcImAb46VRF+Tr3h2pgfuuoVvlJsm3APj2B0r2V00mOvOvYNjx5fy4Rb/tTB8XURGz/FERS/55qqPV3Qj2Rm+1RoN2nanD19m2gq9Hs/fSm8WfCMcTu31g8crRoH3xPeTR6hCCXMs9zOWRUQGNddx1M41LJh8XMhyNj8LP7iORP2qfSOR4jWyw6fu8f/9+T4Y+1R6ZEHobix2B/P+8Tkrd4ZegSiZvLNuL6NvW8hzy3aFLbelOvw0et+l7zwRNIFysGlfE6NvW8idC9YGnV9R28rZj3xKY5styPL2s/C1x8I3Hpzt6kSpUOdftOY9AD4ZPTNkOd9+P/bhtqDjscTJh8PXh5+wuW1ac9VTy1nqngGuMA7L7BT8NFrxShC6isd62lLdwrId9dzxarAIdgfXuvPN3PFa+PYfemdTyGMOZ6ezwuHUnYOhAXrwG3dunWe+CH64/HXJN3y9u4FFG6v9o0II4cMPMfEqFh9+LBb++WvfB2Dd4INCtmOPkMIzUcM1/i6d0OVOmhRmTkZAZ2wOzZJN+7n66eVBPnyjsMzuXNNWfPhCn8Hzx9Rdk1jiJZyO2gIGK0NZyTZ7dIoXKOa+lrEzog+/i2GZ7nqHNtVw3RcvodypiUc2VvPw3Mtozc7z64cvtggPm0QN0GdmGA/aBnL6tCG8v7Ha8FhgTzz3wmOIKNU5cc7Yhy9hmYIQM54/JnMKvrfaDSw7IxxO7R+lE6JsoOVuczj9rFXf/b741uYR5NA+/JDdDMLoLaHV6uC6z1/k1o/+4+qLyUym08GL007i3+Xf7mzH4BojWvjRdy0sWVFa+Eb31tuXgM54HtrK55g3SsfXpeP+vzvDMlPwT0MQYsPrH3WLhFkp2qx2Rt+20C8G3YPD6Vo+7+8fGPmGXf8HauCqXQcYfdtCNuwJju++780NjL5tYdg+GuVQMcLmcPq96nvE2Gp3+rXhK+QrKuoZ98u3DccOfC1KT52B/fC18B96Z5O3nUiuKQ9aaz8Lf8aezfz9td9y2FnHcOtH/2FZ2RQuuORBxv/sNSb89FVuPf1m2rJyO/thcEPeXW9sTXtI1AC9n0snjBc/0xz62F1vrOfJj7d7v1d/XfwN0Gm1K9VZs+8bnO9YjfjwBSFaPLMY7Z0unf1NrtQInsk7vljsrsU8/vz+lpBVBlq976xzTfL5wGBCzpOf7IjYRf/BuggWvnvb6dTexGaNbbaQ9e2sc6WD+GJ7XVB9QWvRGvjwfcXzUfdD0HOPwpHhsDO4uRZtteGw2jhu2wr+/L+Hef2/P/WGXf51zjyuPv8ulo+YilYmLBnB6atjWfrQ6Dq6gpEPf1Bh8EIn4Sx8gEeWbvX+Tv7p+T74JL/rtPA776t34pVT4vAFIWY6bJ0WvsfizDB4Vw60eo2I5FKIFYuB79awXaez06XjY+EHnuFr4XveHozcAuGidIwE32xSOJyavQ0doS8GGNDWyAvP3eZap/XRK7jIZOISpxOHMvHP8rN57PDzvZknIxFq0DgciYvDD/bhD++fy/5mS0C58IJvUgpbwDiGx6WjvJ86v6PgP2grM22FmLHYHTS22xhUmNi82tVNHQzIz4r4pe8qVQ3tDCvKCWvthFt82rMMn8mkOsXf4C+prjXyylhddRl4fhdFuZk0ttn8B+vCPEt8xW9NZSMjBrgGNoN99p3lqptc4mzkFgh8uPk+azxVtljsLK+oZ1hxLkW5mdS3WtkcInS0uL2JQS31XP7Vm4yr281Ts87k1CMnUFXTxIuNuSw++DDq8sMviB5IPAOwicpmaRSHn2XwPY/03VdK+aVN8GCxO0EZW/g9EYcvgt+H+PFzq3hvQ7Xh1P14abc6OPy3i7n4sJE8cN60hNUbyIqKes5/7HN+f8EMzj+0LGS528OEXLZZXflWzErR7hZ/I8E/8Q8fRuxPV1eWuuG5VSzaUM0Z04eycM1e3r35GO+x8Ba+xmPPP/zuZg4fMwAIFu52nzVm//y+y2ds9KCMxqVT1dDOBY99Tmlhdqfg7wsW/EOqNvHfF39FgdWVImHNkIO5++RruRsgP+QlRSSW8E8PyZhp61kTec5BA1m2oz6gXHhBNqng6/C81a2vamL2KNfvsc3i69LxWPjdF1kmPvw+xHsbXANdiXRHNFtcvuNFG8IPonUVj0UZadLUyysrg/Z5/lTarJ0i77sdD7GEJBrhuV8L1+wFYE9jZx6ZcAat3SdKJ/A8X+oN3lIC9d6pdXiXToBA1TRbvIuebNrX5O3s6Poqzl23hNee+RkF1naenXka/z78XH537BWGffvFaRMN94eiwxp5vCCQWJ8RVxw52nC/r+CPLcnno58fz40njAtbzgilQrsKd9S20j/fNXaxt6nTVeabFVVcOkLcWB1OMhLkfrHYQvvCUwWPZdtu7XTptFlc1n68/Q608D2foo1/D2R3fZt3O6yF7xOlE2t73tO0py4d5Gbwrc0rmlozpLmOkY37mFRtY3izlTHb27m69gAnblvOkbvWuPpiMnPtuXew+ODDAffgZnPwugEThhQE7QukKDeTxnaXMdEah+DHOtN28tB+hvt9LXeFYuTAPMNyGREtfBXW3VhS4BL8PT5rA/tZ+OLSSW9WVNQzYkAeg/vF7o+32p0YreW9eGM1VruT4ycOIifTeL3SrftbaLPaqW+1MmJAHuuqGgHY19TBkx9vp3z0ADLNisZ2G5OG9KN/fha769tYsKoKk0lx/qFlLFyzl1mj+lNamM2ehna217Rw1MElrKtqAjQDC7L5bGsd580azogBeXy9u4GnPq0AXJbxb86ewteVDaze3chxE0o5qNQlIIGRKq+vrqK2xcrW/S2u63NPZTerzreddpuDf3+6g31NHeRkmIP+sBZvrGbTvmZGDMhjy75mvq5sAFzT+UcOyGNocQ4b9jTxn89d/Wuz2nlr7V4a222M6J8X9RvEXW+s9247tWbDniY+21bLsOJchhXnsqfRZfkF+s73NYUZPNWaQmsbg1rqGd64H9v2L+DwYpZvcyUie/yjbZQWZmN2Oji0aiPzvn4X9dkjrB88mP0Dh8KWvby8ZRmT9u8g32bcTktWLo8ccQFLDypn9dAJ2M2dkjFyQF7Q4CZAdkbktXB9H2rtcQj+8orY0mfkZhn3yc9fH+ZXaeTX98WkVFjXVEmBK/LH812FQB9+2OoThgh+CqK15vzHPmdoUQ6f335izOdbDF4tN+xp4uqnVwAwr3wEvzt/uuG5J/0xtH/7voX+aWcnD+3HWzfNZe5DS737Hn53s3c7y2wKGmz0ZdXuAzx15WGc/cin3n21LRYe/3g7r35Vxdb9LXy1ayiPXDILgCuf+tLv/JueX21Yb4vFztLNLtHbUt3CPf/bYFjOYnd674nRsZ++9HXQ/n9/WhH2mpR2ot2jdJkOG/06WlFocm0WhjTXMra+itHbMvjXwpfRKD7PLUBpONNuIddm4Zt1SzglK4ORe3ewY8BwlNZM3r+d0tYGMh02Mh12Ci1tDGqtZ1DLAXLtAYL70n0sVSZq84tpyi7ArB0Mba4lz2bBoUzsLhpM2SfVTNFOjgf2FgzkhemnUDx9Eq+15mPSGqs5g53uNWBbsvKC0g97GN4/lxUGLrhSg7DGQK4+eox37CHc/UwUBTnGUpedaZxL5/gJpd7vEHT690MRzqUDkJ8d3L7n79RiS9wbeSRE8FMQj/95b2P40LhQWGzBX7wOn+iAdXsa4+tYABsiLDIR6Q95X4jr299k8b7uN3d0Lnzx1a6GqPpljXXAVWsKrO3kW9vIdDo4YtdahjXVMLiljiy7nZbsXIY31TD5wG7yW5owaycm7aQ2r5gdA4bRkZFNSabmUEsN1j37yGhvozGngOKOFsy662LmUCYq+g/Fas7EZs6gNSuX1UMnsL+gP/vzB7C/oD/7Ckuwm8zcMLCNdcs3MqilnkJLK3ZzBh1zjubgOTM4+8AoNmUUcd6MoSz9fBPtmdlYMrJYc89pFOZkUrn4G/64qHNuwg/mjuHQUQO86w9cfNgIvzzxRbmZQX2tePAMr/tqUGE2X/7yJMA1j8FTz8Ibj2bkgDyv4HcHJfnGD6GC7M5r8B30/ufls7E5nVG9rUBklw64HnL/9JmzUd9qoc1qp93mYGCBwSt5EhDBT0FqW7q2nqrVEfyK7DtAF88rdDLIC/GanZ9t9g7ktVvtwQW0JttuJdthY2weOKv2MKFmJ2ang+KOFoo3FTDOloFGkW9tI8PppLi9if4dzZS0NjCopZ7BLfVkOO20Z2YzrKmGbEdwO7V5RVjNmRRY2thXWELVyHFsVAVopdAoxtZXMaCtiXxrO2RnYTpoGNsGH8RSZxH925uozy1if0F/NApLRhYNuYVsLhmF6leIo7kFk9aUtDZgzcikIyOLjowsMp0O8mwd1OYV0a+jlfbMbBpy+9GS3elbNqnQg5Zvl5fxYm653777zpnKlCNG0fDbxdDUweSyYl7NK/IeL8xxid7w4ly/8wb3y/ETsSH9/I/nZRnLh8e14es2LM7rFFans/uXowwlqL6Wv6+FbzIpsk3RiT0YR+kEMjTgLaGmxUpdi2vw3ePySTZ9QvCXba/jxRWV9M/LZMSAPA4d1R+HU7N5XzMXzh7hV9Zid/B/i7cyuCiHLfuaOXvmMMpHD+D11VUM6ZfD/mYLzR12Wiw29jVamDu+hOMnDOKddftYsqkam0NT1dBOhklx+ZGjyTQr9jVaeG1VJdcccxAnTRrEPz7azqyR/XllZSVHHjyQmmYLs0b15/VVVWzY28SYknxsDs36PY2MG1zI1uoWDh5UQFaGiYLsDOrbOiMwPttWy2/f2khpQTZf7qjnsDED2NdkYeKQQnIyzYwtyeed9fsY5vPHetb/fcqZ04ficGp21bcxtjSfF1d0Rrdsr23lzgVrGT+4kI17m9nb2E6m2cSBKOLTA7nlBWO3ii9Zdhv9LC7fZb61nUyHnWy7ldyKFv5vyzLO3FFN//YmBrY1UtTRQt5nOfy8pY2hzbUUdTSz/kFFptL8r92CWTtdot0amw/XoUw05hRQl1fEvoKBLBs5Fbsyk2/r4L1xc6jNK6YlOw+TdvJNyUhWDZuAzexvwZ41Yxj/+3qPYf1FuZl8fdcpLFi4gSc+jjzzlmKXm2SX23ViRHVh8FKI4HKZVDcZGwW+v+dAPAOPnjGRQEoCXDGlhdl+KRMG9/M/HuqB7ZkPkRtC8B1aJyy9cbSEFHyfh1ZXxk0r6tr4prrF8JjnTTewD//6ZAcvr3C9MZX0BQtfKXUa8BfADDyptX4wGe3c/erXlCz/lFdGHxL0WwsU/Ne+quJvS7d6P//3i51UPHhGSH/wK19V8vVdp3hfR3Mzzd4Y6BaLnTWVne6R5RUrWHbHiTz4dmf62wWrq4J86r4DTlvcX5LN1c0M7hf8h3zJE8v8Pnv8ivubOvwmEPmGM1osVv73xVay3MK6ExjusJNvbaPA2o5DmVi9YCtfKRNOpXAoEzZzBi3ZeeRm5tKRmeW2jttxmEwUWNspbm+mX0crebYOCqxtZDgdFFjaGLJJcU1jC8OaaihtPUD/9mbybO3kWS3072iiqN3l1ojWtdGSlYvZ6cRhMrGnsJTa/GLqMeFQZhwF+TiVie0DythYOpq2rByOnzmKl7c0UNF/GNUFA2jPzEFpTXFHM0prmrPzsZvMNOXko5W/n3RgfhbtNofXhebhzjMm8WXAeMVfLz4Eu8PpFfyB+Vl+99/jgpo9egDvbaj2pjs4YeIglmzyT8cwemAeFXVtxMLIAXnsqm9jWFEOVx09hhdX7PZ+dyJx+jTXQ+WmE8fx708rmDK8H2fPHMbrq/0fXgPzO0Vn4pBCZpQVe4V6SL8cTps6hEc/2MYut8vmmPGlLFhVxfZa13q0Z0x3tTOmJJ/pZUX86szJ3vrGlhR430wmDikk02xixohivt7d4Hc/rjpqDBV1rSzZtJ+fnzqBPy3agt2pyck0+c1SjZbbvzWRT7fVkZ1h5ntzRvHZtjq/gVPf6JuJQ4wjeaIlVO4hj29/5oj+jB9cQPnoAby8spId7vtWkJ3BlGFFhucmmqQJvlLKDDwCnAxUAsuVUm9orY1H0LrAGe8+ww3v/xsnijtOvZ6vhk9kS8koUIp2q8NvhD7wjxvCh3h5/pA9nD1zGM8vdz2VGwKiRiA4PtpoABVgaBZYDzRQYG2jpLWB0tYD3HPYQJ7+fA0ZTgfZdiv9LK1kOOxkOWxk220UdbRQYG1H4eTgAbnsqmnG5HRi0hqzdpJtt1Lc0eydGBMvNpMZjSLLaeBOCUW/flgGD2WNw8yB3H7MnDOKxrx+/PObFiwZmdTmFaPQtGblYjVnYjVnciC3kPbMHCzmTBpy+9GcnUtHZuSopLnjSvjv1a7QwFW7DvDGo58FlWnMLfRuD+mXw9d3uAa/l22vY97jXzBpaD/evmmut8z+5g4Ou38xAN+fO5bPt9WxeNN+Hr10llcwAX7yomsg95UfHclxv/8gqN1TpgzhlClD/PZ5kpGFmhC3cM1e73q3AA+cN807wSzUOd+fO5abnl8VJNoAD543jYsOGxm0/4LyEVxQ7jKA/nLRIUHn+g62vuMzUcy3Dx/derz3ekoKsljys+OY88Bi9jZ28NOTxwMuV84bNxztV3dWhontD/hfy+vXH+Xd9tT567Mm+5W5/viDAde8gNP+/HHQNXm484xJQUEFAD889iB+eKwr7/69Z0/1awv852lMHhad4M8cUczq3dGNJ0Gn4I8pyee9W44FXOMbnjGMRT85Jq5ovHhIpoV/GLBVa70dQCn1PHA2kHDBL9/h+iM0oXnw3b8BLv+rU5kwLR4H4w+G0lLo359J9Q4u21hDnq2DDIeDXLsF66++4PrPdmI3m7ErM9kOl5Bn220UWNtw3rSIX31W4bqorzM56ZudlLQ1kGe34QCcyoTD5LKWh34wmP9Ut5LtsJFtt5LlsGE3mV0+Z7uNHLuFoo7W4OgKgAVwq3vTrkw05RRgM2dgMWdiyciiKTuf5uw8HCYTWWWDqLLUu9pF4TSZsZgzaMjtR1N2PpaMLKzmTDoys1DuyAtLRjYNOQWYtJMMpxOzdmByuqxvj8Web2v3RpY05BSS4XTQlJ1PQ24hTdn5dGRmM3rsUFbvaeb+K+Yye9IwyM6GggIaWyxc4BbNigfPgDYbv7v3vUT/uv0win4Ih8dt4VlD1ltPgD/aYySEiryIFLURCwPy/V/nPf7cggjXFqpvoUIQY+1HJDz3zOO7DuXTTwQDQwy6eohnti50T/y7UfCCr88+0rUlkmQK/nDAd+n3SuDwZDQ0trqCV6ccz4PHXsHRO1dTaGljVtUmnEox6kADQ99ZQr+2ZvI7WpkDzPE5165MZHzu5Och6m7OyqVtnYkLnBqFxlxYiMlcQG1+MXszczC5IzbMbuGs2NdAodWGJSPTK7xmpwOLe1DOI9yDxwxj+QEnHTl57M/pR21Bf968/wIm/uELbGZzkPshkB0PnM4Pbn8rUbcwJkonDmWLbS+tg4ZASaefeUBA8H+/3OR8vXwHA3NDzCcI1Q9PHwP/0D31eOKtPeIXSkhCzWOIh5xM/9+1Z7brsOLwD5VQcwDizXkU7XmF2Rk0W+zeB4vHlx8p/UBXiPQwMscp3PHEv0fznfPFKIbf12fvm88n2fT4oK1S6hrgGoCRI4NfQyOhrVYqZh3FprLpOIcOY8WoEYwtzWe7U1N5oJ1JQztf7U0OO7ltLazZUUdmcT/qrU5GDR1A/7wMdu5tZGC2oqW1g4y8XPY2WZhYVkx2dhZOrdlZ10ZhTgYPnz+DnVWNVB1oZ21VIzvr20Br1u9pYnpZEUOKcli58wAzRxQzdVgR22pa+GJ7PdPLivhqVwNtVjvjBhVw/7nTqN5Wy9xxpXy6tZY5Bw3ENLSIm86YyisrK5k4tJB1VU1MG15Ehlkxo6yYF1fs5sLyEZiUK4TsrrMm88Ly3VQ1tHPUQSVMKyuioraVl1ZWcsz4UszK5fM/ZnwpjW1W+udnUZybyY66NvY0tFPWP5dBhdnuCVGuJFyD++UwsCCL7TWtTB7Wj6Z2G+MGF1KUm8EJEwfR3GFn7rhSygbkcvTB/oOKGWYTd54xiSPGDvT8brn1tAl02JzsaWhnRUU95aMHcPFhI8k0K259eQ3fmVXGlupmhvfPZcGqKqYML8KkFFlmE/Nmj2B5RT1NHa6EcGNL8lleUc+VR43xtjm8OJcrjhzN8OJcWq12Jg7px30LN6AU3PGtSWypbuHsmcO85YvzMrn1tAmcMtnf7WIyKe48YxJHj3Nd089OnUBOppmzZvgPqr7yoyPZ6A5HvfOMSdgcmuK8TMYNCj279E/zZoRNaDejrJjrjz+IY8cP4vNtdcwdV8JNJ47jgvLQOYUA7v72FEYOyCM/O8MruvsaOzh5cpjl+HxYcP1RrA8I0X3wvGkcFOZaAF697kje37jf+9B76srDeGvt3pjfEHz579WHGaaL8GA2ub7vnnt/3qwyXl9dRYvFwfThRVw2ZxQ1LRaOGVfK/OW7yDKbuORwYz3507wZNLXbsdgdZJhN/OWimYYhpoE88b1yHE4nM0f056F3XViktmIAAAc6SURBVON0k4b0Y21VI1/ucP2Nr6tqZOrwIpRy9XnTvmb+fumhQXUdPmYg35lVFrUbKVGoZI2WK6XmAHdrrU91f74dQGv9QKhzysvL9YoVxhNhBEEQhGCUUiu11uWRSyY3edpyYJxSaoxSKgu4CHgjie0JgiAIYUiaS0drbVdK3QC8iyss819a6/URThMEQRCSRFJ9+Frrt4CeGVkUBEEQ/JB8+IIgCGmCCL4gCEKaIIIvCIKQJojgC4IgpAki+IIgCGlC0iZexYNSqgbYGefpJUBtArvTm5F70Ynci07kXnTSl+7FKK11aTQFU0rwu4JSakW0s836OnIvOpF70Ynci07S9V6IS0cQBCFNEMEXBEFIE/qS4D/e0x1IIeRedCL3ohO5F52k5b3oMz58QRAEITx9ycIXBEEQwtDrBV8pdZpSarNSaqtS6rae7k+yUUqNUEotVUptUEqtV0rd5N4/QCm1SCn1jfv//u79Sin1V/f9WaOUmtWzV5B4lFJmpdQqpdSb7s9jlFLL3Nf8gjs9N0qpbPfnre7jo3uy34lGKVWslHpZKbVJKbVRKTUnzb8Xt7j/RtYppeYrpXLS9bvhoVcLvs9C6d8CJgMXK6Umhz+r12MHfqq1ngwcAVzvvubbgMVa63HAYvdncN2bce6fa4C/d3+Xk85NgO8K1r8D/qS1Phg4AFzt3n81cMC9/0/ucn2JvwDvaK0nAjNw3ZO0/F4opYYDNwLlWuupuFK0X0T6fjdcaK177Q+u5Wnf9fl8O3B7T/erm+/B68DJwGZgqHvfUGCze/sfwMU+5b3l+sIPUIZLyE4A3gQUrgk1GYHfEVxrM8xxb2e4y6mevoYE3YciYEfg9aTx98KzpvYA9+/6TeDUdPxu+P70agsf44XSh/dQX7od92vnIcAyYLDWeq/70D7As7BpX79HfwZuBZzuzwOBBq213f3Z93q998J9vNFdvi8wBqgB/u12bz2plMonTb8XWusq4PfALmAvrt/1StLzu+Gltwt+2qKUKgBeAW7WWjf5HtMuM6XPh18ppc4E9mutV/Z0X1KADGAW8Het9SFAK53uGyB9vhcA7rGKs3E9CIcB+cBpPdqpFKC3C34VMMLnc5l7X59GKZWJS+yf1Vq/6t5drZQa6j4+FNjv3t+X79FRwLeVUhXA87jcOn8BipVSntXcfK/Xey/cx4uAuu7scBKpBCq11svcn1/G9QBIx+8FwEnADq11jdbaBryK6/uSjt8NL71d8NNuoXSllAL+CWzUWv/R59AbwOXu7ctx+fY9+7/njso4Amj0ecXv1Witb9dal2mtR+P63S/RWl8KLAXOdxcLvBeee3S+u3yfsHi11vuA3UqpCe5dJwIbSMPvhZtdwBFKqTz334znfqTdd8OPnh5E6OoPcDqwBdgG/LKn+9MN13s0rtfyNcBq98/puPyNi4FvgPeBAe7yClck0zZgLa6ohR6/jiTcl+OAN93bY4Evga3AS0C2e3+O+/NW9/GxPd3vBN+DmcAK93djAdA/nb8XwD3AJmAd8F8gO12/G54fmWkrCIKQJvR2l44gCIIQJSL4giAIaYIIviAIQpoggi8IgpAmiOALgiCkCSL4Qp9AKeVQSq32+QmbOVUpda1S6nsJaLdCKVUSx3mnKqXucWezfLur/RCEaMiIXEQQegXtWuuZ0RbWWj+WzM5EwVxck4DmAp/0cF+ENEEsfKFP47bAH1JKrVVKfamUOti9/26l1M/c2ze61xdYo5R63r1vgFJqgXvfF0qp6e79A5VS77nzrD+JawKTp63vuttYrZT6hzt9d2B/5imlVuNK3ftn4AngSqVUn54hLqQGIvhCXyE3wKUzz+dYo9Z6GvA3XCIbyG3AIVrr6cC17n33AKvc++4A/uPefxfwidZ6CvAaMBJAKTUJmAcc5X7TcACXBjaktX4BV4bTde4+rXW3/e2uXLwgRIO4dIS+QjiXznyf//9kcHwN8KxSagGulATgSmHxHQCt9RK3Zd8POAY4z71/oVLqgLv8icChwHJX6hZy6UxUFsh4YLt7O19r3RzF9QlClxHBF9IBHWLbwxm4hPws4JdKqWlxtKGAp7XWt4ctpNQKoATIUEptAIa6XTw/1lp/HEe7ghA14tIR0oF5Pv9/7ntAKWUCRmitlwK/wJUWtwD4GLdLRil1HFCrXesOfARc4t7/LVwJysCVoOx8pdQg97EBSqlRgR3RWpcDC3Hlan8IV8K/mSL2QncgFr7QV8h1W8oe3tFae0Iz+yul1gAW4OKA88zAM0qpIlxW+l+11g1KqbuBf7nPa6Mzde49wHyl1HrgM1xpeNFab1BK3Qm8536I2IDrgZ0GfZ2Fa9D2OuCPBscFISlItkyhT+NeHKVca13b030RhJ5GXDqCIAhpglj4giAIaYJY+IIgCGmCCL4gCEKaIIIvCIKQJojgC4IgpAki+IIgCGmCCL4gCEKa8P/IG9oI7ZtbrAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the scores\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "plt.plot(np.arange(len(scores)), scores, label='MADDPG')\n",
    "plt.plot(np.arange(len(scores)), avgs, c='r', label='moving avg')\n",
    "plt.ylabel('Score')\n",
    "plt.xlabel('Episode #')\n",
    "plt.legend(loc='upper left');\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Second Best Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episodes 0000-0010\tMax Reward: 0.200\tMoving Average: 0.030\n",
      "Episodes 0010-0020\tMax Reward: 0.000\tMoving Average: 0.015\n",
      "Episodes 0020-0030\tMax Reward: 0.100\tMoving Average: 0.020\n",
      "Episodes 0030-0040\tMax Reward: 0.000\tMoving Average: 0.015\n",
      "Episodes 0040-0050\tMax Reward: 0.000\tMoving Average: 0.012\n",
      "Episodes 0050-0060\tMax Reward: 0.100\tMoving Average: 0.012\n",
      "Episodes 0060-0070\tMax Reward: 0.100\tMoving Average: 0.013\n",
      "Episodes 0070-0080\tMax Reward: 0.100\tMoving Average: 0.013\n",
      "Episodes 0080-0090\tMax Reward: 0.100\tMoving Average: 0.012\n",
      "Episodes 0090-0100\tMax Reward: 0.100\tMoving Average: 0.014\n",
      "Episodes 0100-0110\tMax Reward: 0.100\tMoving Average: 0.012\n",
      "Episodes 0110-0120\tMax Reward: 0.100\tMoving Average: 0.016\n",
      "Episodes 0120-0130\tMax Reward: 0.100\tMoving Average: 0.014\n",
      "Episodes 0130-0140\tMax Reward: 0.100\tMoving Average: 0.015\n",
      "Episodes 0140-0150\tMax Reward: 0.000\tMoving Average: 0.015\n",
      "Episodes 0150-0160\tMax Reward: 0.000\tMoving Average: 0.014\n",
      "Episodes 0160-0170\tMax Reward: 0.100\tMoving Average: 0.013\n",
      "Episodes 0170-0180\tMax Reward: 0.100\tMoving Average: 0.014\n",
      "Episodes 0180-0190\tMax Reward: 0.100\tMoving Average: 0.016\n",
      "Episodes 0190-0200\tMax Reward: 0.000\tMoving Average: 0.013\n",
      "Episodes 0200-0210\tMax Reward: 0.100\tMoving Average: 0.013\n",
      "Episodes 0210-0220\tMax Reward: 0.100\tMoving Average: 0.011\n",
      "Episodes 0220-0230\tMax Reward: 0.100\tMoving Average: 0.012\n",
      "Episodes 0230-0240\tMax Reward: 0.000\tMoving Average: 0.011\n",
      "Episodes 0240-0250\tMax Reward: 0.100\tMoving Average: 0.015\n",
      "Episodes 0250-0260\tMax Reward: 0.100\tMoving Average: 0.018\n",
      "Episodes 0260-0270\tMax Reward: 0.100\tMoving Average: 0.020\n",
      "Episodes 0270-0280\tMax Reward: 0.100\tMoving Average: 0.022\n",
      "Episodes 0280-0290\tMax Reward: 0.100\tMoving Average: 0.023\n",
      "Episodes 0290-0300\tMax Reward: 0.100\tMoving Average: 0.029\n",
      "Episodes 0300-0310\tMax Reward: 0.200\tMoving Average: 0.031\n",
      "Episodes 0310-0320\tMax Reward: 0.000\tMoving Average: 0.029\n",
      "Episodes 0320-0330\tMax Reward: 0.100\tMoving Average: 0.030\n",
      "Episodes 0330-0340\tMax Reward: 0.100\tMoving Average: 0.033\n",
      "Episodes 0340-0350\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 0350-0360\tMax Reward: 0.200\tMoving Average: 0.034\n",
      "Episodes 0360-0370\tMax Reward: 0.200\tMoving Average: 0.039\n",
      "Episodes 0370-0380\tMax Reward: 0.200\tMoving Average: 0.042\n",
      "Episodes 0380-0390\tMax Reward: 0.200\tMoving Average: 0.045\n",
      "Episodes 0390-0400\tMax Reward: 0.100\tMoving Average: 0.041\n",
      "Episodes 0400-0410\tMax Reward: 0.100\tMoving Average: 0.042\n",
      "Episodes 0410-0420\tMax Reward: 0.200\tMoving Average: 0.048\n",
      "Episodes 0420-0430\tMax Reward: 0.200\tMoving Average: 0.050\n",
      "Episodes 0430-0440\tMax Reward: 0.100\tMoving Average: 0.053\n",
      "Episodes 0440-0450\tMax Reward: 0.100\tMoving Average: 0.055\n",
      "Episodes 0450-0460\tMax Reward: 0.100\tMoving Average: 0.052\n",
      "Episodes 0460-0470\tMax Reward: 0.100\tMoving Average: 0.048\n",
      "Episodes 0470-0480\tMax Reward: 0.200\tMoving Average: 0.047\n",
      "Episodes 0480-0490\tMax Reward: 0.200\tMoving Average: 0.045\n",
      "Episodes 0490-0500\tMax Reward: 0.100\tMoving Average: 0.047\n",
      "Episodes 0500-0510\tMax Reward: 0.100\tMoving Average: 0.046\n",
      "Episodes 0510-0520\tMax Reward: 0.100\tMoving Average: 0.041\n",
      "Episodes 0520-0530\tMax Reward: 0.100\tMoving Average: 0.041\n",
      "Episodes 0530-0540\tMax Reward: 0.100\tMoving Average: 0.036\n",
      "Episodes 0540-0550\tMax Reward: 0.100\tMoving Average: 0.033\n",
      "Episodes 0550-0560\tMax Reward: 0.100\tMoving Average: 0.035\n",
      "Episodes 0560-0570\tMax Reward: 0.100\tMoving Average: 0.036\n",
      "Episodes 0570-0580\tMax Reward: 0.100\tMoving Average: 0.032\n",
      "Episodes 0580-0590\tMax Reward: 0.200\tMoving Average: 0.033\n",
      "Episodes 0590-0600\tMax Reward: 0.100\tMoving Average: 0.035\n",
      "Episodes 0600-0610\tMax Reward: 0.300\tMoving Average: 0.036\n",
      "Episodes 0610-0620\tMax Reward: 0.200\tMoving Average: 0.041\n",
      "Episodes 0620-0630\tMax Reward: 0.300\tMoving Average: 0.039\n",
      "Episodes 0630-0640\tMax Reward: 0.000\tMoving Average: 0.038\n",
      "Episodes 0640-0650\tMax Reward: 0.300\tMoving Average: 0.041\n",
      "Episodes 0650-0660\tMax Reward: 0.400\tMoving Average: 0.049\n",
      "Episodes 0660-0670\tMax Reward: 0.200\tMoving Average: 0.052\n",
      "Episodes 0670-0680\tMax Reward: 0.200\tMoving Average: 0.054\n",
      "Episodes 0680-0690\tMax Reward: 0.400\tMoving Average: 0.055\n",
      "Episodes 0690-0700\tMax Reward: 0.500\tMoving Average: 0.074\n",
      "Episodes 0700-0710\tMax Reward: 0.500\tMoving Average: 0.083\n",
      "Episodes 0710-0720\tMax Reward: 0.500\tMoving Average: 0.088\n",
      "Episodes 0720-0730\tMax Reward: 0.700\tMoving Average: 0.101\n",
      "Episodes 0730-0740\tMax Reward: 0.600\tMoving Average: 0.120\n",
      "Episodes 0740-0750\tMax Reward: 0.500\tMoving Average: 0.136\n",
      "Episodes 0750-0760\tMax Reward: 0.800\tMoving Average: 0.144\n",
      "Episodes 0760-0770\tMax Reward: 0.100\tMoving Average: 0.145\n",
      "Episodes 0770-0780\tMax Reward: 0.900\tMoving Average: 0.175\n",
      "Episodes 0780-0790\tMax Reward: 0.400\tMoving Average: 0.182\n",
      "Episodes 0790-0800\tMax Reward: 1.500\tMoving Average: 0.186\n",
      "Episodes 0800-0810\tMax Reward: 0.700\tMoving Average: 0.197\n",
      "Episodes 0810-0820\tMax Reward: 2.400\tMoving Average: 0.243\n",
      "Episodes 0820-0830\tMax Reward: 0.500\tMoving Average: 0.252\n",
      "Episodes 0830-0840\tMax Reward: 0.600\tMoving Average: 0.256\n",
      "Episodes 0840-0850\tMax Reward: 2.500\tMoving Average: 0.310\n",
      "Episodes 0850-0860\tMax Reward: 1.700\tMoving Average: 0.331\n",
      "Episodes 0860-0870\tMax Reward: 1.000\tMoving Average: 0.352\n",
      "Episodes 0870-0880\tMax Reward: 0.700\tMoving Average: 0.343\n",
      "Episodes 0880-0890\tMax Reward: 2.100\tMoving Average: 0.366\n",
      "Episodes 0890-0900\tMax Reward: 2.900\tMoving Average: 0.403\n",
      "Episodes 0900-0910\tMax Reward: 1.100\tMoving Average: 0.414\n",
      "Episodes 0910-0920\tMax Reward: 2.500\tMoving Average: 0.419\n",
      "Episodes 0920-0930\tMax Reward: 2.700\tMoving Average: 0.470\n",
      "<-- Environment solved in 837 episodes!                 \n",
      "<-- Moving Average: 0.502 over past 100 episodes\n",
      "Episodes 0930-0940\tMax Reward: 1.700\tMoving Average: 0.509\n",
      "Episodes 0940-0950\tMax Reward: 1.300\tMoving Average: 0.477\n",
      "Episodes 0950-0960\tMax Reward: 2.100\tMoving Average: 0.506\n",
      "<-- Best episode so far!                \n",
      "Episode 0964\tMax Reward: 4.800\tMoving Average: 0.564\n",
      "Episodes 0960-0970\tMax Reward: 4.800\tMoving Average: 0.620\n",
      "Episodes 0970-0980\tMax Reward: 1.600\tMoving Average: 0.640\n",
      "<-- Best episode so far!                \n",
      "Episode 0985\tMax Reward: 5.200\tMoving Average: 0.699\n",
      "<-- Best episode so far!                \n",
      "Episode 0987\tMax Reward: 5.200\tMoving Average: 0.731\n",
      "Episodes 0980-0990\tMax Reward: 5.200\tMoving Average: 0.742\n",
      "Episodes 0990-1000\tMax Reward: 1.500\tMoving Average: 0.715\n",
      "<-- Best episode so far!                \n",
      "Episode 1001\tMax Reward: 5.200\tMoving Average: 0.765\n",
      "Episodes 1000-1010\tMax Reward: 5.200\tMoving Average: 0.801\n",
      "Episodes 1010-1020\tMax Reward: 3.300\tMoving Average: 0.823\n",
      "Episodes 1020-1030\tMax Reward: 0.900\tMoving Average: 0.772\n",
      "Episodes 1030-1040\tMax Reward: 3.900\tMoving Average: 0.879\n",
      "Episodes 1040-1050\tMax Reward: 3.000\tMoving Average: 0.907\n",
      "Episodes 1050-1060\tMax Reward: 2.300\tMoving Average: 0.893\n",
      "Episodes 1060-1070\tMax Reward: 4.500\tMoving Average: 0.819\n",
      "Episodes 1070-1080\tMax Reward: 0.300\tMoving Average: 0.786\n",
      "Episodes 1080-1090\tMax Reward: 0.500\tMoving Average: 0.663\n",
      "Episodes 1090-1100\tMax Reward: 0.300\tMoving Average: 0.631\n",
      "Episodes 1100-1110\tMax Reward: 0.300\tMoving Average: 0.526\n",
      "Episodes 1110-1120\tMax Reward: 0.300\tMoving Average: 0.457\n",
      "Episodes 1120-1130\tMax Reward: 0.300\tMoving Average: 0.443\n",
      "Episodes 1130-1140\tMax Reward: 0.300\tMoving Average: 0.286\n",
      "Episodes 1140-1150\tMax Reward: 0.300\tMoving Average: 0.228\n",
      "Episodes 1150-1160\tMax Reward: 0.900\tMoving Average: 0.198\n",
      "Episodes 1160-1170\tMax Reward: 0.300\tMoving Average: 0.142\n",
      "Episodes 1170-1180\tMax Reward: 0.300\tMoving Average: 0.143\n",
      "Episodes 1180-1190\tMax Reward: 0.500\tMoving Average: 0.141\n",
      "Episodes 1190-1200\tMax Reward: 0.700\tMoving Average: 0.161\n",
      "Episodes 1200-1210\tMax Reward: 1.100\tMoving Average: 0.184\n",
      "Episodes 1210-1220\tMax Reward: 0.500\tMoving Average: 0.182\n",
      "Episodes 1220-1230\tMax Reward: 0.900\tMoving Average: 0.206\n",
      "Episodes 1230-1240\tMax Reward: 0.500\tMoving Average: 0.211\n",
      "Episodes 1240-1250\tMax Reward: 1.300\tMoving Average: 0.231\n",
      "Episodes 1250-1260\tMax Reward: 2.700\tMoving Average: 0.262\n",
      "Episodes 1260-1270\tMax Reward: 1.300\tMoving Average: 0.283\n",
      "Episodes 1270-1280\tMax Reward: 0.500\tMoving Average: 0.290\n",
      "Episodes 1280-1290\tMax Reward: 0.300\tMoving Average: 0.287\n",
      "Episodes 1290-1300\tMax Reward: 0.300\tMoving Average: 0.271\n",
      "Episodes 1300-1310\tMax Reward: 0.100\tMoving Average: 0.242\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episodes 1310-1320\tMax Reward: 0.100\tMoving Average: 0.239\n",
      "Episodes 1320-1330\tMax Reward: 0.100\tMoving Average: 0.211\n",
      "Episodes 1330-1340\tMax Reward: 0.300\tMoving Average: 0.206\n",
      "Episodes 1340-1350\tMax Reward: 0.500\tMoving Average: 0.189\n",
      "Episodes 1350-1360\tMax Reward: 0.300\tMoving Average: 0.143\n",
      "Episodes 1360-1370\tMax Reward: 0.600\tMoving Average: 0.125\n",
      "Episodes 1370-1380\tMax Reward: 1.900\tMoving Average: 0.138\n",
      "Episodes 1380-1390\tMax Reward: 1.700\tMoving Average: 0.178\n",
      "Episodes 1390-1400\tMax Reward: 5.200\tMoving Average: 0.277\n",
      "Episodes 1400-1410\tMax Reward: 0.300\tMoving Average: 0.280\n",
      "Episodes 1410-1420\tMax Reward: 2.800\tMoving Average: 0.318\n",
      "Episodes 1420-1430\tMax Reward: 2.100\tMoving Average: 0.361\n",
      "Episodes 1430-1440\tMax Reward: 4.100\tMoving Average: 0.424\n",
      "Episodes 1440-1450\tMax Reward: 2.500\tMoving Average: 0.479\n",
      "<-- Training stopped. Best score not matched or exceeded for 200 episodes\n"
     ]
    }
   ],
   "source": [
    "# run the training loop\n",
    "\n",
    "# BUFFER_SIZE = int(1e6)  # replay buffer size\n",
    "# BATCH_SIZE = 128        # minibatch size\n",
    "# LR_ACTOR = 1e-3         # learning rate of the actor\n",
    "# LR_CRITIC = 1e-3        # learning rate of the critic\n",
    "# WEIGHT_DECAY = 0        # L2 weight decay\n",
    "# LEARN_EVERY = 1         # learning timestep interval\n",
    "# LEARN_NUM = 5           # number of learning passes\n",
    "# GAMMA = 0.99            # discount factor\n",
    "# TAU = 8e-3              # for soft update of target parameters\n",
    "# OU_SIGMA = 0.2          # Ornstein-Uhlenbeck noise parameter, volatility\n",
    "# OU_THETA = 0.15          # Ornstein-Uhlenbeck noise parameter, speed of mean reversion\n",
    "# EPS_START = 5.0         # initial value for epsilon in noise decay process in Agent.act()\n",
    "# EPS_EP_END = 300        # episode to end the noise decay process\n",
    "# EPS_FINAL = 0           # final value for epsilon after decay\n",
    "\n",
    "scores, avgs = maddpg()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJztnXd8lEX+x9+zmx5qINSACUrvGtphAbF3ORXs5c5y6nlyTb27n6Le2U49y4n1bCdiQUUPLICAiAUMCAFCCxAwICEkpPfd+f2xJbubrcm2ZL/v12tf2WeeeZ757mz2M/N8Z+Y7SmuNIAiC0PExRNoAQRAEITyI4AuCIMQIIviCIAgxggi+IAhCjCCCLwiCECOI4AuCIMQIIviCIAgxggi+IAhCjCCCLwiCECPERdoAR3r27KkzMzMjbYYgCEK7Yf369Ue01un+5I0qwc/MzCQnJyfSZgiCILQblFL7/M0rLh1BEIQYQQRfEAQhRhDBFwRBiBGiyofvjsbGRgoLC6mrq4u0KTFDUlISGRkZxMfHR9oUQRCCSNQLfmFhIZ07dyYzMxOlVKTN6fBorSkpKaGwsJCsrKxImyMIQhCJepdOXV0dPXr0ELEPE0opevToIU9UgtABiXrBB0Tsw4zUtyB0TNqF4AuCEHzKaxqZ8+5GDpVbnuY+2/wzpdUNEbaq/bNiexE/l9dG2gy3iOD7gVKKq666yn7c1NREeno65513nlO+iy66iMmTJzulzZ07l/79+zNu3DgGDx7MzJkzycvLs5+fNm0aQ4cOZcyYMQwbNozbb7+dsrIy+3mj0ci4ceMYNWoUl156KTU1NQAUFRVxxRVXMGjQIE444QSmTJnCRx99FIqPL3RQ7nz3Rz768QCTH/6Skqp6fjN/Aze+KQsf28oNr+dwwb+/ibQZbhHB94PU1FS2bNlCba2l1V62bBn9+/d3ylNWVsb69espLy9nz549TufmzJnDxo0b2bVrF7NmzeLUU0+luLjYfn7+/Pnk5uaSm5tLYmIiF154of1ccnIyGzduZMuWLSQkJPDCCy+gteaiiy7i5JNPZs+ePaxfv5533nmHwsLCENaC0NE4WNY8TtNo0gD8VFoTKXM6FMWV9ZE2wS0i+H5yzjnnsGTJEgAWLFjA5Zdf7nT+ww8/5Pzzz2f27Nm88847Hu8za9YszjjjDN5+++0W5xISEnjsscfYv38/mzZtanH+pJNOIj8/nxUrVpCQkMAtt9xiP3fMMcfw29/+trUfT4hxZNgmNoj6aZmO3P+/reQdrAjqPUf068J954/0mW/27Nk88MADnHfeeeTm5nLDDTfw9ddf288vWLCAe++9l969e/PLX/6Sv/zlLx7vdfzxx7N9+3a354xGI2PHjmX79u2MHTvWnt7U1MRnn33GWWedxdatWzn++OMD+JSCIAgh7uErpQqUUpuVUhuVUu3aOThmzBgKCgpYsGAB55xzjtO5oqIidu3axYknnsiQIUOIj49ny5YtHu+ltfZaluP52tpaxo0bR3Z2NgMHDuRXv/pVi/y33XYbY8eOZcKECQF+KkFwxvt/ptDeCUcPf7rW+kgwbuRPTzyUXHDBBfzxj39k1apVlJSU2NPfe+89jh49al+oVFFRwYIFC/jHP/7h9j4//vgj2dnZbs+ZTCY2b97M8OHDgWYfviMjR47kgw8+sB8/99xzHDlyxOM9BcEXNo+Oj76I0M4RH34A3HDDDdx3332MHj3aKX3BggV8/vnnFBQUUFBQYB9EdccHH3zA0qVLW4wBgCWMxD333MOAAQMYM2aMRztOPfVU6urqeP755+1pttk7guAvWvrzMUeoe/gaWKqU0sCLWuuXQlxeSMnIyOCOO+5wSisoKGDfvn1O0zGzsrLo2rUra9euBeBf//oXb731FtXV1YwaNYoVK1aQnt68X8GVV15JYmIi9fX1nHbaaXz88cde7VBKsWjRIubMmcNjjz1Geno6qampPProo0H8tIIgdDRCLfgnaq0PKKV6AcuUUtu11qsdMyilbgJuAhg4cGCIzWkdVVVVLdKmTZvGtGnTADhw4ECL8xs2bABg0qRJzJ071+O9V61aFXDZAH379vU6G0gQfKFQjgdWpNffkQmpS0drfcD69zDwETDRTZ6XtNbZWutsx16vIAihxdGl4yT+QoclZIKvlEpVSnW2vQfOADxPXREEQRBCSihdOr2Bj6yBuOKAt7XWn4ewPEEQ2ojM0unYhEzwtdZ7gLE+MwqCEBEcxV1W2sYGMi1TEAQhRhDBF4QYoaiijqM+wh+LR6djI4IfIV544QXefPPNSJshxBCTHvqS8Q8usx87invzSluR/I5Muwqe1pFwjHQpCJFGdjmLDaSH74OCggKGDRvGddddx5AhQ7jyyitZvnw5U6dOZfDgwaxbtw6A0tJSLrroIsaMGcPkyZPJzc3FbDaTmZnptKHJ4MGDKSoqYu7cuTz++OOAZRHXXXfdxcSJExkyZIg9CmdNTQ2XXXYZI0aM4OKLL2bSpEnk5LSMQffAAw8wYcIERo0axU033YTWmu3btzNxYvOyh4KCAntIiE8//ZRhw4ZxwgkncMcdd7TYyEUQhI5J++rh33knuAQSazPjxsFTT3nNkp+fz/vvv8+rr77KhAkTePvtt1mzZg2ffPIJDz30EIsWLeK+++5j/PjxLFq0iBUrVnDNNdewceNGLrzwQj766COuv/561q5dyzHHHEPv3r1blNHU1MS6dev49NNPuf/++1m+fDnz5s2je/fu5OXlsWXLFsaNG+fWvttvv517770XgKuvvprFixdz/vnn09DQwN69e8nKyuLdd99l1qxZ1NXVcfPNN7N69WqysrLcxvQRYgN37htx6HRspIfvB1lZWYwePRqDwcDIkSOZMWMGSilGjx5NQUEBAGvWrOHqq68GLMHNSkpKqKioYNasWbz77rsAvPPOO8yaNcttGTNnzgTghBNOcLrn7NmzARg1apTHgGorV65k0qRJjB49mhUrVrB161YALrvsMnvZNsHfvn07gwYNskf2FMEXhOAR7WMg7auH76MnHioSExPt7w0Gg/3YYDDQ1NTk9dopU6aQn59PcXExixYt4m9/+5vXMoxGo897OlJXV8ett95KTk4OAwYMYO7cudTVWbaumzVrFpdeeikzZ85EKcXgwYNbhFoWBCF2kB5+kDjppJOYP38+YAmI1rNnT7p06YJSiosvvpjf//73DB8+nB49evh9z6lTp/Lee+8BkJeXx+bNm1vksYl7z549qaqqYuHChfZzxx57LEajkQcffND+ZDF06FD27Nljf4qwPQEIsYe7vmiUd1CFNtK+evhRzNy5c7nhhhsYM2YMKSkpvPHGG/Zzs2bNYsKECbz++usB3fPWW2/l2muvZcSIEQwbNoyRI0fStWtXpzzdunXjxhtvZNSoUfTp06fFrlezZs3iT3/6E3v37gUsG6rMmzePs846i9TUVNklSwCi3xXRXoj2alTR9EVnZ2dr11ko27Zts+/+FGuYTCYaGxtJSkpi9+7dnHbaaezYsYOEhIQ23beqqopOnTqhtea2225j8ODBzJkzxylPLNd7RyXz7iUAFDxyLgCnPrGKPcXVAKz/22mc8PfldEmKI3fumRGzsb1jNmsG/eVToLmeQ41Sar3W2q/t7qSHH8XU1NQwffp0Ghsb0Vozb968Nos9wMsvv8wbb7xBQ0MD48eP5+abbw6CtUK7I3r6ekKYEMGPYjp37ux23n1bmTNnTosevRDbiPYHh2ivx3YxaBtNbqdYQOo7dgnkm/+ptIbjH1zG/hLZT7m9EPWCn5SURElJiYhQmNBaU1JSQlJSUqRNEUKM219UAD+zDzYUUlrdwMINhcEySQgxUe/SycjIoLCwkOLi4kibEjMkJSWRkZERaTOEMNKm/pR0xuxEe8c06gU/Pj7evipUEARBaD1R79IRBCE0BC2WjkTabDeI4AuCgG7L/JIod2OEk2ivCRF8QRBahUJ69u0NEXxBiFHcx9KJ9j6q0BZE8AVBsCNy37ERwRcEoU1KL41EM9H+gCSCLwgxSrSLkxB8RPAFQbAjjUDHRgRfEIRWuWVk+n1L2jS9NQyI4AtCjBLt4iQEHxF8QRDsSCPQsRHBFwShTb578fs3409dFFXU8fCn2zCbw19xIRd8pZRRKfWjUmpxqMsSBMF/RKgjwx/f38SLq/ewdm9p2MsORw//d8C2MJQjCEIbkUYg9DSazEBk3GchFXylVAZwLvBKKMsRBKFttEZ8ZJJO+yPUPfyngD8D5hCXIwhCgLjrzUsHP/w88tl2zn3m67CUFTLBV0qdBxzWWq/3ke8mpVSOUipHdrUSBCHWKK9t5HBlfVjKCmUPfypwgVKqAHgHOFUp9ZZrJq31S1rrbK11dnp6egjNEQTBE22apSPPBXZaV4/hq7+QCb7W+h6tdYbWOhOYDazQWl8VqvIEQQgCot0RIVzjITIPXxAErxRV1JF59xLez/nJ7XnZCKX9EBbB11qv0lqfF46yBEEIHG8d+93FVQAsXF/o4Vp5LLDRmroI51RY6eELQozifhPzlmm2HrzIeugIVyA6EXxBiDG8bWPo7pRExew4iOALQozx3+/3BZTfrvcujYE0BMFBXDqCIISM3MJywFm/vfX6lbK5dMSp44vWine4Br5F8AUhxvAmLd70SuLstH9E8AVB8Iov1400BG0jnE9OIviCEGPYBNxRqP0RbdF137S2jmSWjiAIYcedL9+gPJ8T2o4M2gqCEBHca4/7efhKpukEDQmtIAhCSGheSBVY11I6+L6J9qcgEXxBELwiHfnQEs4mQgRfEGIMbwLurYPq6VR092nbB+Fyj4ngC0KM4u8sHeVPJgGI/sZPBF8QYoxAO5PNK22FUCCzdARBCCGBKb4tt3Tw2z8i+IIQozjF0vHSf5eVtv4T7XUhgi8Igl9I8LTQIKEVBEGIGuzz9j3okkzbbDsSWkEQhJAQaCwdd/kdiXY3RlhpTV3IoK0gCNGG6HrokB6+IAghwR9t2XukGpPZIvHisuk4iOALQoyhPO1ZaGV/SQ3TH1/F40t3OKW7xomRhiA4SGgFQRDCiqPoHK6sA2DtnhIgfNvvdQRaO+NGtjgUBCEk+BIX15679OQ7DiL4ghCj+Du7xucsHRnObRPhDKksgi8IglvRsaW0Nn5+LNJa7ZZZOoIgRAj36iPz7ds/IviCEGPYXTQ+8tkE3lPPXgZzg4PM0hEEIWS4k2lH0fHkXpAOvm9aW0ftfk9bpVSSUmqdUmqTUmqrUur+UJUlCELwcRWvaN+vtb0SzmqNC+G964FTtdZVSql4YI1S6jOt9fchLFMQBB/YNzTxoDSuvc1m144HpB1oM+Ha4jBkgq8t/01V1sN460v+NQQhCnEKpOYxUzgsad9E+1NQSH34SimjUmojcBhYprVeG8ryBEEIPo4a9kNBKTOeWEVtgylyBnUwOsygrdbapLUeB2QAE5VSo1zzKKVuUkrlKKVyiouLQ2mOIAgOeBIaT84FDTy4OI/dxdXsLKqUFbhBpN0P2jqitS4DVgJnuTn3ktY6W2udnZ6eHg5zBEFogRv5d3FPRLu7IhqI9hoK5SyddKVUN+v7ZOB0YHuoyhMEITi4DiDa5uFHu5i1V6IytIJS6kSl1PXW9+lKqSwfl/QFViqlcoEfsPjwF7feVEEQMu9ewh/f39Sme7TWFSMd/BASTaEVlFL3AXcB91iT4oG3vF2jtc7VWo/XWo/RWo/SWj/QNlMFQQBYuL6wTde726PWnZhrL+cE90R7Xfnbw78YuACoBtBaHwQ6h8ooQRAih+dB2yhXs3ZKNM7SabDOq9cASqnU0JkkCEIo8del49pb9TRXX5oB39Q1mjhYVuvxfLTN0nlPKfUi0E0pdSOwHHg5dGYJghBqPK609RRLR5S91dz4Zg6/eGRFpM3wb6Wt1vpxpdTpQAUwFLhXa70spJYJghASfAVPa04ThQ8UT3X29a4j3i4KGz4FXyllBJZrracDIvKC0MFxDXvsqWcv666CR7hi6fh06WitTYBZKdU1DPYIgtAK9pVUs37f0YCu8bdjeajCsqn5gbJa9pXUAM5iH+0Lsn4oKOWn0ppIm+GRcD5J+Rs8rQrYrJRahnWmDoDW+o6QWCUIQkCc8s9VABQ8cq7PvO46k26nZVrTbnwzx55WXtvYGvMiyqUvfAf4VzdtprVbHAbXCo/4K/gfWl+CIHRwJEZOx8XfQds3lFIJwBBr0g6tdftr6gVBaPYX++iNevPUaKRhCBZRtwGKUmoa8AZQgOXpY4BS6lqt9erQmSYIQriQGTnBodVbHIap8fTXpfMEcIbWegeAUmoIsAA4IVSGCYIQGvzVFmkCOh7+LryKt4k9gNZ6J5Z4OoIgtFM8xsMPsLcZ5ZN0op6oc+kAOUqpV2gOmHYlkOMlvyAI0YqPWToi4K2ntXXnuvYhVPjbw/8NkAfcYX3lWdMEQQgC5TWNTH98FdsPVQR8baPJHAKLhFDguGbhp9IaTvnnSooq68JWvr+CHwc8rbWeqbWeCTwDGENnliDEFqt3FbP3SDXPfpkf8LXFlfWtKjPQWDot8sla2zbxxrf72FdSw4/7y8JWpr+C/yWQ7HCcjCWAmiAIQaB5pmTo/SnuhNr9wivx7QSKr+/PY5iKaNoABUjSWlfZDqzvU0JjkiDEHtHUWxad77j4K/jVSqnjbQdKqWzAc3BnQRBa8PCn28i8e4nXPOEQWz/XXQkhINJ17u8snTuB95VSB63HfYFZoTFJEDomL67e4/GcXYQjpAiy8Co4BPL9RWKlstcevlJqglKqj9b6B2AY8C7QCHwO7A2DfYIQE4Tzt++rLHHphA7HcZFI1LMvl86LQIP1/RTgL8BzwFHgpRDaJQgxRVgHbf18mvB2fr9DuGFpH9pOtMTDN2qtS63vZwEvaa0/0Fr/H3BcaE0ThFjC8oOPht61P43OHQt+ZFsr1gzEOp5qdtvP4alLn4KvlLL5+WcAjpsy+uv/FwQhynE7LdOH8BcelXkbrkRBe+0VX6K9APhKKXUEy6ycrwGUUscB5SG2TRA6JFrrFo/wkZg540nQo+Epo6PiFMIiAuV7FXyt9T+UUl9imZWzVDePOBiA34baOEGIFWzyH5ZpmT6GbUXvOy4+3TJa6+/dpO0MjTmC0PHR2tuUvOiRW+npB46v1cmOT1WRWGrn78IrQRBCiM3FE9aFVx7K8jekQjifSjoikag2EXxBCDPufujKy7lgIxugRI5IN44i+IIQZtz1oP2Zhv3qmsDXOm45UM6b3xX4YZPDez/vvfmAzNtwxZegv/iV59XW4SBkgq+UGqCUWqmUylNKbVVK/S5UZQlCR8GbO+WBxXkB3++8Z9dw78dbnRNt7iOPNvh375oGU8D2xDr/Wr6TyrqmiJUfyrn0TcAftNYblFKdgfVKqWVa68D/awWhA+HWpROFAc0kPHLHI2Q9fK31z1rrDdb3lcA2oH+oyhOE9oI7HY10eGTnOfki9KEkEkHTbITFh6+UygTGA2vDUZ4gtDsiES0zSGVJpM32Q8gFXynVCfgAuFNr3SJghFLqJqVUjlIqp7i4ONTmCELEcSeQ0TRLRzw5oUXZ4yaFv6JDKvhKqXgsYj9fa/2huzxa65e01tla6+z09PRQmiMIUU84RMCdS8F9LB0hUKK9sQzlLB0F/AfYprV+MlTlCEJ7w60P34djNxQNgcdYOkEvSXBHuEIiOxLKHv5U4GrgVKXURuvrnBCWJwjtlmD99DPvXsJ9H2/xUZavxiVIxsQIM55YxVlPrQ74uki4dEI2LVNrvYbIhIsQhHaL53AH/t/jje/2cf+FowIr1+9EwZXdxdX29/4MYEdykFtW2gpCmHHv0rGeC5GbxbE3GaxYOkLriGT1iuALQphxP0vHe/A0byIcKv3w977SPgSGCL4gxDjhHL/zOS0zLFYIkUAEXxDCjLcenscevtf7+eE39jHt0vEetrfi2gkcf6oskrUqgi8IYcZ7eGTZdrCjE8mGVARfEKIBq+J/v6eUWjdRKL3N7Ai0V+nLfSShElqH2axZsvnnSJvhFRF8QQgzvnp4f120Oex2uLNIZD8w3l//E//8YkekzfCKCL4gRBk/lda0SAumF8DnCk9R+lZxuKLer3wyS0cQYghfv/dAQyX75dJphcrIuEFosLnMIlG/IviCEGbc/tB9/PjbKg5mh+v/tWwnJVX1LrN0/DZF8EB7qDcRfEGIAtoiFoEu528ya+7+0PM4gfTsQ4u4dAQhlvDt03FzSdtUwlVk6hp970crs3VCg7b/7WDx8AVBaIm7H7qvXl9rFmv5tMPpOscZOyL0rcHf70Hm4QtCB+SZL3fx4OI8v/L6ElnXs6XVDfb3z6/a7fP+9/9vq9Ox60ydF77a4/MeHm0THxAQeEMpg7aC0IF4ctlO/rNmb4v0YPzQv9h6yP7+3ZyffOZfsM57nmV5Rfb3ot+tw+8efmjN8IoIviCEGbeTdBwS3U3KjEQvuj0IWLtEBm0FIbbxpQGBaoSvBsLbTP/AywrwghhHu/wNJyL4ghBmgrEIymeI4zaoSUfyyYfzs7iWlHn3Eu/5I1DPIviCEAW425GqTffzcd6fMvx36XScBiIcyCwdQYghWhWoLIwa0ZFcOmG1zc/CIlldIdvEXBAE9/jSBXexdALtRf9cXktaaoKXMoJHtOp9WU1DwHGJgo3W2mOwukjUmwi+IEQDAS688uWSOfHRlQzp3SkktrTIHqWKP+6BZU7HK3ccZvrQXiErz9PsK9fvSkIrCEIM4XalLcH14QPsLKpq1XWB++SjVPFdyDtYEdL7+9pGsjlNez4ZYkTwBSHctOKHHmxt8BkTn441WydSuKtD6eELQozjO5aOc4ZQ+qYDfTKQdsFCQUl1izRvVbPnSMv8oUYEXxDCjM+Vtm6jZYaPRz7bHlCZ7UXwzebQGro4t+V+tm7dPNLDFwShvdJe5uFHwkpzlLWGIviCEGb8HdzzdU1b8Mch5H+43zaZEjaixc5Ijo3ItExBCDPu4+E7zNLxYx5+k1nzyGfbaWgyB9/AAImEfP24/yi7DldxWfYAv6+JxJNItDQyNkIm+EqpV4HzgMNa61GhKkcQYgIX4fhwQyE5+45GxpYo4OJ53wIEJvgREF/3U3AjRyhdOq8DZ4Xw/oLQLmmNS8eVRlPbevZ+xdLx06po68V6IhJmxsygrdZ6NVAaqvsLQnvF1ywdf6+JFtrLoG0klNbrwqsIIIO2ghBh1u0t5Za31tuPbb3vJ5bu4MmlO4BQaFXw5vFLD99LmVFWOREXfKXUTUqpHKVUTnFxcaTNEYSQ4yoCf1q4yW2+Z1fk88yKfMs1LnIVXTLSPoiMD99NWkd06fiL1volrXW21jo7PT090uYIQsiJsk5fq1HazPTdP5BQXRlpU/wiWmbpSHhkQRC8EvR5+G3YACWr9AArX76Znzv1oG9VCV/W7YUbTgquga2gyWQmzui5DxuRhjbKGveQ9fCVUguA74ChSqlCpdSvQlWWIEQLN72Zw9j7lwb9vq66kVtYHvQy3JW5v6TGfty3opjLNi1l5cs3W46rSgDI2uneJdUarnl1HRP/sdxrntH3fdEi7dHPtzPivi+oaWgCIP9wy6eOyOh9dAVPC1kPX2t9eajuLQjRytK8opDcN9iDf746+CfvWU9tvwzW7x8GwODifSx79Tb7+SZlYP74s7l2wxIyCnZAXR0kJbXZrtU7fY/jVdY3tUh7ftVuy7m6JlIS4vhuT8sJgpEQWvfhe2SlrSDEDK7C03Jzk8ju0hRnauLN9+8D4IuTVoLWdrHf160PL02cyfzx5wDwQ8ZI/v3JY7BjB4wdGzGbbdhj17gNSxwJH3509fAjPmgrCO0RrTWFR2t8Z3TD0ZoGr+fdyX04RKJ/+WFe+vDv5D9+kT1t1DMP06/S0usuTunGKTe9bBd7gM19jsOsDHDVVVAeejeTL7wFxIwWF351Q8snlHAhgi8IreDtdfs58dGVbPypLOBrL3zumxBY1HbmLXqYM3Z9D0B5YiqHu/Sk/+qlfPv8DQDcfPFfW4z27uvej9du+wds2QK9e8O558Inn4Tddhv2HrWbp6TI9PBbptU1Ri7+kQi+ILSC9QWWODb5h1u3jWCghEqrkhvq6FFdBlpzbGkhP/Ydyqm/foHxd7zN765/lJLhYwCoSExlU78hbu+xYfIZ8OmncMUVsGoV3H57aIz1A2/1FC2xdCKJ+PAFoTUE0c3uKgrhcOHbynhnwT2MPbSLOef+nk4NtSwaOY09PTIA2Nf7GJb852PefnkxB7r2wmQwer7h2WdbXv37w9//DtXVkJoa+g/iglcffphtiVyhnpEeviB44UBZrVu3jS2EsT9ugur6JlbuONxqGz7fcijoPcWleUX0Lz/M2EO7APjXkicB2Nu9nz2PUgqTWbO9VxaViZ7F28m2ftbrP/ssqPbaOFrtffzDqw8/SlbaRhIRfEHwwtRHVnCRG5+7rYfszw/6rg9yuf61H/wu01WYbnlrPUs2t9w+ry1oDWfu/A6A7weMosEQx4IxZ7Amc5xTPpMf2wI62XvJJZa/+fkhUdhrXl3nUram2mGaplcffpSstI0k4tIRhFZgkxN/evi7iwPbrNrdpibFlfUB3cMbyQ11DC7Zz9R9G2kwxDH78oc9+pH82aLPKUvPntCnD9xzD8ybB48/DpddFiTLYcch5wVVZg1NJu107JedIWBgWgr7S51nbokPXxA6APYevh+/50Bnh7ibthcMsYozNfGvxU9w/vav7Wkb+g31KPZKgT9h951ETSlYvhzefdci9rfcAqecYpnBEwRcTXVtkLQ3H36IFT/aVtW6Q1w6guAHlXWNPLF0B01WBXS3DWFr+am01um4qi4087RPOLDNLvZ/n34D111yH9dc9qDH/Eq1oocPMHIkPPCAZXrm0aPw7bdtMdsJo8G53l3L9sMDFTTqm0w8/sUO+7Hr9wjR58OXHr4g+MHjX+zgje/2kdUzlZnHZwTkw/emmUUVdS3SJg3q0SLNH+H1xfDDewGY8evn2d3D99aACuWXD98j2dmWv7t3t/4eLhjCJ3GaAAAgAElEQVRcuvhmrZ3qxls9BVt8F6zdz79X5nvNYw5nC+QH0sMXBD+obTQBzVsLBuTS8SI17gSqc1LLflgwXAO/3PIl+WkZ7E7L8PsavwZtPZ3o1g169IAf/B+w9oWrS0dr5zq0v3XjpgpGo+lIfRRsIB8oIviC0CosgtIqlwfehdSdrzlQsepffpgrf/yUJ5Y8SWJTA0qbGVq8j6+zxvs90d/QWpeOI5dcAu+9ZxnMffppP633jKtLx9LDdz72bVT48MeMM3Z+x63fvRcWm8WlIwhA4dEabp2/gdeum0BygpER937BtKHNG/I4/hbzDlawYN1+AP62aAvH9erEuAHduNZlyqA3Ln3hWxbcNNntWIC7372tPF+kV5Xy56/e5NItzSGGp+zLpV/lEYvtvbL8ttE2D983zXlKqxu49tV1zLvyeAakpVhE3miEhQvhzjvhzDNh2DC/bbCRd7CCc575ukV6Q5OZy178zn5cUFLNec+uIaN7cou8b32/n7e+38/vTx9CVX0TfzlneEA23PvxFo5N78S1v8j0+xqN5uONB7zmuSDvK8b9vDMsK+6khy8IwCtf7yW3sJxFGw+yfp8lbMKqHe5D9f59SZ7T8R/e28ThinrW7m0Zkhfcuzw27C9jV1GV+5kdbvL7p7uaf3/yGJduWU5lQjJvjTsbwC72c2fcxMLRp/lxIwsKMAXYw1+ce5DNB8p5afUeS0JiIjz3HKxebTl+9llL+IUAtzN9YPFWt+l7jlSx90jztNd/W7eELDzacgDVxpPLdjbbFwBvfreP+z5xb4cntIbfvbPR7TmD2cTNaxdy3o41FHTvG7A9rUF6+ILggrtet6/Ol9fBQg/nahtNboW8NU/2Z29fw78/eQyjNvPShIt5ePr1aGVg2eDJXLNhMU9NvYLNfQcHfF//xiiasa9PcG22Bg+2rMKdN8/yMhph2TKYPt2hLG916D7dtf6ixJNjx5s5529bzT2rXgfghUmXcGIY7BHBFwQHFL7F3a1PvhWzQ2oaTG5ncfizWKdbbQWXbF5O/4pivs4cz7NWsX9v9Gk8cfLVaGV5eP9q0Al8NegEn/dzi/Jv0Nb5GlvICZd0g8HSy9+xA44cgeuvh/vucxJ8b4OgnhpUV/uCPTDbVjw1YseW/MRjn1nGNC6+6nF+7B+4m6s1iOALMcFLq3fz0Kfb2fWPs4l32Pf0p9IaTnpsJYN6WmLFeJILX7rnKtwb9h9tPvBwrSef/2vfFPDaNwVuzz30+bNcscl5i7/r1/8PgBcnzuTh6Td4NzQAFP6GVvAzUNmxx1peAOvXwzPPwP79MHAgAFVudrKy0SUp3kPZzsdRJ/hu0ibvz+WdBX+h0WDkl1c+FjaxB/HhCzHC08stQcLqrNMrbdiEeY+DH9hdB99R+Nw9Abj28L/Yesj+vrUSlNRYx7TdPzD24A7iTE1ctWGJXezfGXMG9512M4+eci3lial8PmQKT029opUlucffQVt3Lh2f2MIt5OTYkxxDSrg2oFOObbk2wVK260pbfw0ID672ZJUe4JUPLIvdLrr6CdZnjAirPdLDF2IKv/TAnaBbBcjTCltXYWztStwT9/5I5tGD9Kwp48K8VWQdtQRNq41LJLnJEk/nukvuY9WxE+zXPD/50laV5QtF66dl+rxsvHV66Lx5MHMm4FyHZq0xONShp4anxXaRPq0NHv5Nqmm26MKtK3l68RMArO83jK19jguNYV4QwRfaPQfLaunTJQkNHK6so29Xy5S8moYm6hrNpKUm2H92vlY+7jhUQXrnxBbp236uAKCwrJaDZW6W0Hu5bY2fW9rduWY+d36zwGIniqJOaSwacQplSZ3JKC+iLj6JB079NYc7u+/thoJD5S1XAruisbh18g9X2VcOV9U3UVJlaaCO1jRyoKyW4X07g4aEOANxxgSMZ59L8qeLKf58BZ1nnOK06tikNWVV9STFG6mpb6LJw/d2uNLZvkBcOjUNTRiUorKuye13DlBe24hBQWcHl9LBslr6dUv262nisDXo3QV5X9nFfvqNLzqFoQ4nIvhCu2ZfSTWn/HMVfzxjCPVNZp5dkc+3d59Kv27JnP/sGnYXV1PwyLl2IfDlongvp5D3cgpbpO+y7mz1zJe73F7nel9Hv3ZRhe9Il53qa7j1u/c53CmN6395L/u696MqMcXndcHEaHB24ZjMmi+3+47jr7Xm/ZxC/vxBrj3tf5sO8r9NB71elzz0OrZ+/hkpF5zL7Nn/YGO/ofZz236udApL/dtT3feG57y7yenYk+CPPJRP9oFtzMhfR4+acrrVVlI7z8TRnn1YOHAid//fVTBjRovrxt6/FIOCeVc2D3z/4pEVfPWnaV4/W7faCk7LX8cXt/yPP1eVcOv3CwE499qn2JvW3+u1oUQEX2jXHLD2tr/edYSaBot//nBlPf26JTuFJbbpQJtiw3jB1YfvradpNJvoXVXClH2baTQaOXvHt5y90xJg7J5L/8LWXuF/1AeYPCiNb/JL7Me2+vRFo8nMugL3axC8UZuQxLdjTuLEjat4/f37GHfHArufpLKu0Tmvn7aY3Uz0uX/Z81y7YQkAhV3S2du9PwXd+1KdkMz0PTncvfd1+Op1i5vpn/+ECRMgLg5SLA2uWcOmQudNcH728uQzffcPvPLBgxh1szE7eg7k2V/MjogbxxERfCEmsMmvPwuJWoOrq8htu6I1V/+4hP/78hUSzM5unp09BrJo5DTWZwyHhsiMPLqKpbswze5wF7/fXx69/gG+/eAV/rz6Te5e9RqPWGcZ1bts9F3b6H/jY+Pa9f/j7lWvk9xUz97uffntBXextfcg+5RVsCx+6lJfzfr4DRifnwenWRemKQVjxvCLYZfyrcumML54YNkLVCam8PTUy/l4xDTKkjph9rY9ZBjpEIJfXFlPQUk1EzLTIm1KxFi14zATs9JISfD+lX6bf4T0zol89OMB7pgxmN3FVdQ0mDAoOOEY5/orOFJNcVU9dY0mThqczsrth5k8qAfJCe7/eZtMZr7cfpgzRvRGKcX3e0rYXVzFFRMHooKwbPy73SX07JTA0ZpGJmY52+q4ytVtSVYN3VVURVlNIyu2H+bWace22SYbrgL/wYZCelSX0au6lL3d+3Hq7hzuX/4C6dVl5KdlsHTIZFYcO4GqhBQqklI52KUXAMnaAPgnbsHG9amk0s8wzQ3+BM33gFbw4qRfMnvTF9yy7kNWHDeRpMZ6quqdRdZXDz+z9AC3fv8+SXEGehcfYGDZIfpWlZCflsGKYyfwyoSL3I59mA1GypK7UD33Ibrc9WeYPx9MJjhwAP77X97e9DcqElLgP104JbknjYY47jrnDpbnFdGrS0u/f3JDHQPKi3js5Gt4LfvCVtdLqOgQgn/xvG8oPFpLwSPnRtqUiFBwpJrrXvuBC8f14+nZ4z3mq20wccUra+3HlXVN/Pf7fc33cam/aY+vsr9/9bpsbng9h8snDuDhmWPc3n/eqt08uWwn/7k2mxnDezP7pe8B6NMliRnD27YBhsmsufzl7+3Hex8+J6BGxCZK1722zi7OI/p1aZNNrvYB9K48wm++X0jm0Z+ZWLiFlEZn//2RlK6c+avnPG4I3uTOJ9EKRvTtQp51oNlf/BV4V9rSw9caTAYjvz/v9yycfxfvvX03AO/8+iKnfI49/GOOHuTyjZ8z+actFHVKo2d1GScc3A5Acafu1BoTWN9/OEdSu/HUiVdQluz7e65rMNElPd0S78fGnDnMv+AmmgxGTjEdoevBIoYXF/Dt8zfw3prT+P6q23GUUIPZxLUbFgPwc+eera2SkNIhBN8WN0NrHZSeZHvDtmAl3zqw6InKeme/6I4i5+3iTGbdIhqhjf0llq3bXLdwc2R3saX8shrncg76MdPDF66iUt9kJik+8Mdkx574ofI6Ujw8rfhCaTO/2JdL17oqOndJocuqw9y4dhnXbvgf6dVH2detH/u69WV3WgZ70/rTZDDyddZ49qT19yj2EJx55OeP7Ue/rkkBC/4ZI3sHfA20TfBt5GSM5J8nXc1v1i6kU0MtGR+/y7TD8cSbTdTEJ9Ej+Rgu2rqREws2MnPLCgxoSpO7kNpQi0kZ+HzIFF6aMJO8rJHUNQZuj9trBgzgr2feDsCvT8zilTV7OW3XWu5f9gKXbV6O8cs+fDh+NmhNUlM9N6/9kDnfvE1pche+OWZsW6skJHQIwbfRaNIkxMWe4PtLdb3J5di5R1fd0ORxRaOth5xg9LxWz/bDT4gzOPm0gyEIrveoqm/yKPj+Ts1raDIHJPjxpkamFmxiVFE+szctJaPCYQbLf2EksC09kzvO/zMbMgKLxNgWJmR254eC5pW9BuW//92RVB/uQE+05ft1HER/7hezeG/M6ax66SZOnPeQ29gydXEJfHncRJ6aejlbex/bYjJ8nKl1LaavMQLbtNDlgyexfPAkPvjvH7l42XyGbvqOUUXNG7zkp2Vw3nVPURef1Co7Qk2HEvyDZbVk9kxlc2E5izYeYFifzgxKT3XyTS/OPUhaSgLf7y2lur6J66dm8u4PP3HFpIEsWLufS04YwKvf7OVAWS0nHteT43p14vrXf+CskX24bmomS3J/pk+XJG48eZBbG97P+YkhvTvz5bYinlmRzxWTBvLQxaPt51fvLOarncWMH9iNXUVV5BdXMbR3Z2ZNGEDvLkm8vHoPn2w6yP7SGm48KYvFuT9z0fj+PPKZ5ZH18okD7aFyTxmSzpYD5Zw0uPnx8UhVPdMfX8W9543g8aU7uG36cWR0T2bVjmIW5/7sZOvWg869uUn/+JI7Zgxmce5Bhvd1fgx+6FNL+St3FJN1zxK0tmzafPfZw7h1/gb6d0u2z5j57YIfufPd5giBDy7O49j0VJZvK2JzYTkTs9I4b0w/bnwzh8OV9Sy6bSqfbv6Z0uoGHvvlGP60MJcN+4/aoyB+e/ep3P3hZid7Jj/0JZdNGGDfctAR1znbH6xvOc0S4NkV+Rypcj9lMrGxnmHFBaRXl5FVeoBBpYWcvfNbutVZnmK2pWfyf6ffQl6vQdTHJRBvaqI2PpHt6ZltCnNrNCiPc8490dmlkW4ya2obAhdhT2MzvjhYXsdCD3Xsiz0uG7wXd0rj9F/PY4q5lL1VGpPBQGpDLcNriik0pvLFkCle6zfQurPxm/nrGZvRzeN51xDHfzh3Dk//73HGHrJM093RcyBLB0/h7XFnRa3YA6hQbuyrlDoLeBowAq9orR/xlj87O1vnOCy19pexv3uHrKMH0ZMm8fFtU7nwuW/Y9FPzNCpH33Tm3UsCvr87Nvzf6aSlJrRId3d/R3+zp/KnDOrB2zdOIuueT1tt08h+XejfLZmleUWtvkekeWrWOKfGojUsuHEyU47t4fd3rbSZrnVVXHwolxF5OQwsP8TYn3eS1NRgz1Mdn8TSwZP53/CT2dxnMMWdunu838h+XVo0pjamHteDwb068/q3Bfa0AWnJdE6M56Lx/TAoxd+XbGtx3bA+nTl3dF+eWLYTgE6JcVTVNzF2QDcev2QMp/9rtT3vF3eeTHlto1OceIAHLhxJakIcf3jfee66jQ3/dzp/X5zHhz+2jN/+9Oxx5BaW8581ez1+7taQ0T3ZKZTxiL5dUMoynuDoPhyYluLVnQjQLSW+hTsRLGNIfbsl8eP+MjdXNTMwzXndQ2l1g91d2q9rkkfXZHJDHbUJzSI/qn8XthwI3DXW2jFIpdR6rXW2P3lD1sNXShmB54DTgULgB6XUJ1rrPO9XBs7LHz7IxMI85pXcBLdN5VC551jYwcLdI6CnxtMff3NFXWNQtkwrqW6wGUOc2UTn+mpMBiMNxjgajPGkNtaRXnWU3lWlpNWU06mhhs711fSsKafREEd1QjLlSZ2INzWS1NRAevVRetSUk9pQS2JTI8mNdXRuqKF7TQXx5ibqjQnUxSfSaDBSnNqdoyldMCkDJoOR0uQuGLSZtNoKtFLUxSVQF5dIZWIKpcldqE5IodEYR70xnuLU7tQkJGEs6EqfiiOkNNah0DQY42kwxmFWRkCjtMagNQrLy6A1SY319KksoVtdJT2ry0jrUQCZvbgg7weMZhNx5ia61NeQefQgfSuKSW6qZ1DJAQxo6uIS6FNZQqLJIhRHUrryU9c+zB93Njn9h1Mbn8S2XpmUpnSl0RjPwLQUil2E5+nZ4+wxz9+/ZQoTMtPsjc20oencMDWLa15dx0mDe/LfX00CsAu+u8HnX580qEVj9fmdJwPYBX/L/Wc6nXcnFp4ExJPgp6Um8OSscTw5yzJDZuojK+xPbReO68+F4/rbBb/gkXPtNp40uCdf77LE3D+mRwr7SjwL88TMNC7NzuBPC3O5eHx//jXL85RH2/1dP0f+4UpOe3K1U9rXf55O56Q4xj2wzOtnd8SxYzg2oysf3+7sRPr74jxeWbOXX5+Yxd/OG+FkE8BxvTqx/PeneC3j5dV7+Men25xsOlJVT/bflxNvVOQ9cBaD//qZT1uDRShdOhOBfK31HgCl1DvAhUDQBd9gFdpbP30JztrHrbXdSTpaQk1CEo2GOHh0C6SnQ2oqZ+7MpdEQh1kZSDA1YjSbMBmMNBmMKDRxJhMGbSbObKLRGIdGkdpoETuzUpiVQisDhjd/gu6plrCv1lejGc7evpF4s4kEq2CmNNTS9Gw+dOsMSUmctms7ZqUwajMGbbYL2DFFiTT+dw8Xbd2M0WymW10lXeqq6NRQS2pDLUlN9SQ0NRJvFbB4UxOdGmrpX3GY5MZ6jGYzRjQGswmD2YwhwKgiDYY4jNrstFgELDFcSlK6UpmYQn1cArXxiRR27c3WXsdSHxdvaQSa6kkwNdKrqpT+FYct9zGbSK8+ahX+rgAkNjWQ1FRP5wYvDfJrcH5gX39LvrT8ecYluTwxlcKuvamLS+C7Y8ZQb4wnuame2rhE9qb15+RfXczVm3Gap+2Ku0FtR8F2XaxkVMoeWsHdeEFHmGTg+BlcNxn3el0ry0tNbClbnRLj3Kb7W36cm7Epm6vM00fyNp7VfN+WF3ey2tnYyvGGthBKwe8P/ORwXAhMCkVB2Qe2sS09k8+GTuWy9V9x7ZFCGgxxaGX98f7wkT3vi8Eq1I3nJQF43l3eVc1vX/F2z9fhKYdDM4qahCSqE5KpjUukwRhPk9HSW28yxFGRmMq2YydSnZCMyWDErAyYlcKkDJgNBkzKQGViKgatrQ1QPTXxSRR1SqM0pSuHOvWgKjGF6oRkqhKSMSsDKY11dK6vpsEYT31cAjXxSUHfek1pMz2ryy2NmO1JouooiU0N9FYNNNbUUZ2QBCjiTU3EmxqJ02Y0YFaWpkxbG19Q1MfFU5yaRnFqNyoSU8lK0nQxmNld1kCTwYjJYKQ6IYmypM5eP8uYMePRW370antqYkvRTnD4UTe6PKUlxBlosP6wu6e0dAFGgjg/xwncbabujlSHhsxd/TiSnGAkIc4ilImtmGUFuF1rkpoY5xT22h86OTQQ7j5rpyTvwtw12f0EB0eS3XzGxLhmO23/OamtHD8JlIgP2iqlbgJuAhhojYsdKN9OPot30kdTe8llPGScg7G6mu8P1tCAon/3FIZ0UnSqPEpCQx3b9pXQI0Fx4EgVDcY4TjiuNzm7i5ma2ZXv95QydlA66/aXYVJGOtHEuIHdWXWghrq4BMYP6Ebu/lIMWjN9SA+UtoiXMpstbgZtZsPeUvr27ERJo2JPtRmdmsqpx3QmobGe+IZ6Dh44wsGjtXRJTaS0rski0igmHduTuHgje0pqqW7U5DfFk9SrB0WVDWT2SKHAzWPyWSP78PnWQyTGGahvMjNjWC+MBsXSvCJ+cWwPvt1dwtgB3SipqqeitpEKh3nWZ4zoTWVhOZ2S4vjZYTpnfNcuFNVago+NzejKpsJyAHp2SqTRZCbOoKhtNNl7ssP6dGb7oUq6JMURZzRQVddE99T4FvFjHP3aWhnoOyyTXOu9s4/pzirrtoLnjO7Dp5sPcc7oPuwrqWHrwQrOHNmbL7ZaxiVs7oNB6amcMiSdZXlFLbazyx7dB4Dkkhp+Kq2hZ+dEhqSlMLh3Z7qnJPDo59tJTTBS7dAb798tmbNG9mHa0HRW7Si271P79a5iMrqncKi8juevOp601AQWri/kjBF9+OcX27lofH9mDO/NrOwBKIV9H9xFt03lqlfWcv+FI+meksDNJw/i1unNy+oX3TaVzQfKPf5Pv3vTZG7673rmXjDCSchevS67xSrUQFlyx0l8vavYHjBszrsbeeKyltMIX7o6m1+98YPdnQEW95Wt4Zr/60kcrqzjxOPS2V1cxR0zBjNuQDfu/XgrRRV19u/7ikkDue4XmXy88QDXTMmke0oCeSdX8BsfC99eviYbk5t1CV2T47nrrGFkZ3bnroW5nD26j70Ruf+CkYwb4Hnw1ZHHLx3Lm98VAHDd1MwW508b3outB8q55IQMe9qzl48nNdHIhn1lXD3lGJ9lzDw+g3mrdvOHM4bY05RSzD1/BNmZacQZDdxz9jBmDO/ll81tJWSDtkqpKcBcrfWZ1uN7ALTWD3u6prWDtoIgCLFKIIO2odwA5QdgsFIqSymVAMwGPglheYIgCIIXQubS0Vo3KaVuB77AMi3zVa11YFu+C4IgCEEjpD58rfWnuB3eFARBEMKN7GkrCIIQI4jgC4IgxAgi+IIgCDGCCL4gCEKMIIIvCIIQI4Q0WmagKKWKgX0+M7qnJ3AkiOaECrEzuIidwUXsDC7hsPMYrXW6PxmjSvDbglIqx9/VZpFE7AwuYmdwETuDS7TZKS4dQRCEGEEEXxAEIUboSIL/UqQN8BOxM7iIncFF7AwuUWVnh/HhC4IgCN7pSD18QRAEwQvtXvCVUmcppXYopfKVUndH2JYBSqmVSqk8pdRWpdTvrOlpSqllSqld1r/drelKKfWM1fZcpdTxYbbXqJT6USm12HqcpZRaa7XnXWtYa5RSidbjfOv5zDDa2E0ptVAptV0ptU0pNSUa61MpNcf6nW9RSi1QSiVFQ30qpV5VSh1WSm1xSAu4/pRS11rz71JKXRsmO/9p/d5zlVIfKaW6OZy7x2rnDqXUmQ7pIdUDd3Y6nPuDUkorpXpajyNWnx7RWrfbF5awy7uBQVh2GNwEjIigPX2B463vOwM7gRHAY8Dd1vS7gUet788BPsOy09lkYG2Y7f098Daw2Hr8HjDb+v4F4DfW97cCL1jfzwbeDaONbwC/tr5PALpFW31i2c5zL5DsUI/XRUN9AicDxwNbHNICqj8gDdhj/dvd+r57GOw8A4izvn/Uwc4R1t96IpBl1QBjOPTAnZ3W9AFYQsHvA3pGuj492h+OQkL4zzwF+MLh+B7gnkjb5WDPx8DpwA6grzWtL7DD+v5F4HKH/PZ8YbAtA8t236cCi63/lEccfmD2urX+I0+xvo+z5lNhsLGrVUiVS3pU1SfN+zenWetnMXBmtNQnkOkipAHVH3A58KJDulO+UNnpcu5iYL71vdPv3Faf4dIDd3YCC4GxQAHNgh/R+nT3au8uHXcbpfePkC1OWB/TxwNrgd5a65+tpw4Bva3vI2n/U8CfAdumoT2AMq21beNbR1vsdlrPl1vzh5osoBh4zep6ekUplUqU1afW+gDwOLAf+BlL/awn+urTRqD1Fw2/sxuw9JbxYk9E7FRKXQgc0FpvcjkVVXZCB/DhRyNKqU7AB8CdWusKx3Pa0qRHdGqUUuo84LDWen0k7fCDOCyPz89rrccD1VhcEHaipD67AxdiaaD6AanAWZG0yV+iof58oZT6K9AEzI+0La4opVKAvwD3RtoWf2jvgn8Ai+/MRoY1LWIopeKxiP18rfWH1uQipVRf6/m+wGFreqTsnwpcoJQqAN7B4tZ5GuimlLLtguZoi91O6/muQEkY7CwECrXWa63HC7E0ANFWn6cBe7XWxVrrRuBDLHUcbfVpI9D6i9jvTCl1HXAecKW1ccKLPZGw81gsDf0m6+8pA9iglOoTZXYC7V/wo2qjdKWUAv4DbNNaP+lw6hPANhJ/LRbfvi39Guto/mSg3OFRO2Rore/RWmdorTOx1NkKrfWVwErgEg922uy/xJo/5L1CrfUh4Cel1FBr0gwgjyirTyyunMlKqRTr/4DNzqiqTwcCrb8vgDOUUt2tTzNnWNNCilLqLCxuxwu01jUu9s+2znbKAgYD64iAHmitN2ute2mtM62/p0IsEzcOEWX1aTO4Xb+wjITvxDI6/9cI23IilsfjXGCj9XUOFv/sl8AuYDmQZs2vgOestm8GsiNg8zSaZ+kMwvLDyQfeBxKt6UnW43zr+UFhtG8ckGOt00VYZjVEXX0C9wPbgS3Af7HMIIl4fQILsIwrNGIRo1+1pv6w+NDzra/rw2RnPhZft+239IJD/r9a7dwBnO2QHlI9cGeny/kCmgdtI1afnl6y0lYQBCFGaO8uHUEQBMFPRPAFQRBiBBF8QRCEGEEEXxAEIUYQwRcEQYgRRPCFDoFSyqSU2ujw8hopUSl1i1LqmiCUW2CLjhjgdWcqpe63Rq78zPcVgtB24nxnEYR2Qa3Wepy/mbXWL4TSGD84CcvCrJOANRG2RYgRpIcvdGisPfDHlFKblVLrlFLHWdPnKqX+aH1/h7LsYZCrlHrHmpamlFpkTfteKTXGmt5DKbVUWWLfv4JlcY2trKusZWxUSr2olDK6sWeWUmojcAeWAHYvA9crpSK2QlyIHUTwhY5CsotLZ5bDuXKt9Wjg31hE1pW7gfFa6zHALda0+4EfrWl/Ad60pt8HrNFajwQ+AgYCKKWGA7OAqdYnDRNwpWtBWut3sURR3WK1abO17Ava8uEFwR/EpSN0FLy5dBY4/P2Xm/O5wHyl1CIs4RvAEibjlwBa6xXWnn0XLBtgzLSmL1FKHbXmnwGcAPxgCadDMs1ByVwZgmXTC7EZIJoAAAE/SURBVIBUrXWlH59PENqMCL4QC2gP722ci0XIzwf+qpQa3YoyFPCG1voer5mUygF6AnFKqTygr9XF81ut9detKFcQ/EZcOkIsMMvh73eOJ5RSBmCA1nolcBeWUMWdgK+xumSUUtOAI9qyt8Fq4Apr+tlYgrmBJRjZJUqpXtZzaUqpY1wN0VpnA0uwxM9/DEuAr3Ei9kI4kB6+0FFItvaUbXyutbZNzeyulMoF6rFsL+eIEXhLKdUVSy/9Ga11mVJqLvCq9boamsMJ3w8sUEptBb7FEhoZrXWeUupvwFJrI9II3IZlj1NXjscyaHsr8KSb84IQEiRaptChsW5Kka21PhJpWwQh0ohLRxAEIUaQHr4gCEKMID18QRCEGEEEXxAEIUYQwRcEQYgRRPAFQRBiBBF8QRCEGEEEXxAEIUb4fx//yyr+xvvYAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the scores\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "plt.plot(np.arange(len(scores)), scores, label='MADDPG')\n",
    "plt.plot(np.arange(len(scores)), avgs, c='r', label='moving avg')\n",
    "plt.ylabel('Score')\n",
    "plt.xlabel('Episode #')\n",
    "plt.legend(loc='upper left');\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### 5. Test a saved agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "CONSEC_EPISODES = 10\n",
    "PRINT_EVERY = 1\n",
    "ADD_NOISE = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "## reinitialize the agents (if needed)\n",
    "agent_0 = Agent(state_size, action_size, num_agents=1, random_seed=0)\n",
    "agent_1 = Agent(state_size, action_size, num_agents=1, random_seed=0)\n",
    "\n",
    "# load the weights from file\n",
    "agent_0_weights = 'models/best/checkpoint_actor_0.pth'\n",
    "agent_1_weights = 'models/best/checkpoint_actor_1.pth'\n",
    "agent_0.actor_local.load_state_dict(torch.load(agent_0_weights))\n",
    "agent_1.actor_local.load_state_dict(torch.load(agent_1_weights))     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def test(n_episodes=10, max_t=1000, train_mode=False):\n",
    "\n",
    "    scores_window = deque(maxlen=CONSEC_EPISODES)\n",
    "    scores_all = []\n",
    "    moving_average = []  \n",
    "\n",
    "    for i_episode in range(1, n_episodes+1):\n",
    "        env_info = env.reset(train_mode=train_mode)[brain_name]         # reset the environment\n",
    "        states = np.reshape(env_info.vector_observations, (1,48)) # get states and combine them\n",
    "        scores = np.zeros(num_agents)\n",
    "        while True:\n",
    "            actions = get_actions(states, ADD_NOISE)           # choose agent actions and combine them\n",
    "            env_info = env.step(actions)[brain_name]           # send both agents' actions together to the environment\n",
    "            next_states = np.reshape(env_info.vector_observations, (1, 48)) # combine the agent next states\n",
    "            rewards = env_info.rewards                         # get reward\n",
    "            done = env_info.local_done                         # see if episode finished\n",
    "            scores += np.max(rewards)                          # update the score for each agent\n",
    "            states = next_states                               # roll over states to next time step\n",
    "            if np.any(done):                                   # exit loop if episode finished\n",
    "                break\n",
    "\n",
    "        ep_best_score = np.max(scores)\n",
    "        scores_window.append(ep_best_score)\n",
    "        scores_all.append(ep_best_score)\n",
    "        moving_average.append(np.mean(scores_window))\n",
    "\n",
    "        # print results\n",
    "        if i_episode % PRINT_EVERY == 0:\n",
    "            print('Episodes {:0>4d}-{:0>4d}\\tMax Reward: {:.3f}\\tMoving Average: {:.3f}'.format(\n",
    "                i_episode-PRINT_EVERY, i_episode, np.max(scores_all[-PRINT_EVERY:]), moving_average[-1]))\n",
    "            \n",
    "    return scores_all, moving_average            "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episodes 0000-0001\tMax Reward: 5.200\tMoving Average: 5.200\n",
      "Episodes 0001-0002\tMax Reward: 0.000\tMoving Average: 2.600\n",
      "Episodes 0002-0003\tMax Reward: 5.200\tMoving Average: 3.467\n",
      "Episodes 0003-0004\tMax Reward: 0.000\tMoving Average: 2.600\n",
      "Episodes 0004-0005\tMax Reward: 0.000\tMoving Average: 2.080\n",
      "Episodes 0005-0006\tMax Reward: 0.400\tMoving Average: 1.800\n",
      "Episodes 0006-0007\tMax Reward: 5.200\tMoving Average: 2.286\n",
      "Episodes 0007-0008\tMax Reward: 0.000\tMoving Average: 2.000\n",
      "Episodes 0008-0009\tMax Reward: 0.100\tMoving Average: 1.789\n",
      "Episodes 0009-0010\tMax Reward: 5.200\tMoving Average: 2.130\n"
     ]
    }
   ],
   "source": [
    "scores, avgs = test()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzsnXd8m9W9/99Hw5ZtyU685CziDEtJyIKEVUYp6eBSoIFcsCl0QFva20FLW25vF4W2dPDr7bj3lrZAKW2hAcoqZZURKKEDGlYIXhk405YdZ0gesjXO74+jx1EcD9mW9Gic9+vll22N5zmWpe/zPd/1EVJKNBqNRpP7WMxegEaj0WjSgzb4Go1Gkydog6/RaDR5gjb4Go1Gkydog6/RaDR5gjb4Go1Gkydog6/RaDR5gjb4Go1Gkydog6/RaDR5gs3sBcRTWVkpa2trzV6GRqPRZA2vvPLKfillVSKPzSiDX1tby6ZNm8xehkaj0WQNQoidiT5Wh3Q0Go0mT9AGX6PRaPIEbfA1Go0mT8ioGP5IhEIh9uzZQzAYNHspeYPD4WD27NnY7Xazl6LRaJJIxhv8PXv24HK5qK2tRQhh9nJyHikl3d3d7Nmzh3nz5pm9HI1Gk0QyPqQTDAapqKjQxj5NCCGoqKjQOyqNJgfJeIMPaGOfZvTrrdHkJllh8MciGonQs3MPffsPmr2UjCAalRzoHUBLV2pG4pE39rG/Z8DsZWjieKbRx6/+uj0tn9msN/jCYqGwuwu6ulJ3DiG44oorhn4Ph8NUVVVx/vnnH/W4tWvXcuqppx512w033MCsWbNYuXIldXV1XHzxxTQ2Ng7df/bZZ+P1elm+fDmLFi3is5/9LIcOHRq632q1snLlSpYuXcoll1xCX18fAD6fjw9+8IPMnz+fVatWcdppp/HQQw9xqD/EnoP99A1GUvFSaLKY9sP9XLP+NX73j4T7dDRp4E9v7OP3/9yZlp119ht8IegrKcXRF4BIaoxcSUkJW7Zsob+/H4Cnn36aWbNmHfWYQ4cO8corr3D48GF27Nhx1H3XXnstr7/+Olu3bqW+vp5zzjmHrrgL1N13383mzZvZvHkzhYWFfOADHxi6r6ioiNdff50tW7ZQUFDAL3/5S6SUrF27lrPOOosdO3bwyiuvcM8997Bnzx4GQuo1CIa0wdccTeM+PwCtHQGTV6KJp6XDz6IaV1rOlfUGH2CwtAyLlMg4zzjZnHfeeTz22GMArF+/nssuu+yo+x988EEuuOACGhoauOeee0Y9Tn19Pe9973v5wx/+cMx9BQUF3HzzzezatYs33njjmPvPPPNMtm3bxoYNGygoKOBTn/rU0H1z587lc5/7HMFwFGDou0Zj0NQeM/g+bfAzhcFwlB1dvXjc6TH4GV+WGc+Nf35ryEuJJxyJYu3vQ7zoh6KiCR1zycxSvnXB8eM+rqGhgW9/+9ucf/75bN68mauuuoqNGzcO3b9+/Xquv/563G4369at42tf+9qoxzrxxBNpbm4e8T6r1cqKFStobm5mxYoVQ7eHw2GeeOIJzj33XN566y1OPPHEEZ8f1B6+ZhQaYwa/rbuXYCiCw241eUWaHft7CEcl3lzw8IUQbUKIN4UQrwshUjYVTVgEYYsVImFIUeJj+fLltLW1sX79es4777yj7vP5fGzdupUzzjgDj8eD3W5ny5Ytox5rvORM/P39/f2sXLmS1atXc9xxx/Gxj33smMd/5jOfYcWKFaw+6SRCkSgCwUBIe/iao2lqD+CwW4hK2NbZY/ZyNEBLLLy2qKY0LedLh4f/Linl/mQcaDRPPByJ8vauLuq6d0NtLVRWJuN0x3DhhRfy5S9/meeff57u7u6h2++77z4OHjw41Kjk9/tZv349N91004jHee2111i9evWI90UiEd58800WL14MHInhx3P88cfzwAMPDP3+85//nP3797NqlTpmSaGVnoEwoUgUuzUnonaaKdI7EKatu5fzl8/kz2/sY2tngKWzysxeVt7T0hHAZhHMqyxJy/lywhrYrBZChUWE7AVw4EDKznPVVVfxrW99i2XLlh11+/r163nyySdpa2ujra1tKIk6Eg888ABPPfXUMTkAUGMkvvrVrzJnzhyWL18+6jrOOeccgsEgv/jFL4Zu6+vrQ6J2BtOKCwCGErgaTXNHACnhvKU12K2Clg7t4WcCLR0BFlQ5KbClxxSn+iwSeEoI8YoQ4upUnshhtxAocoHfD6FQSs4xe/ZsrrnmmqNua2trY+fOnUeVY86bN4+ysjJeeuklAH7yk58MlWXeddddbNiwgaqqI3oFl19+OcuXL2fp0qX09vbypz/9acx1CCF4+OGH+etf/8q8efM4+eST+chHPsJXv/VdrELgcqiNm07cagyM+P3yOdNYUOXUidsMobkjgCdN8XtIfUjnDCnlXiFENfC0EKJZSvlC/ANiF4KrAY477rhJn8hht9JdUEI53XDwIFRXT2nh8fT0HOsNnX322Zx99tkA7N2795j7X331VQBOOeUUbrjhhlGP/fzzz0/43AAzZsw4ZhexvasHKcFmEdgsQiduNUM0tfspddiYWebA43bxyk7dqGg2gWCIvYf6+eApk7d7EyWlHr6Ucm/seyfwEHDyCI+5VUq5Wkq5Ot7rnSgOu4V+WwFRhyOlYZ1MZiAUxWG3IISg0G4lqBO3mhiN+/wsmVmKEAJvjYu9h/rpGQibvay8ptWnnDlvmkoyIYUGXwhRIoRwGT8D7wVGL12ZIg6bKjEbLJ0GPT0wOJiqU2UkoUiUcDQ69Do47FYGQhE9YkFDJCpp6QiweIaqBKmrdgKwVYd1TMWo0ElXSSak1sN3Ay8KId4AXgYek1I+maqTFcZqinuLYy/ewfzashoJWodd/UsdNgsRKQlFtMHPd3Z299IfigwZfMPA6Di+ubT6ApQUWJk1bWK9Q1MhZTF8KeUOYMW4D0wSVougwGqhV9ioKC5WYR23O12nNx0jQWtc+IymmmA4krYKAE1mYiRsl8QM/pzpxTjsFl2pYzLNHX48NS4slvRNp80pS+CwWwmGIzB9OvT2wkD+TAUMhiJDyVqAwpiR16WZmqZ2PzaLoM6tQjkWi8DjdmkP30SkVGG2dMbvIccMfqHdwkA4SnT6dHVDHiVvg6EohXbr0MQ9m9WC3WrRiVsNjfv8LKx2Umg7MkrB43bRog2+aXT1DHCwL5TW+D3kmMF32K1IKRm02qGkJKPj+L/85S/53e9+l5RjSSkZGGE2SqHNokszNTS1H0nYGnjcTroCAxzsza/ihkzBjIQtZNnwtPFwxMIYwVAER3k57N4NwSA4HCav7FjiJ11OlVBEEpFy6O83cNitHOgdREqpVazylAO9g3T4g0PxewNjOmOrL8Ap8yvMWFpeM2TwdUhn8hTarAhiCcwkhXXa2tpYtGgRH/3oR/F4PFx++eU888wznH766dTV1fHyyy/HTnOAtWvXsnz5ck499VQ2b95MNBqltrb2KEGTuro6fD4fN9xwAz/60Y8A1cT1la98hZNPPhmPxzM0hbOvr49LL72UJUuWcNFFF3HKKaewadOxM+hu/PaNfPD953DmKau4+uqrkVLS3NzM+e8+i6iUDIajtLW1DY2EePzxx1m0aBGrVq3immuuOUbIRZM7GCORh3v4ulLHXFo6AlQ6C6lwFqb1vNnl4X/hCzBskFg8FmDBYASLAOxW6OtT0zNLxhhMtHIl/PSnY55227Zt/PGPf+SOO+7gpJNO4g9/+AMvvvgijzzyCN/73vd4+OGH+da3vsUJJ5zAww8/zIYNG/jwhz/M66+/zgc+8AEeeughrrzySl566SXmzp2Le4TqoXA4zMsvv8zjjz/OjTfeyDPPPMMtt9zC9OnTaWxsZMuWLaxcuXLE9X3kY5+k/uprWTKjlCs/+hEeffRRLrjgAsKhQfbs2snciiXce++91NfXEwwG+eQnP8kLL7zAvHnzRpzpo8kdjHHii2cc7UnWlDpwOWw6jm8SLb5A2kRP4skpDx/AIiBqlJ7b7RCNqq8pMG/ePJYtW4bFYuH4449nzZo1CCFYtmwZbW1tALz44ot86EMfAtRws+7ubvx+P/X19dx7770A3HPPPdTX1494josvvhiAVatWHXXMhoYGAJYuXTrqQLXnnnueKy58NyesXMGGDRt46623ALj00kv5y58fZCAUGTL4zc3NzJ8/f2iypzb4uU1Tux936bGepBACr9tFqy7NTDuRqKTVF0ib6Ek82eXhj+OJAxw6HKQrEOT4mWVYImF44w2YMQOGSRJOhMLCIx8Wi8Uy9LvFYiEcHrs9/bTTTmPbtm10dXXx8MMP841vfGPMc1it1nGPGU8wGOTr132Bh59+gTNWLuKGG24gGAwCcFlDAxesvZiL1l6MEIK6urpjRi1rcpvGdv8x4RyDOreLJ7a06xxPmtl9oI9gKKo9/GTgsFuQwEA4ojz80lIVx0/xiIEzzzyTu+++G1AD0SorKyktVbNLLrroIr74xS+yePFiKioST5Cdfvrp3HfffQA0Njby5ptvHvOY/v5+JDDDXU1PTw/333//0H0LFizAbrPxkx99f2hn4fV62bFjx9Auwth9aHKPgXCEbZ09xyRsDbxuJ4f6QnQF8qdfJRNoNqlCB7LNw0+AIx2mUYoKUMnbnTtVPH+sWP4UueGGG7jqqqtYvnw5xcXF/Pa3vx26r76+npNOOok777xzQsf89Kc/zUc+8hGWLFnCokWLOP744ykrO1q0othZyrrLPsy7T1/NzBkzOOmkk466/wPr/p3vfPNr/OTm7wNKUOWWW27h3HPPpaSk5JjHa3KHbZ1KPm80D98Yy9viC1BdmnmVbLlKS0cAIRhqhEsrUsqM+Vq1apUcTmNj4zG3jUUkGpWb9xyS+w71qRtCISk3bZJy164JHScTCIfDsr+/X0op5bZt22Rtba0cGBg46jGH+gblG7sPyt6B0IjHONA7IN/YfVD2D4aHbgsEAlJKKaPRqPyP//gP+eMf//iY5030dddkHvf9a5ec+5VH5bbOwIj37w8E5dyvPCpve2F7mleW33z6rlfkWTdvSNrxgE0yQRubcx6+RQgKbZYjmq42mwrrHDwIs2dDFsUq+/r6eNe73kUoFEJKyS233EJBQcFRjzFGJ8R3UcZjTM+MF62+7bbb+O1vf8vg4CAnnHACn/zkJ1P4V2jMwtCwra0YeWdb4Syk0lnAVp9O3KaT5g5/2uvvDXLO4IMycn2DcYnP8nJ4+201X8dpwjZqkrhcrhHr7uMJhqIU2CxYRxnAVGizIBBHjVi49tprufbaa5O6Vk3m0dh+mEU1paO+NwDqqvWIhXQSDEVo6+7jvGUzTDl/ViRt5QQTrg67hcFIlIhRnzltmvLsc3C2TjAcGfLiR8JiERTYLCqJnSATfb01mYeUcsSRCsPx1rjY6gsQjer/eTrY3tVDJCpNSdhCFhh8h8NBd3f3hIyQMSJ4aFKk1QplZSqsk0PGLColA6Eohfax/40Oe+JD1KSUdHd348jAcRSaxNl3OMjh/hBLZo5t8D1uF72DEfYe6k/TyvIbY6SCGSWZkAUhndmzZ7Nnzx66uroSfk44EsXnH2Bwv52Swtif2NsL+/erJqwcMWYh4+8ssXOoYPR/pb8/RCAYJnLAkVC9tcPhYPbs2clcqibNNO0zZuCPbVi8NSrE2eoLMKe8OOXryndafAEKrBbmjpJXSTUZb/DtdvtQV2iiRKOSS771JB88eS7XX7BY3djbqwRRLr8cfvWrFKw0/Ty6eR+ffeQ1Hr/mTBaP4ck9/mY7n77vVR793BksnVU26uM0uYMxQ8dbM7aHXzc0RK2HNYvzRzDILFo6AiyodmK3mhNcyfiQzmSwWAR11cMEHkpK4MIL4YEHIBQyb3FJpLUjgNUimF81trcQPxlRkx80tvuprSjGWTi2T1fqsDOjzKHfG2mipcOcGToGOWnwgZEVferrobsbnn3WnEUlmRZfgNqK4mPm4A+ntqKYAqtFV2PkEU1jjFQYjsftGoota1LH4f4Q7YeDpszQMchZg++tcdI5XODh3HNV8jZHxgm0+noSevPYrBbmV5XQqj/UeUHPQJi27r5RRyoMx1vjYltXD+GIVkdLJYYDqj38FDBiGKOwENauhQcfzHq9W1XP25uwt+CtcdGqG2zygpaOkWfgj4bH7WIwHGXngb5ULivvMWboeLTBTz6jCjw0NIDfD08+acKqkse2zh6kTHwAk8ftYu+hfgLB3MhfaEbHmIE/XkmmgSc202WrDvmllNaOAC6HjZll5lUJ5qzBryl14CocQeBhzRqoqMj6sI5xIUvYw489bmun9vJzncb2AGVFKhmbCAurnQgBLXo2fkpp6QjgdbtMHUWdswZfCIFnpDCG3Q7r1sEjj6gJmlmKUc9bW5FY7fRQiEvH8XOexnY/S2aUJmxYigtsHFderCt1UoiUkuYOv6nhHMhhgw9HKnWO6dJtaFB1+Y89Zs7CkkBrR4D5VSXYEqznnT29iCK7VVfq5DiRqKSlI/EKHQOPW8/USSU+/wD+YNjUhC3kuMEfVeDhrLOgpgbuucechSWBVl/PhOZxWCwCj9upvbgcp627l2AoeoyG7Xh43S7e3t87oZlLmsRpjiXSzZqSaZDTBj9e4OEorFa45BLl4fv9JqxsagSCIfYe6p9wPa/a8eg4bS4z0YStQZ3bSSQqeXt/byqWlfe0mKhyFU9uG/yYQRyxqaS+XpVmPvJImlc1dYzE60S9BW+Ni67AAAfiexM0OUVTux+bRbCwemJjwA1DpBuwUkOLL4C7tJBpxQXjPziF5LTBr3QWUlEyisDDaafBnDlZGdYxEq8T9fDr9IiFnKex3c/CaueogjijMb/Sic0i9HsjRbR0BMada5QOUm7whRBWIcRrQohHU32ukRg1GWWxKC//qaeybk5+iy9Akd3K7OlFE3qeVxv8nKcpVqEzUQpsFuZVlujSzBQQjkTZ2tmD1wwN22Gkw8P/PNCUhvOMyJgCD/X1apDaQw+lf2FToNUXwON2YhlDyWgk3KWFlDpsetueo3T3DODzD0w4fm+gypj1eyPZ7DzQx2A4mvsevhBiNvB+4PZUnmcsxhR4WLUKFizIuiasRGfoDEcIEbsAai8uF2lqV8Z6oiWZBp5qF7sP9h0tD6qZMmaLnsSTag//p8B/AqZNZTLaxkf0XIRQXv6zz0JnZ5pXNjkO9A7SFRiYdLbfCHFpGcPco7H9MDB5g++tcSKlGtuhSR7NHQEsggkn0lNBygy+EOJ8oFNK+co4j7taCLFJCLFpIqpWiRIv8DAiDQ1KBev++5N+7lRgXLjqJlnP63G7ONwfonN4b4Im62lqD1BT6qC8ZHKVIGNWtWkmTWtHgNqKknHHmKeDVHr4pwMXCiHagHuAc4QQdw1/kJTyVinlainl6qqqqqQvwpgpMmpsculSWLIka8I6xt8x2QYO/aHOXdQM/MmHDeZWlFBgs+g4fpJp8QVMr783SJnBl1J+VUo5W0pZCzQAG6SUV6TqfGMxpsCDEdbZuBH27k3vwiZBS0eAUocNd2nhpJ4/ZohLk7UMhCNs6+yZdMIWwGoR1FU7adE5nqTRPzixMeapJqfr8A3GFXiorwcp4Y9/TO/CJsHW2EiFyU7cq3AWUuks1AY/x9jq6yEclZOO3xt43C49JjmJGGPMMyFhC2ky+FLK56WU56fjXCNRV+0cW+DB64WVKzO+CUtKSYsvMGVvwVujvbhcozEmWj6ZGvx4PG4X7YeDHO7XugnJYGiGTj4ZfLMxXuwxPZeGBnjpJXj77TStauJ0BgY43B+assGvqx6jN0GTlTS1+ymyW5lbMbag/Xh4a7QYSjJp6QhQaLNM+f+SLPLC4Cck8FBfr77fd196FjUJWiY5UmE43hoXfaP1JmiyksZ9fhbNcGGdYDPecIaS+trgJ4UWX4A6t3PK/5dkkRcGPyGBh9paOOWUjA7rHFG5mlo9r67UyS2klLEKnal3cs6aVkRJgVUL5SQJpXJlfoetQV4YfEhQ4KGhAV5/HVpa0rOoCdLqC6iBcM7JVegYDFXqdOoPdS6w91A//mB4yvF7UN3YdXqMdlI42DtIZ2BgKEyWCeSRwXeOL/BwySWqTDNDa/JbfD1JefO4HHZmTSvSXlyOMNWRCsPxuvVMnWRgOJiZMEPHII8Mvmt8gYdZs+DMM1VYJ8NGD0Sjkq2+AHXVycn217l1pU6u0NTuR4jklf55alx09w6yv0d3Y0+FIdGTDKnBhzwy+AkLPDQ0QFMTbNmShlUlzt5D/fQNRpJW3uV1u9jeOUZvgiZraNznp7aihJJCW1KONzRGW+8Ap0SLL0BZkX3STZKpIG8MfsICD+vWqVn5GZa8TVaFjoHH7WIwEqWte5TeBE3W0NQxtZEKw/HEwoa6UmdqKNGTyTdJpoK8MfgJCzxUV8OaNSqOn0FhHSPBOtUKHYOEehM0GU8gGGJnd19SErYGVc5CphXbdeJ2Ckgpae0IZFQ4B/LI4IMh4p2Agauvh+3b4ZUxB32mldaOALOmFeFy2JNyvKHeBG3wsxpj55eshC2oSp2EPyuaEdl3OEhgIJwxHbYGeWfwExJ4uOgisNszKqzT4uuhLokSaQ67lbnj9SZoMp6hkQpTGJo2El63i9YOrZswWVpiIxUyZYaOQV4Z/IQFHsrL4X3vU123UfOTmuFIlO2dPUnfHo45RVSTFTS1+5lWbKem1JHU43pqXAQGwrQfDib1uPlCc8fUdCtSRV4Z/Al1mNbXw+7d8I9/pHhV49PW3cdgJJr0EaveGhdt3X1j9yZoMprGfUq0PNmJQa8esTAlWjsCzCxzUFaUnBBsssgrgz8hgYcLLwSHIyOasLYONXAk38OPRCU7usboTdBkLJGomp6azPi9gVEcoJP6k6O5I3NET+LJK4NvtQgWViXYcFRaCuedp8I6EXM94BZfAJGoJmZbG3zjG3Dw4LgPNd6QOo6fnby9v5dgKJrUCh2DacUFVLsKx69q0xxDKBJle1cPHm3wzcdbMwGBh4YG8Pngr39N7aLGodUXYG558fiamNu2wVlnwU03qdLS/fvHfHhtRQk2i9Bx/CzFSNimwsMH9VnRzsDEadvfSygiMy5hC3lo8Cck8PD+90NJielhnZaOBERPmpuVse/vh5/9THULn3MOdHaO+pQCm4X5VSX6Q52lNLX7sVtFYju/SeBxu9jaGSCidRMmRPPQSIXMmaFjkHcGf0ICD8XFKpZ///0QMkcBaCAcoa27b+x44JYt8M53qoqi55+Ha66BRx9VHv/ZZ0N7+6hP9ejJiFlL4z4/C6tdFNhS8zH2ul0EQ1F2j6YUpxmRlo4AVotgQXVmiJ7Ek3cGf8ICDw0NcOAAPPtsClc1Oju6eolE5ege/uuvK6Nus6nQ0/HHq9vXrIEnn1SVRu98J+zZM+LTvW4Xuw4k0JugyTjUDPzUhQ3qtOD9pGjxBZhXWUKhbZwQrAnkncGfsMDD+94HZWWmNWG1jlWhs2mTCtsUFytj7/Ueff9ZZ8Ff/qLyEO98J+zcecwhPEMjFrSXn03s7xmgMzCQkoStgVFDrg3+xGjJ0AodyEODP2GBh8JC1Xn70EMwkP5xsS0dAWwWQe1wTcx//lN58WVl8MILsHDhyAd4xzvgmWfULuWss9TIiDi0pF120pQk0fKxcBbamD29SI/RngC9A2F2HejLuBk6Bnln8GESAg8NDeD3qxBJmmn1BZhfVXJ0nHbjRnjPe9SgtxdeUPKMY3HSSbBhA/T2Kk8/TtHruPJiCm0WPQo3y2jcl9oKHQNjxIImMbbGuvi1h59BTFjg4ZxzoKLClLBOq6/n6Pj9hg1w7rkwe7YK48yZk9iBTjgBnntOJZ/PPhsaGwHVm1DndtI63rgJTUbR1O5nRpmD6SUFKT2Pp8bFjv09hLRuQkJk6gwdg7w0+BMWeLDb4d//HR55RHnJaaJvcNj28KmnVKno/PmqGmfmzIkdcNky9TxQRn/zZiBWqaO9uKyiqT2Q0nCOgcftJBSRtI2lFKcZorkjQJHdypzpxWYvZUTy0uAbbeMTilvX10NfHzz2WIpWdSxGItVT41LnveACWLRIeepu9+QOunix2hkUFMC73gWvvorX7aLDH+Rwnzmlp5qJEQxF2NbVk/JwDugcz0Rp9QXwuJ1YLJkjehJPXhr8KtckBB7OOgtqatLahGV8yFZuek4ljpcvV+WhlZVTO7DHo2L/LhesWcMq31bgiMiKJrPZ1tlDJCrTYvAXVDmxCC13mCiZXKEDeWrwJyXwYLXCpZcqT9vvT93i4mjtCLC29UWqP/YhWLVKVduUlyfn4PPnK0+/vJwTP34pq/Y06hELWYKRsE32DPyRcNit1FaWaA8/Afb3DLC/ZzDpU22TSV4afJikwEN9vSrN/NOfUrewOCr/9Ef+++GbEe94h4rfl5Ul9wRz58ILLyBmzOD3913P4LMbknt8TUpobPdTXKAEbNKBV3djJ4SxC1pUk3kjFQzy1uBPSuDh1FPhuOPSE9b5zW+4+tc3smPJKnjiCRV+SQWzZiGef579FTVcfuN/qF2EJqNpbPezqMaVtjhxndvFzu5egiGtmzAWQzN08jGkI4RwCCFeFkK8IYR4SwhxY6rONRk81ZNI3FosKqzzl7+oRqZU8atfwVVX8eLclbzw33eoAW6pZMYMfve9O9hZPhN5/vnqAqPJSKSUsZEK6fMivW4X0USU4vKclo4A5SUFVDpTWyo7FVLp4Q8A50gpVwArgXOFEKem8HwTwoizTVjgoaEBwmHVeZsK/vd/4VOf4tA57+UT677J/LnVqTnPMGbW1XJp/U2EFy+BtWtVCaom49hzsJ9AMJyW+L2BMXBQj1gYmxZfAK/blXT1sWSSMoMvFYZLYI99Zcyc1eklkxR4OPFENcYgFU1YP/qRmnR50UU8fsP/MWArSJuIgrfGxaGiUl759R9h5UpYt05NCdVkFE0pnoE/EnMrSiiwWnTidgyiUUmrL7MrdCDFMXwhhFUI8TrQCTwtpXwpleebKJMSeBBCJW83bFBDyZLF974H112njn3vvbQcGMBZaGNmWXLFqUeXFWXxAAAgAElEQVTD2PE09Vvg6afh5JPVbmb9+rScX5MYTe1K/SydnZx2a0w3QVdxjcqeg/30DUby2+BLKSNSypXAbOBkIcTS4Y8RQlwthNgkhNjU1dWVyuUcw6QFHhoa1Oz5Bx6Y+iKkhBtugK9/Ha64Au66C+x2WmINHOnaHlY6CygvKVAXwNJSlac44wy1pt/+Ni1r0IxPY/th5lWUUFxgS+t5tW7C2LSMNdU2g0hLlY6U8hDwHHDuCPfdKqVcLaVcXVVVlY7lDOFxOycn8LB0KSxZMvWwjpTwta/BjTfClVfCnXeCzYaUMjGVqyQihKCu2nmkFt/phMcfV3OErrwSbr89bWvRjE5Te2pEy8fDW+Ni76F+AkHdjT0SxgydTK7Bh9RW6VQJIabFfi4C3gM0p+p8k8EzlXnfDQ3w4oujCouMi5TwpS/BD34An/qUMqhWJZiwv2eQg32htL95VIir50hvQnEx/PnPaljbJz4BP/95WtejOZpAMMSuA31pTdgaDBU56EqdEWnuCDB7ehHOwvTuvCZKwgZfCHGGEOLK2M9VQoh54zxlBvCcEGIz8C9UDP/RyS81+UxJ4KG+XhntP/5x4s+NRuFzn4Of/EQlaW+5RZV8xhhT9CSFeNwuegbC7IvvTXA4VEXShRfCZz+r1qwxBaPOO5UqV6Mx4YGDeUarL5CxEzLjScjgCyG+BXwF+GrsJjtw11jPkVJullKeIKVcLqVcKqX89tSWmnymJPDg8aiRwxMN60SjyqP/+c/hy1+Gn/5UJYLjMAy+GR5+/PmHKCxUF7Z16+CLX4Qf/jCt69IohkYqzEhyx3UCzJ5eRJHdqit1RmAwHGVHV2/Gh3MgcQ//IuBCoBdASrkPyPy/LgGmJPDQ0AAvvwxvv53Y4yMRuOoquO02laS9+eZjjD0og2tGA4enegwvrqBAXdwuuwz+67/gO99J69o0qiRzerEdd2lh2s9tiekmaCnMY9ne1UM4KjM+YQuJG/xBqQK7EkAIkXly7JOkzj0FgYdLL1XfExm1EA7Dhz+sKl6+/W347ndHNPagOvbqqtNXoWNQFjMmo3pxNhv8/vfq77j+evjGN1RYS5MWGtv9LJlZalpjj8ft0h7+CBg74kyeoWOQqMG/TwjxK2CaEOITwDPAbalbVvrw1kxB4KG2Vs3XGc/gh0LKM/7DH1SS9pvfHPWhUkpafT2meQvjThG1WuE3v4GPfxxuugn+8z+10U8D4UiUlo4Ai000Kl63i67AAAd6B01bQybSHNOdnleZ+X5wQgZfSvkj4H7gAcALXC+l/N9ULixdTFngoaEBXn8dmkcpQBoYgEsuUV2rP/4xfOUrYx5u3+EgPQNh0+KBXreLrb6esXsTLBY17+fTn1bdwV/4gjb6Kaatu5eBcNSUCh0Dz2g5njyntSPAgirn0brTGcq4K4x1yz4npXxaSnmdlPLLUsqn07G4dDBlgYdLLlGhmZG8/GAQLr5YjVP+v/+Da68d93BmVegYeGpcDIQT6E2wWI78Tf/zP8r4R7Xuaap4K02i5WPhnez8qRynOcNFT+IZ1+BLKSNAVAiR/tKANOCwW6mtmILAw8yZSg3rnnuO9nL7+pQk4RNPwK23wmc+k9DhjAuPkUBNN96J7HiEgP/+b7Vr+eUvVa1+RI/QTQVN7QHsVsGCKqdpa3CXFuJy2HQcP45AMMTeQ/1ZY/AT7RLoAd4UQjxNrFIHQEp5TUpWlWamnIyqr1ce7ptvKhnCnh5l7F94QcW7P/KRhA/V4gvgLi2krNg++fVMgYWxsdGtHQHed3zN+E8QAr7/fVW6+e1vw+Cg+pttmd2Akm00tvupq3aZGjYQQsSq2nSljoExbsKbBSWZkHjS9kHgm8ALwCtxXzmBp2aKAg/r1qlk5r33KvnDc8+FjRvVXJwJGHswRJDNe/OUFNqYU140sQugEGo8xHe/q/7mK65QiWpN0kj3DPzR8NQo52hCSnE5TEsWiJ7Ek5AbJqX8rRCiAPDEbmqRUubMJzpe4GHprElErqqrYc0aVYXzzDPw6qvK+K9bN6HDRKKSrb4ePnTq3ImvIYl4J6r3a/D1rytP/7rrlKd/zz2qfl8zJboCA3QFBkxN2Bp43S7+0L+LzsAA7tL0THLNZFo6/JQUWJk1rcjspSREop22ZwNbgZ8DtwCtQoizUriutJIUgYf6emhrg9deU1M0J2jsAXYf6GMgHE3bDPzR8Lhd7OjqZTA8iSTsl78MP/vZkXEMe/cmf4F5xpEZ+OZ7kVOaP5WDtPgCeNIoNzlVEg3p/DfwXinlO6WUZwHvA3JmqMrcihLsVjG1OP6//ztcdJFSirrwwkkdYmjEqsnxQG+Ni3BU0tY9id4EUPOBfvUreP558HpVjD84Ae1gzVE0thsjFcz38D3umDSonqkzNNU2G2boGCRq8O1SyhbjFyllK2qeTk5gt1pYUOWc2mCo0lJ48EEVv58kxvmNxKlZ1MUqhKb0ob76amhqgve+V42AXrpUXQx17HfCNLX7mVnmYFqx+eGxCmchlc4C7eGjQm1mTLWdCoka/E1CiNuFEGfHvm4DNqVyYekmEwQeWnwB5pQXUWLyiNX5VSVYLWLqH+p589RF8OmnVWz/Ax+Af/u30ZvUNCPSuM+fEfF7A1XVpit1skX0JJ5EDf5/AI3ANbGvxthtOUMmCDy0xkSQzUb1JhQnb9v+7nerbuSf/hT++U9YtkxpARw+nJzj5zDBUIQd+3szokLHwON2sdUXIDpRpbgcY6hCJwM+s4mSqMG3AT+TUl4spbwY+B/AmrplpR+zBR4ybcSqt8aV3NfCbofPfx62blUKWj/5iRoxfccdukN3DIwxF5lk8L01LvoGI+w91G/2UkyluSNApbOQCmf6p5dOlkQN/rNAfN1REWqAWs5gJKPMEnho6+7NqBGrHreLtqn0JoxGVZXqPP7Xv2DhQvjYx9QAun/+M7nnyREa29UuKBMStgZDn5U8j+Nni+hJPIkafIeUcsjdi/1cnJolmcOc6cU47BbT2saN7WGdSSMVhuNxu5Cx3oSUsGqVkoi86y5VunnaaapJrb09NefLUpraA5QUWDmuPHM+bnVTHTiYA0SiUoVgc9Tg9wohTjR+EUKsBnJqP2exiFhs0pyQTqsvgNUimF+VGSNWh6aIpnLHIwRcfjm0tMBXv6oatTweJQwzMJC682YRjfv8LJpRmlF13qUOOzPLHHktd7jrQB/BUDSr4veQuMH/AvBHIcRGIcRG4B7gs6lbljmYKfDQ0hGgtqIYhz0zUiO1FcUUWC3p2bY7nfC978Fbb8G73qWGsS1bBo8/nvpzZzBSythIhcwzKmrEQv5W6mTbSAWDMQ2+EOIkIUSNlPJfwCLgXiAEPAkkqOuXPZgp8LC10zzRk5GwWS0sqHamN067cKGq1X/iCTV++f3vV1+trelbQwax52A/gYGwKRq24+F1u5S032SU4nKAlo4AQkCd29yemYkynof/K8CwfqcBX0ONVzgI3JrCdZlCnUnJqGAoQlt35lToGHjdTnN6E849FzZvVuIqGzeqpq2vfAUC+RVCaMygkQrDqXO7GAxH2TmebkKO0uLzc1x5McUF2TUVdjyDb5VSHoj9XA/cKqV8QEr5TWBhapeWfrwmKfps6+xBSjLO4Ne5TexNKChQtfqtrWr65s03q/j+736XN2Wcjfv8WEQStVLfeEOVxF5/vfp5Cl3PRuw6X+P4LR2Z0TMzUcY1+EII4xK2BtgQd192XdoSoKbUgcthS7vBN+KBmWbwhz7UZsZqa2pUrf5LL8HcuaqS5x3vUGWdOU5Tu5/ayhKKCqaY19m1S71uJ5ygpDZvuglWrlQhtOuug3/8Y8IX0YXVToTIz0odtSPvy6gQbKKMZ/DXA38VQvwJVZWzEUAIsRDIuTZJswQeWn0BCqwWaisyp/QOjux4MkLS7uST4e9/hzvvVFNJTz5Z1fD7fGavLGU0dUxxBv7Bg0pk3uNR47qvu04Z//Z2uO02dfvPfqYuoHPmwGc/Cxs2QDg87qGLCqzMLS/Oy1r8bZ2qGS7nDL6U8ibgS8CdwBnyiOqBBfhcapdmDmYIPLT6AiyodmKzZpYI8qxpRRQXWDPHi7NYlKfa2qqM1+9/r4zWj3+s5u/nEP5giN0H+ifXcDUwoF6TBQtUHqShQb1mP/whTJ+u9Bs+/nGVHO/sVL0Qp56qdlJr1qhd1VVXwWOPjVkemwnzp8zAuMhlW9MVJKZp+08p5UNSynhpw1Yp5aupXZo5eKqdHO4P0RlIXx14q68HbwZm+y0WQd1kxVBSSWmpiulv2QKnn65i/StWwF/+YvbKkkZzu3rNJ2Two1G4+25YtEi9JiefrPQZ7rwTjjtu5OdMm6Z6IR54ALq61Pdzz1Xfzz9fdUZfdhn88Y9KujMOj9vF2/t7GQjnl45xS4fakc+tyIyemYmQWS5lBmCIj6Rr3rchglyXYfF7A0+1k5ZM1TD1eFSt/qOPqjDEueeqiZzbt5u9sinTuC82UiHRKZnPPgurV6sE97Rp8NRT8OST6kKYKCUlcPHFyuPv6lKvbX29UnG79FJl/NeuVYnzgwfx1LiIRCU7uiapm5ClNHeoHbk9mTvyNEUUtMEfhjfNij6ZLoLsrXGxv8ec3oSEef/7lbf/wx+qGPSSJWoGf0+GXqgSoKk9QHlJAdWucQZzbd6sRk6/+91w4IAy1q+8Au95z9QWUFCgjnvbbSrm//zzSuPglVdUWK26mjXXXMHlrz3Ozrey/wI7EZI2Q0dKlZf6+MfV/y8NaIM/jHQLPGzN8JnaWSNpV1ioEpQtLcor/f73ldrW3XdnpehKY7ufJTNKEWKUkQq7d6sSy5UrVQXTj36kdAYuv1zlOpKJzQbvfKdK8O7cqc73pS9RvG83Nz11C+977yo480w1AbWtLbnnzjAO94VoPxycWkVdRwf8v/+nHJPTT1cjRebOTUseKmUGXwgxRwjxnBCiUQjxlhDi86k6V7JJp8BDiy9AcQaLIJvVmzBpZs5UIYe//x1mzFAhjhUr4NvfhjffzArjH45EafEFRm64OnQI/uu/VDhr/XqlIbx9u4rZO9IgKm6xqNzAD36AaG3l6uvu4JEPfAL8fvjiF5XozapValRGDgrdtEw2YRsKwZ/+pEKOs2cr56SiAn79a7WDuuMOtatKMan08MPAl6SUS4BTgc8IIZak8HxJI50CD62+AHXuzBVBrnYVUlZkzz4N09NOg5dfVh8klwtuuAGWL4e6OmUk//Y3iGRmsnHHfiUgf1T8fmBAedALFqiE9SWXqN3MzTeryhszEAL7yhX892kNqpFr61a1noIC+PrXYfFi5cV+4xsqeZwFF9vxmLDKVVOTqiibPVvlP15+Wb3/mpvVtNirrlLvzzSRMoMvpWw3KnmklAGgCZiVqvMlE487fQIPLR09eEzWsB0LIQQed5pn6iQLi0WFPf72N9i3Twmr19XB//wPnHGG2g1cfbVKTmbQdM6moZEKparyZv16VXnzxS+qxOyrr6pdzNy5Jq9U5Z52HeijbzB8dCPX7t3wv/+rSjy//3048USYP1/tRP72t6ztlm7p8ONy2JhRNsZuyu+H229X/Q1Lliilt3e8Q82J2r0bfvADFW40gbTE8IUQtcAJwEvpON9U8dakZ6bOgd5B9vcMZGz83sDjdtHSkd7ehKRTU6OM+xNPwP79yoiefbb6/v73qwqUhgYVT/X7TV1qY7ufAquFhW++rMInH/wglJWpstO//EXF7TOEIaW44SHQ2bOPNHJ1dCgDuGSJuggYF9uGBvj5z1XiOUsuAMZIhWNyK1LCCy/ARz+qQomf+IQKv/3oR7BnDzz0EFxwgcqHmEjKDb4Qwgk8AHxBSnnMJ0kIcbUQYpMQYlNXV1eql5MQ6RJ4MC4omTZSYTjeGhf+YDitvQkppbRUGZt771XG/7HH1O/PPadqzisrVYXKrbcqY5Vm/C+/yt0P3Yjtve9R5ZG//73y6t/73rSvZTwMZ2XMz0pVleqKfuwx9ff84Q9qDPbGjeqisGKFimeff77yfv/2t4zacRlIKZXBj3fQ9u5VOxiPRyW2H3xQJc7/8Q817vtLXwK327xFDyOllxshhB1l7O+WUj440mOklLcSm7y5evXqjHAh0yXw0JrhFToG8WIo7tI0JAbTSWEhnHee+vrFL9QH9eGHlUf2yU/Cpz6l8gEXXaRisAtTODNwzx64/npuuvNOgsVOVcnx2c+mJxk7SY4rL6bQZkl8/EZZmbqoXnaZ8orb2pThN74ee0w9rrBQ7W7OPFN9veMd6kJtIh3+IP5gmMUVhaox7Y47VK9DNKqM/Te/CevWqX6GDCVlBl+oPc+vgSYp5Y9TdZ5UUZeGSp2WjgBlRfbxa61NJr408yxPlcmrSSFWqwo3nHGGMrZbtijD//DDKjZ93XVqVPPateoCcMIJSrVrqhw+rHoIfvITZDTK7avXUnzDN7nivBOmfuwUY7UIFlY7J/dZEUJV9cybBx/+sLqts1N5+MYF4Ic/VBU/FotKuhsXgDPPVGG6NLLnry/xzWdvo/62F+FAN8yapZTaPvrR1DoCSSSVHv7pwIeAN4UQr8du+5qUMitkjLw1Lv6xvZtwJJqyGTetvgAet3P0WusMobykgEpnYfZV6kwFIZTq1rJlapxwW5sqq3voIWWAvvtdNa7AMP5nnDHx+OzgoNpVfOc70N0NV1zBS1d+ge891cF67yijEDIQr9vF37d3J+dg1dXq9bzoIvV7T48SuDcuALffrvIAoIzsGWccuQAsXJicC3A8hw6pvM6vf81JmzaxwmJDfuBC+MTHVYjNmhkKdYmSMoMvpXwRyGxLNgYet4vBiBJ4WFCV/CoaIx54wYqZST92KvDWZGmlTrKorYXPf1597d8Pf/6zMv6/+pWq+qmoUEm5tWuVISgao68iGlWzab72NdixQw0su/lmOPFEXnt+O9AxuaFpJuGpcfHga3s53BeirNie3IM7naoL1ehEDYVUPsO4ADzyiJoVBCpWHn8BWLFicgY5GlWdxXfcoUI3wSAsW8ZDH76OX8w+haduWpesvy7t6E7bUUi1wENnYAB/MJzx8XsDj9vF1s6etPQmZDyVlarc85FHlPG//341x+ehh5TBr6xUsdzf/16NKI7n+efhlFNUktjpVDHgp59WZYuoksxZ04qSbzhTyNBnpTMNDoHdrl6/L39Z7bi6ulRy9Je/VBeFTZvgC19QzV/Tp8P73qd2Y3/9K/SPU2a9a5fabS1cqC7Cjz6q6uQ3bYI33uD2VRdQs2BO6v/GFJJzIibJIl7g4d+WzUj68TNV9GQ0vHG9CXPKM2tuv6k4ncq4r1unvM/nn1cx/4cfVhUbVqsq/7zgAmXYH3tMzZ7/7W9VNccwD7SxfYoz8E3AE9eNfVJteXpPbrGocs8lS1SSHVStu7EDePFFlUwFdbFYvfrIDuD009VO7E9/Ut7800+rRPKaNeoicdFFQzu1cCTK1s4eTl9Ymd6/L8logz8KRQVWjkuhwEO2lGQa1MVV6miDPwp2uxpa9p73qDjzpk3K63/oIeV1lpWpJOTnPjdiyCcYirCjq4fzlqY3GTlVZpY5cBbaMkfucM4c1bvwwQ+q3w8cOJIIfvFF1bF8883qvpIS6O1V+Zjrr1eD4ebNO+aQbd19DIajWfN5HQ1t8MfAaDhKBS0dASqdhZSXpH5+RjLwxOb1t/gCvHtJ5tQVZyzGzJmTT1Z12jt2QHm5Gl08Cq2+AFE5gZHIGYIQgjq3M3OEcoZTXq52WBdcoH7v61MjDjZuVHX069Ypr36MoXPZLHoSjzb4Y+B1u9jQ3MlAOEKhLbnZ+FZfYKijNxtwOezMmlaU34nbqTB//rgPadwXN1Ihy/C6XfzlrQ6klBlfdUZxsQqznX12wk9p7ghgESrUm83opO0YpErgIRqVbO3sybrtoZqpk70z5jOdpnY/JQVW5kzPvpCZx+3iYF+I/T0ZrJswBVo6/NRWlOCwZ1cZ5nC0wR+DVImh7D3UT99gJGNFT0bDU+Nie2cP4Uh2zD3JNoyEbaZOTh2LjBK8TwGtvp6sqagbC23wx2BeZQk2i0h6HN84XqbKGo6Gp1r1JrR195m9lJwjGpU0tQeyMpwDUBeX48k1+gcjtHX3Zt2OfCS0wR+DApuFeZUlSffwW4YqdLIrHph1YihZxJ6D/fQMhLMuYWtQ5SxkerE9J98bWzsDSJn9CVvQBn9cPDWupMetW30BZk0rwuXInuYaiOtNyJTyuxyisT17E7Zg6CakrqrNTIy/SYd08oCjBB6SRKuvJ+u8ewCH3UptRQlb09FRmWc0tvuxiMwVs08Eb42Lrb6e7NZNGIGWjgCFNgtzKzJ3CmaiaIM/DqMKPEyScCTK9s6eoe7EbMPjduakF2c2Te1+5lWWUFSQvVUgHreLwECY9sNBs5eSVFp8AercTqxZmEwfjjb44+BJcjKqrbuPwUgUT3W2GnwXbd19BEOZqQebrTS1+1kys8zsZUwJT5qEg9KNUrnKzlDbcLTBH4e5FSUU2CxJaxvPFtGT0fC4U9ObkM8c7g+x52A/i2dk53vCwHCOMmbEQhI42DtIZ2Agq5okx0Ib/HGwWgR11U5aO5MT0mnpCCCyuGNPV+okn+YsT9gaTCsuwF1amFMefvNQwja7/zcG2uAngNftSprXsrUzkNUde7UVJditQhv8JGJU6Byf5QYf1A4wl94buTJDx0Ab/ATw1Ljo8Ac53Bea8rFaOgJZWaFjUGCzML8yz8VQkkxTu5+KkgKqMlzqMhG8bhfbOnuI5IhuQnOWyJAmijb4CTAUm5xiOWIwFKGtuy/rO/YyejJiFtLY7mfJzNLMHzqWAB63i2Aoyu4DudGN3dLhx1vjyon/DWiDnxCeuFnwU2FHVy+RqMx6g+91u9h9oJ/egeT1JuQroUiUVl9P1sfvDYxy41xwCKSUtPp6ciacA9rgJ8SsaUWUFFinPBgq2yt0DIwP9dYkJbLzmR1dvQyGo1mlYTsWddW5U6mz95Aad5HtDlo82uAngBACT41ryl5Lqy+A3SqozfKOvVRNEc1HmnKkQsegpNDGnPKinPDwjR299vDzEG9sTshU2sZbfQHmVzopsGX3yz6nvBiHPXm9CflMU7tfJcKrstsJiMfrdiWtM91MhoYcaoOff9QlQeDBaNHOdqwWwcJqnbhNBo3tfjxuJ3Zr7nwU69wutnf1MBjObt2Elo4AM8sclGbZkMOxyJ13WYqZahijdyDM7gP9WT0cK55cq7c2Aykljfv8LM6Rph4Dr9tFOCpp687ubuyWjkDW59uGow1+gnhirdWTNXJGgjNXtodetwuffyApvQn5SldggO7ewaydgT8ayapqM5NQJMr2rp6c6bA10AY/QaYq8DBUoZMrHr4xYkGPSp402T4DfzTmV5VgtWR3N/bb+3sJRWTOzNAx0AY/QaYq8NDaEcBhtzCnPPsEqkfCmwNenNnkqsF32K3MrSjOaoM/NEMnR6ZkGmiDPwFU3HpyAg8tvgALq3NjpjbAjDIHrkJbVn+ozaapXSmflRXlTlLQwOtOvlJcOmntCGC1CBZU5071FGiDPyE8NS56BsLsm4TAQ6svkFMNHEIINWJBe/iTpnHf4ZyL3xso3YTerNVNaO4IMK+yhEJbdg45HI2UGXwhxB1CiE4hxJZUnSPdTLZS53BfCJ9/IGfi9wbeGlWpk2uSdukgGIrw9v7enAvnGHhrXEgJ27K0G7vVl3sVOpBaD/9O4NwUHj/tTFbgwUhs5kqFjoEnCb0J+UpLR4CoJGdGKgwnmyt1egfC7DrQx6Icc9AghQZfSvkCcCBVxzeDyQo8DKne59gbSI9YmDxGwjZXDX5tRTEFVktWVnG15mCHrYGO4U+QyTQctfoCuAptzChzpGhV5lCXxV6c2TS1+3EW2pg9vcjspaQEm1WNi8jG8Ru5JnoSj+kGXwhxtRBikxBiU1dXl9nLGRdPbE7IRAQeWjrUSIVcmaltUOksoLykQHv4k6Bxn5/FM1xYcqRqayRUjif7YvjNHQGK7FbmTM+NEup4TDf4UspbpZSrpZSrq6qqzF7OuHjdLgbCiQs8qJnauZkAUr0JWv1qokSjkuaOQM4mbA08bhd7D/UTCGZXN7ahSpeLF2PTDX62MVGBh/09gxzsC+VUSWY83in0JuQruw/20TMQztn4vYGR48k23YRcddAgtWWZ64F/AF4hxB4hxMdSda50MlGBh1wbqTCcqfQm5Cu5NgN/NAwnJ5vi+Pt7BtjfM5izDpotVQeWUl6WqmObSUks0Zaoh28kNOty9A0U/6GeNS03E5DJpnGfH4vIfuWz8Zg9vYgiuzWrxmgfET3JzYuxDulMAu8EKnVafQHKSwqodBakeFXm4KnOHQ3TdNHYHmB+lROHPbe6OIdjsWRfjmeohDpHL8ba4E8CT41rSIt0PNRIhdyr0DEoK7ZTU+rIqg+12TS1+3M+fm+gBg5mTwy/pSO3HTRt8CdBogIPhup9rsbvDTw1WgwlUQ73hdh7qD/n4/cG3hoX+3sGONCbHd3Yzb4AXrcrZx00bfAngSFTOF7D0b7DQaV6n6PbQwOv2znh3oR8panDSNjm9nvCoC6LurGjUcnWHK7QAW3wJ8WCKicWMf6b2KhOyNWMv0FdrDdhV4K9CflM477YSIUcnZI5nGwav7HnYD99gxFt8DVH47Bbqa0sGdfDH1K9r87dNxBoMZSJ0NTup9JZQLUrt8ZsjIa7tJBShy0r3hvNsd2XNviaY/C6XeM2lLT6AtSUOigrzj2Bi3iMENfWLPDizKax3Z838XtQ3djeLMnxDA1Ny+EduTb4kyQRgYdWXyDn4/cAxQU2jisv1qWZ4xCKRNnq68mbCh2DqSjFpZPmjgCzpxfhLExZe5LpaIM/STzusQUeIlHJVl8PXtc5770AAA23SURBVHduiSCPxmSmiOYb27t6GIxE8yZ+b+BxuzjcH6IzMGD2UsakpSOQkxMy49EGf5IYavajxSZ3HehjIBzN2Q7b4XjczoR7E/KVfBmpMJxsEEMZCCsFslyO34M2+JNmbkWJEngYxavNVdGT0fDWqN6Et/eP3ZuQzzS1ByiwWZhfmVvC2OMxpBSXwTvAHV29hKMyp+P3oA3+pLEbAg+jvImNBGZdHoV0ILM/1GbTuM+P1+3CZs2vj12Fs5BKZ2FGe/i5PkPHIL/eeUlmLIGHFl+A48qLKS7I3QRQPPOrSrBahDb4oyClpKndnzcNV8Px1jhpzeAxyS2+AHarYF6O7760wZ8CYwk8GDN08oVCm5XaiuKM9uLMpDMwQHfvYN5V6BjUVbvY6gsQzdBu7JaOAPMrnRTYctsk5vZfl2KOhDGO9lwGw1F2dPXmfDxwONlSb20GjXmasDXw1rjoG4yw91C/2UsZkZaO3B6pYKAN/hQYrW387f0qAZQPb6B4PG4XOw/00T84em9CvmKMVFicZyWZBplcqRMIqoF2+fB51QZ/ChgCD8MNfj507I2EN9absL0rc2O1ZtHU7mf29CJKHbnddT0aRngzE5vzcl2VLh5t8KfAaAIPrb4AVotgflVuJ4CGM6T3m4FenFl09wzws2e28lxzJ8fnqXcP4HLYmTWtKCPHbzTnuOhJPPlRQpJC6twunm/pOuq2lo4AtRXFFNpyW9FoOHPLi8fsTcgntnf18OsX3+aBV/YwEI7yLm8VXzl3kdnLMpU6t5OWUarazKS1I0BJgTUvJDq1wZ8iXreL+1/ZQ3fPABXOQkB5+PnWPg9gs1pYUO3MyG17OpBS8tLbB7h94w6eaeqkwGbh4hNm8fEz57EwxyemJoLX7eLv27oJR6IZ1YvQ3KFmXlksuSl6Eo82+FPECGO0+no4zVlI/2CEnQf6WHvCLJNXZg5et5OX3z5g9jLSSigS5fE327l949u8ufcw5SUFXLOmjg+dOpcqV6HZy8sYPG4Xg5Eobd19LKzOjJJlKSUtvgD/trTG7KWkBW3wp4iR6NnaGeC0BRVs7+pByvxIAI2Ep8bFw6/vIxAM4crxBGUgGOLef+3mN39rY++hfuZXlnDTRUtZd+LsnBconwzemiNVbZli8LsCAxzqC+VNgYU2+FNkuMCD8T0fxiKPhDeuN2HV3OkmryY17DvUz2/+9jb3vLybwECYU+aVc+OFx3POouq8CAtMloXVTkRMKe68ZTPMXg6QXwlb0AZ/ygghjhoN3OoLUGC1MLe82OSVmUP8TJ1cM/hv7jnMbRt38Nib7QCct2wGnzhzHstnTzN5ZdmBw25lbnlxRiX1823IoTb4ScBT4+LRN/YNxQMXVDszKimVTmZNK6K4wJozpZnRqOS5lk5ufWEHL719AGehjSvfUcuVZ8zLi6qOZONxuzLqvdHiC1DpLBwquMh1tMFPAl63iz8Ew/j8A7R2BDh5XrnZSzINi0VQlwNiKMFQhAdf3cvtL+5gR1cvM8scfP28xdSfPCdvm6eSgbfGxbPNnQRDkYzIc+SD6Ek82uAnASOM8equg+w7HMzb+L2B1+1kQ3PX+A/MQPb3DPD7f+zkrn/upLt3kKWzSvlZw0rOWzYDe57u2pKJx+0iEpXs6Oo1vXQ5EpVs7Qxw+SlzTV1HOtEGPwkYbeOPbt4H5E88cDQ8bhf3bTq6NyHT2dbZw69f3MEDr+5lMBxlzaJqPn7mfE6dX44QOhGbLIzk6NZO83tVdh3oIxiK5tXnVRv8JKAEHgp4tqkTyL8ZOsOJnyJ6WgYbfCkl/9xxgNs27mBDcyeFNgvrTpzNx86YlzFlg7lGbUUJNotIexw/GpXs7xlg3+Eg7Yf6aT8c5NVdB4H8qdCBFBt8IcS5wM8AK3C7lPIHqTyfmXjcLv6+vZviPGnRHov4euvTFlSYvJpjMRqlbtu4gy17/VSUFPCFd6tGqWzZkWQrBbaxleImg5SS7t5B2g8F2Xe4n47D6nv7oSDth/vZdyiIzx8kPGwWf6HNwgnHTdMGPxkIIazAz4H3AHuAfwkhHpFSNqbqnGZiGPw6d360aI9FtauQsiJ7xo1Y8AdD3PPyLu78Wxv7DgeZX1XC9y5axsUnzsqIBGK+4HG7eGPPoYQeK6XkUF/oiAH3H/HQ98W+dxwOMhiJHvW8AquFmjIHM8ocnDyvnBllDmZMK2JGqYMZ0xzMLCtiWrE978J1qfTwTwa2SSl3AAgh7gE+AOSkwTe8BG8eqVyNhhACr9uVMZMR9xzs4zd/a+Pef+2mZyDMqfPL+c7apbzLqxulzMDrdvHo5nZ6B8KEo5L2IW/8iEfefrh/6Pdg6GhjbrMI3KUOZk5zsHLONGYsUwa8pkx9nzHNQUVJQd4Z80RIpcGfBeyO+30PcEoKz2cqRtw63+P3Bp4aJ/e8vJv3/Pivpq5DogRpAM5fPoNPnDmfpbPKTF1TvmNUsa367tPHGHOrReB2FVJT5mDJzFLevbiaGWVFQx76zDIHlc5CfaGeJKYnbYUQVwNXAxx33HEmr2byLJ9dxiffOZ8LVsw0eykZQcNJx3GwL4SU5muYvmeJmw+dOpeZeZ5byRROX1jJZSfPochuY+Y0Ryz0UsTMaQ6qnIV527SYDkSqPpBCiNOAG6SU74v9/lUAKeX3R3vO6tWr5aZNm1KyHo1Go8lFhBCvSClXJ/LYVF5K/wXUCSHmCSEKgAbgkRSeT6PRaDRjkLKQjpQyLIT4LPAXVFnmHVLKt1J1Po1Go9GMTUpj+FLKx4HHU3kOjUaj0SSGzo5oNBpNnqANvkaj0eQJ2uBrNBpNnqANvkaj0eQJ2uBrNBpNnpCyxqvJIIToAnZO8umVwP4kLieb0a/F0ejX42j063GEXHgt5kopqxJ5YEYZ/KkghNiUaLdZrqNfi6PRr8fR6NfjCPn2WuiQjkaj0eQJ2uBrNBpNnpBLBv9WsxeQQejX4mj063E0+vU4Ql69FjkTw9doNBrN2OSSh6/RaDSaMch6gy+EOFcI0SKE2CaE+C+z12MmQog5QojnhBCNQoi3hBCfN3tNZiOEsAohXhNCPGr2WsxGCDFNCHG/EKJZCNEU06zIW4QQ18Y+J1uEEOuFEA6z15Rqstrgxwml/xuwBLhMCLHE3FWZShj4kpRyCXAq8Jk8fz0APg80mb2IDOFnwJNSykXACvL4dRFCzAKuAVZLKZeiRrg3mLuq1JPVBp84oXQp5SBgCKXnJVLKdinlq7GfA6gP9CxzV2UeQojZwPuB281ei9kIIcqAs4BfA0gpB6WUh8xdlenYgCIhhA0oBvaZvJ6Uk+0GfySh9Lw1cPEIIWqBE4CXzF2JqfwU+E8gOt4D84B5QBfwm1iI63YhRInZizILKeVe4EfALqAdOCylfMrcVaWebDf4mhEQQjiBB4AvSCn9Zq/HDIQQ5wOdUspXzF5LhmADTgR+IaU8AegF8jbnJYSYjooGzANmAiVCiCvMXVXqyXaDvxeYE/f77NhteYsQwo4y9ndLKR80ez0mcjpwoRCiDRXqO0cIcZe5SzKVPcAeKaWx47sfdQHIV94NvC2l7JJShoAHgXeYvKaUk+0GXwulxyGEEKgYbZOU8sdmr8dMpJRflVLOllLWot4XG6SUOe/BjYaUsgPYLYTwxm5aAzSauCSz2QWcKoQojn1u1pAHSeyUatqmGi2UfgynAx8C3hRCvB677WsxbWGN5nPA3THnaAdwpcnrMQ0p5UtCiPuBV1HVba+RB123utNWo9Fo8oRsD+loNBqNJkG0wddoNJo8QRt8jUajyRO0wddoNJo8QRt8jUajyRO0wdfkBEKIiBDi9bivMbtIhRCfEkJ8OAnnbRNCVE7iee8TQtwohCgXQjwx1XVoNImQ1XX4Gk0c/VLKlYk+WEr5y1QuJgHOBJ6LfX/R5LVo8gTt4WtympgHfrMQ4k0hxMtCiIWx228QQnw59vM1MQ2BzUKIe2K3lQshHo7d9k8hxPLY7RVCiKdic9RvB0Tcua6IneN1IcSvYuO7h6+nPtYUdw1quNttwJVCiLztENekD23wNblC0bCQTn3cfYellMuA/0MZ2eH8F3CClHI58KnYbTcCr8Vu+xrwu9jt3wJelFIeDzwEHAcghFgM1AOnx3YaEeDy4SeSUt6LmmK6JbamN2PnvnAqf7xGkwg6pKPJFcYK6ayP+/6TEe7fjBo58DDwcOy2M4B1AFLKDTHPvhQ1U/7i2O2PCSEOxh6/BlgF/EuNZqEI6BxlPR7UaAOAkph2gUaTcrTB1+QDcpSfDd6PMuQXAF8XQiybxDkE8Fsp5VfHfJAQm4BKwCaEaARmxEI8n5NSbpzEeTWahNEhHU0+UB/3/R/xdwghLMAcKeVzwFeAMsAJbCQWkhFCnA3sj2kLvAB8MHb7vwHTY4d6Fvh3IUR17L5yIcTc4QuRUq4GHkPNYr8Z+LqUcqU29pp0oD18Ta5QFDchFJR2q1GaOV0IsRkYAC4b9jwrcFdMAlAA/yOlPCSEuAG4I/a8PuAjscffCKwXQrwF/B01ZhcpZaMQ4hvAU7GLSAj4DLBzhLWeiErafhrI6zHWmvSip2VqcpqYAMpqKeV+s9ei0ZiNDuloNBpNnqA9fI1G8//brQMZAAAAAGH+1oH0Q7SYcPgAE4IPMCH4ABOCDzAh+AATgg8wEZp3uxjlWKqTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the scores\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "plt.plot(np.arange(len(scores)), scores, label='MADDPG')\n",
    "plt.plot(np.arange(len(scores)), avgs, c='r', label='moving avg')\n",
    "plt.ylabel('Score')\n",
    "plt.xlabel('Episode #')\n",
    "plt.legend(loc='upper left');\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When finished, you can close the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Failed Models (for reference)\n",
    "The noise decay mechanism was setup incorrectly, so the noise decayed to zero too quickly (within ~20 episodes). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episodes 0000-0010\tMax Reward: 0.000\tMoving Average: 0.000\n",
      "Episodes 0010-0020\tMax Reward: 0.000\tMoving Average: 0.000\n",
      "Episodes 0020-0030\tMax Reward: 0.000\tMoving Average: 0.000\n",
      "Episodes 0030-0040\tMax Reward: 0.100\tMoving Average: 0.003\n",
      "Episodes 0040-0050\tMax Reward: 0.100\tMoving Average: 0.006\n",
      "Episodes 0050-0060\tMax Reward: 0.200\tMoving Average: 0.012\n",
      "Episodes 0060-0070\tMax Reward: 0.100\tMoving Average: 0.011\n",
      "Episodes 0070-0080\tMax Reward: 0.300\tMoving Average: 0.016\n",
      "Episodes 0080-0090\tMax Reward: 0.000\tMoving Average: 0.014\n",
      "Episodes 0090-0100\tMax Reward: 0.100\tMoving Average: 0.018\n",
      "Episodes 0100-0110\tMax Reward: 0.100\tMoving Average: 0.020\n",
      "Episodes 0110-0120\tMax Reward: 0.000\tMoving Average: 0.020\n",
      "Episodes 0120-0130\tMax Reward: 0.200\tMoving Average: 0.023\n",
      "Episodes 0130-0140\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 0140-0150\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 0150-0160\tMax Reward: 0.200\tMoving Average: 0.024\n",
      "Episodes 0160-0170\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 0170-0180\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 0180-0190\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 0190-0200\tMax Reward: 0.200\tMoving Average: 0.027\n",
      "Episodes 0200-0210\tMax Reward: 0.300\tMoving Average: 0.029\n",
      "Episodes 0210-0220\tMax Reward: 0.200\tMoving Average: 0.034\n",
      "Episodes 0220-0230\tMax Reward: 0.100\tMoving Average: 0.033\n",
      "Episodes 0230-0240\tMax Reward: 0.000\tMoving Average: 0.031\n",
      "Episodes 0240-0250\tMax Reward: 0.200\tMoving Average: 0.031\n",
      "Episodes 0250-0260\tMax Reward: 0.100\tMoving Average: 0.030\n",
      "Episodes 0260-0270\tMax Reward: 0.200\tMoving Average: 0.029\n",
      "Episodes 0270-0280\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 0280-0290\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 0290-0300\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 0300-0310\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 0310-0320\tMax Reward: 0.200\tMoving Average: 0.024\n",
      "Episodes 0320-0330\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 0330-0340\tMax Reward: 0.200\tMoving Average: 0.030\n",
      "Episodes 0340-0350\tMax Reward: 0.000\tMoving Average: 0.028\n",
      "Episodes 0350-0360\tMax Reward: 0.200\tMoving Average: 0.029\n",
      "Episodes 0360-0370\tMax Reward: 0.300\tMoving Average: 0.030\n",
      "Episodes 0370-0380\tMax Reward: 0.100\tMoving Average: 0.035\n",
      "Episodes 0380-0390\tMax Reward: 0.100\tMoving Average: 0.034\n",
      "Episodes 0390-0400\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 0400-0410\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 0410-0420\tMax Reward: 0.300\tMoving Average: 0.034\n",
      "Episodes 0420-0430\tMax Reward: 0.100\tMoving Average: 0.032\n",
      "Episodes 0430-0440\tMax Reward: 0.100\tMoving Average: 0.032\n",
      "Episodes 0440-0450\tMax Reward: 0.000\tMoving Average: 0.032\n",
      "Episodes 0450-0460\tMax Reward: 0.100\tMoving Average: 0.032\n",
      "Episodes 0460-0470\tMax Reward: 0.100\tMoving Average: 0.029\n",
      "Episodes 0470-0480\tMax Reward: 0.000\tMoving Average: 0.022\n",
      "Episodes 0480-0490\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 0490-0500\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 0500-0510\tMax Reward: 0.000\tMoving Average: 0.022\n",
      "Episodes 0510-0520\tMax Reward: 0.100\tMoving Average: 0.019\n",
      "Episodes 0520-0530\tMax Reward: 0.100\tMoving Average: 0.019\n",
      "Episodes 0530-0540\tMax Reward: 0.100\tMoving Average: 0.015\n",
      "Episodes 0540-0550\tMax Reward: 0.100\tMoving Average: 0.016\n",
      "Episodes 0550-0560\tMax Reward: 0.000\tMoving Average: 0.012\n",
      "Episodes 0560-0570\tMax Reward: 0.100\tMoving Average: 0.013\n",
      "Episodes 0570-0580\tMax Reward: 0.200\tMoving Average: 0.016\n",
      "Episodes 0580-0590\tMax Reward: 0.100\tMoving Average: 0.016\n",
      "Episodes 0590-0600\tMax Reward: 0.100\tMoving Average: 0.016\n",
      "Episodes 0600-0610\tMax Reward: 0.100\tMoving Average: 0.019\n",
      "Episodes 0610-0620\tMax Reward: 0.200\tMoving Average: 0.022\n",
      "Episodes 0620-0630\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 0630-0640\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 0640-0650\tMax Reward: 0.100\tMoving Average: 0.029\n",
      "Episodes 0650-0660\tMax Reward: 0.100\tMoving Average: 0.032\n",
      "Episodes 0660-0670\tMax Reward: 0.100\tMoving Average: 0.032\n",
      "Episodes 0670-0680\tMax Reward: 0.300\tMoving Average: 0.036\n",
      "Episodes 0680-0690\tMax Reward: 0.100\tMoving Average: 0.033\n",
      "Episodes 0690-0700\tMax Reward: 0.200\tMoving Average: 0.034\n",
      "Episodes 0700-0710\tMax Reward: 0.100\tMoving Average: 0.034\n",
      "Episodes 0710-0720\tMax Reward: 0.100\tMoving Average: 0.030\n",
      "Episodes 0720-0730\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 0730-0740\tMax Reward: 0.100\tMoving Average: 0.029\n",
      "Episodes 0740-0750\tMax Reward: 0.200\tMoving Average: 0.029\n",
      "Episodes 0750-0760\tMax Reward: 0.200\tMoving Average: 0.030\n",
      "Episodes 0760-0770\tMax Reward: 0.200\tMoving Average: 0.034\n",
      "Episodes 0770-0780\tMax Reward: 0.200\tMoving Average: 0.030\n",
      "Episodes 0780-0790\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 0790-0800\tMax Reward: 0.100\tMoving Average: 0.030\n",
      "Episodes 0800-0810\tMax Reward: 0.200\tMoving Average: 0.031\n",
      "Episodes 0810-0820\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 0820-0830\tMax Reward: 0.200\tMoving Average: 0.032\n",
      "Episodes 0830-0840\tMax Reward: 0.000\tMoving Average: 0.030\n",
      "Episodes 0840-0850\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 0850-0860\tMax Reward: 0.200\tMoving Average: 0.027\n",
      "Episodes 0860-0870\tMax Reward: 0.200\tMoving Average: 0.026\n",
      "Episodes 0870-0880\tMax Reward: 0.300\tMoving Average: 0.027\n",
      "Episodes 0880-0890\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 0890-0900\tMax Reward: 0.100\tMoving Average: 0.033\n",
      "Episodes 0900-0910\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 0910-0920\tMax Reward: 0.100\tMoving Average: 0.033\n",
      "Episodes 0920-0930\tMax Reward: 0.100\tMoving Average: 0.032\n",
      "Episodes 0930-0940\tMax Reward: 0.100\tMoving Average: 0.035\n",
      "Episodes 0940-0950\tMax Reward: 0.200\tMoving Average: 0.040\n",
      "Episodes 0950-0960\tMax Reward: 0.200\tMoving Average: 0.039\n",
      "Episodes 0960-0970\tMax Reward: 0.100\tMoving Average: 0.036\n",
      "Episodes 0970-0980\tMax Reward: 0.200\tMoving Average: 0.036\n",
      "Episodes 0980-0990\tMax Reward: 0.100\tMoving Average: 0.036\n",
      "Episodes 0990-1000\tMax Reward: 0.100\tMoving Average: 0.030\n",
      "Episodes 1000-1010\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 1010-1020\tMax Reward: 0.100\tMoving Average: 0.029\n",
      "Episodes 1020-1030\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1030-1040\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1040-1050\tMax Reward: 0.100\tMoving Average: 0.022\n",
      "Episodes 1050-1060\tMax Reward: 0.100\tMoving Average: 0.020\n",
      "Episodes 1060-1070\tMax Reward: 0.100\tMoving Average: 0.019\n",
      "Episodes 1070-1080\tMax Reward: 0.100\tMoving Average: 0.016\n",
      "Episodes 1080-1090\tMax Reward: 0.300\tMoving Average: 0.018\n",
      "Episodes 1090-1100\tMax Reward: 0.100\tMoving Average: 0.017\n",
      "Episodes 1100-1110\tMax Reward: 0.200\tMoving Average: 0.017\n",
      "Episodes 1110-1120\tMax Reward: 0.300\tMoving Average: 0.021\n",
      "Episodes 1120-1130\tMax Reward: 0.100\tMoving Average: 0.023\n",
      "Episodes 1130-1140\tMax Reward: 0.100\tMoving Average: 0.023\n",
      "Episodes 1140-1150\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 1150-1160\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 1160-1170\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1170-1180\tMax Reward: 0.200\tMoving Average: 0.031\n",
      "Episodes 1180-1190\tMax Reward: 0.300\tMoving Average: 0.032\n",
      "Episodes 1190-1200\tMax Reward: 0.100\tMoving Average: 0.033\n",
      "Episodes 1200-1210\tMax Reward: 0.100\tMoving Average: 0.033\n",
      "Episodes 1210-1220\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 1220-1230\tMax Reward: 0.000\tMoving Average: 0.028\n",
      "Episodes 1230-1240\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1240-1250\tMax Reward: 0.200\tMoving Average: 0.029\n",
      "Episodes 1250-1260\tMax Reward: 0.300\tMoving Average: 0.034\n",
      "Episodes 1260-1270\tMax Reward: 0.100\tMoving Average: 0.032\n",
      "Episodes 1270-1280\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1280-1290\tMax Reward: 0.200\tMoving Average: 0.031\n",
      "Episodes 1290-1300\tMax Reward: 0.100\tMoving Average: 0.030\n",
      "Episodes 1300-1310\tMax Reward: 0.100\tMoving Average: 0.030\n",
      "Episodes 1310-1320\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1320-1330\tMax Reward: 0.200\tMoving Average: 0.033\n",
      "Episodes 1330-1340\tMax Reward: 0.100\tMoving Average: 0.035\n",
      "Episodes 1340-1350\tMax Reward: 0.100\tMoving Average: 0.033\n",
      "Episodes 1350-1360\tMax Reward: 0.100\tMoving Average: 0.029\n",
      "Episodes 1360-1370\tMax Reward: 0.100\tMoving Average: 0.030\n",
      "Episodes 1370-1380\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 1380-1390\tMax Reward: 0.100\tMoving Average: 0.025\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episodes 1390-1400\tMax Reward: 0.100\tMoving Average: 0.029\n",
      "Episodes 1400-1410\tMax Reward: 0.100\tMoving Average: 0.030\n",
      "Episodes 1410-1420\tMax Reward: 0.000\tMoving Average: 0.029\n",
      "Episodes 1420-1430\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1430-1440\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1440-1450\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 1450-1460\tMax Reward: 0.200\tMoving Average: 0.026\n",
      "Episodes 1460-1470\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1470-1480\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1480-1490\tMax Reward: 0.200\tMoving Average: 0.030\n",
      "Episodes 1490-1500\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1500-1510\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1510-1520\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1520-1530\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 1530-1540\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1540-1550\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1550-1560\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1560-1570\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1570-1580\tMax Reward: 0.000\tMoving Average: 0.022\n",
      "Episodes 1580-1590\tMax Reward: 0.300\tMoving Average: 0.023\n",
      "Episodes 1590-1600\tMax Reward: 0.200\tMoving Average: 0.026\n",
      "Episodes 1600-1610\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 1610-1620\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1620-1630\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1630-1640\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1640-1650\tMax Reward: 0.100\tMoving Average: 0.030\n",
      "Episodes 1650-1660\tMax Reward: 0.200\tMoving Average: 0.030\n",
      "Episodes 1660-1670\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 1670-1680\tMax Reward: 0.200\tMoving Average: 0.033\n",
      "Episodes 1680-1690\tMax Reward: 0.100\tMoving Average: 0.030\n",
      "Episodes 1690-1700\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1700-1710\tMax Reward: 0.200\tMoving Average: 0.033\n",
      "Episodes 1710-1720\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 1720-1730\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 1730-1740\tMax Reward: 0.000\tMoving Average: 0.028\n",
      "Episodes 1740-1750\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1750-1760\tMax Reward: 0.000\tMoving Average: 0.021\n",
      "Episodes 1760-1770\tMax Reward: 0.100\tMoving Average: 0.021\n",
      "Episodes 1770-1780\tMax Reward: 0.100\tMoving Average: 0.020\n",
      "Episodes 1780-1790\tMax Reward: 0.100\tMoving Average: 0.021\n",
      "Episodes 1790-1800\tMax Reward: 0.200\tMoving Average: 0.021\n",
      "Episodes 1800-1810\tMax Reward: 0.100\tMoving Average: 0.015\n",
      "Episodes 1810-1820\tMax Reward: 0.100\tMoving Average: 0.016\n",
      "Episodes 1820-1830\tMax Reward: 0.100\tMoving Average: 0.017\n",
      "Episodes 1830-1840\tMax Reward: 0.000\tMoving Average: 0.017\n",
      "Episodes 1840-1850\tMax Reward: 0.100\tMoving Average: 0.016\n",
      "Episodes 1850-1860\tMax Reward: 0.100\tMoving Average: 0.019\n",
      "Episodes 1860-1870\tMax Reward: 0.100\tMoving Average: 0.021\n",
      "Episodes 1870-1880\tMax Reward: 0.100\tMoving Average: 0.021\n",
      "Episodes 1880-1890\tMax Reward: 0.200\tMoving Average: 0.021\n",
      "Episodes 1890-1900\tMax Reward: 0.200\tMoving Average: 0.022\n",
      "Episodes 1900-1910\tMax Reward: 0.000\tMoving Average: 0.021\n",
      "Episodes 1910-1920\tMax Reward: 0.300\tMoving Average: 0.027\n",
      "Episodes 1920-1930\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1930-1940\tMax Reward: 0.100\tMoving Average: 0.029\n",
      "Episodes 1940-1950\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 1950-1960\tMax Reward: 0.200\tMoving Average: 0.032\n",
      "Episodes 1960-1970\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 1970-1980\tMax Reward: 0.300\tMoving Average: 0.038\n",
      "Episodes 1980-1990\tMax Reward: 0.200\tMoving Average: 0.041\n",
      "Episodes 1990-2000\tMax Reward: 0.100\tMoving Average: 0.040\n"
     ]
    }
   ],
   "source": [
    "# run the training loop\n",
    "\n",
    "# GAMMA = 0.99            # discount factor\n",
    "# TAU = 5e-2              # for soft update of target parameters\n",
    "# OU_SIGMA = 0.2          # Ornstein-Uhlenbeck noise parameter, volatility\n",
    "# OU_THETA = 0.15         # Ornstein-Uhlenbeck noise parameter, speed of mean reversion\n",
    "# EPS = 2.0               # explore->exploit noise process added to act step\n",
    "# EPS_DECAY = 4e-3        # decay rate for noise process\n",
    "\n",
    "scores, avgs = maddpg()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzsnXd4VEXXwH8nnV5DLwEMIE1KpIgFxYJIURTB3rEh9u4niK++6msvoKhYaQqCSFVEQDqhSC8hBEik95Ke+f7Ym2U32WRL9u7eTeb3PHmyd+6UM+fOnTPtzohSCo1Go9FoiiMs2AJoNBqNxvpoY6HRaDQat2hjodFoNBq3aGOh0Wg0GrdoY6HRaDQat2hjodFoNBq3aGOh0Wg0GrdoY6HRaDQat2hjodFoNBq3RARbAH9Rs2ZNFRcXF2wxNBqNJqRYvXr1YaVUrDt/pcZYxMXFkZiYGGwxNBqNJqQQkd2e+NPDUBqNRqNxizYWGo1Go3GLNhYajUajcUupmbNwRXZ2NqmpqWRkZARblDJFTEwMDRo0IDIyMtiiaDQaP1GqjUVqaiqVKlUiLi4OEQm2OGUCpRRHjhwhNTWVJk2aBFscjUbjJ0wdhhKRXiKyTUSSROQFF/cfEpENIrJORBaLSCuHey8a4baJyDW+pJ+RkUGNGjW0oQggIkKNGjV0b06jKWWYZixEJBz4DLgWaAXc4mgMDMYrpdoqpdoD7wDvG2FbAYOB1kAvYJQRny9y+JgDja9onWs0pQ8zexadgSSlVLJSKguYCPR39KCUOulwWQHIP+O1PzBRKZWplNoFJBnxBZWc3DyOn83yW3zZuXmcSM/2W3wFUUpx9EwWRR2dm5unOFbM/WCjlGLK6lTSs3IDmm5mTi4/J+4Nql72HDnLou2H7Nf5usjIdq8LpRQ/Je4lKyfPrd95mw9w4GTo9wJnbdjH0TP+ezeDhafP7p+9x9mYdiJAUtkw01jUB/Y6XKcabk6IyKMishNbz2KYl2GHiEiiiCQeOnSo4G2/s+foWfYcPUtWjueVl4hw++23269zcnKIjY2lT58+7Dp8ht1HzpCXp7j++uvp2rWrU9gRI0ZQv3592rdvT3x8PAMGDGDz5s32+z169KBFixa0a9eOli1bMnToUI4fP26/HxERwUWdO9GqdRsGDhzI2bNnAThw4AC33norTZs2pXu3znTp1o2pU6f6qhbTWLrzCE///A9vzNrs3rMfef+P7Tw7eT2/bz4Q0HQdufR/f3Hn2JX268VJh3n65394c9YWt2FnbtjHc5PX88n8HcX6U0px//eJ3PzFshLLG0wOnsrgkXFreOiH1cEWpcT8tt727D79K6lYf/0/W0KfTxYHSCobQV86q5T6TCnVDHgeeMXLsGOUUglKqYTYWLdfq5eY7FxbSzPPiwZnhQoV2LhxI+np6QD88ccf1K9vs3v5rYdjx4+zevVqTpw4QXJyslP4J598knXr1rFjxw4GDRrEFVdcgaNhHDduHOvXr2f9+vVER0fTv/+5zltMuXL8NPdvFq1YTVRUFJ9//jlK2QzTpZdeyoLEjUyctYCx3/1IamqqTzoxk1MZOQAcPJkZ0HQPncp0St8KnPZCF/m91cOnPWtp7z5y1nfBLED+e5R2PD3IkpSc/Gd35HRgy7wnmGks0oCGDtcNDLeimAhc72NYS9O7d29mzpwJwIQJE7jllluc7k/95Rf69u3L4MGDmThxYpHxDBo0iKuvvprx48cXuhcVFcU777zDnj17+Oeffwrdv+SSS0hKSmL+/PlERUXx0EMP2e81atSYxx57zNfsmY41B8mCg9La0AQJM5fOrgLiRaQJtop+MHCrowcRiVdK5feVrwPyf08HxovI+0A9IB5YSQl47bdNbP73pHuPxZCelUueUpSLCidMhFb1KjO8b2u34QYPHszIkSPp06cP69ev59577+Xvv/+23584cSLDh79K7dq1ufHGG3nppZeKjKtjx45s3brV5b3w8HAuuOACtm7dygUXXGB3z8nJYfbs2fTq1YtNmzbRsWNHL3KtsQLerBmw6BSUxgus+AhN61kopXKAocBcYAvwk1Jqk4iMFJF+hrehIrJJRNYBTwF3GWE3AT8Bm4E5wKNKqcDOcvqRdu3akZKSwoQJE+jdu7fTvSOHDpKUtIOLL76Y5s2bExkZycaNG4uMy92kq+P9jPR0br7mEq66rDuNGjXivvvuK+T/zZefoXuXTlx44YVe5ipw6LVVvqEXpYUeVn5kpn6Up5SaBcwq4Paqw+/Hiwn7BvCGv2TxpAfgjm37T5GZk0vz2pWIifRuJW+/fv145plnWLBgAUeOHLG7z50xjWPHjtk/YDt58iQTJkzgjTdcZ33t2rUkJCS4vJebm8uGDRs4//zzgXNzFnWrxBBbKQaA1q1bM2XKFHuYl954l0oqncsv6eZVfgKJFVtZoYC7HobugVgPKz+SoE9wlxXuvfdehg8fTtu2bZ3c5/w6mZmzZpOSkkJKSgqrV68uct5iypQp/P7774XmPMC2tcmLL75Iw4YNadeuXZFyXHHFFWRkZDB69Gi729l0a05w6paxb2i9hT5WfISlersPK9GgQQOGDRvm5Ja2dzf/pqY6LZlt0qQJVapUYcWKFQB88MEH/Pjjj5w5c4Y2bdowf/58HFd+3XbbbURHR5OZmcmVV17Jr7/+WqwcIsK0adN48sknefOtt6lSrQbVq1Ti7bff9mNuNWbhSW9A9xhCHys+Qm0sTOb06dOF3Hr06EGPHj3YmHaCeYmbC33xvGbNGgC6dOnCiBEjiox7wYIFxaadlHbIvgzUkbp16zJx4kR2HT7DqYxs4mpUoHI5vemftfG+ral7GKGHlR+ZHobSaDQajVu0sdBoyihWHOrQWBdtLDQajUbjFm0sNJZHT9hqNMFHGwuNJoTwxG5q2xr6WLGBpI2FxvLoVT2+6UCrLfSwclnXxiIE+fzzz/n++++DLUbAsGIrKxRwpzarnmNSlrHyI9HfWYQgjjvGlmYs3MiyNFpvoY8Vexi6Z2EiKSkptGzZkrvvvpvmzZtz2223MW/ePLp37058fDwb1toOazl69CjXX3897dq1o2vXrqxfv568vDzi4uKcDjOKj4/nwIEDjBgxgnfffRewfeD3/PPP07lzZ5o3b27fzfbs2bM8ePft3HBFV+669Wa6dOlCYmJiIRk///AdelzcjTZt2jBkyBCUUmzdupXOnc8dTJiSkmLfpmTWrFm0bNmSTp06MWzYMPr06WOa/jS+YeHGqcZDrNjDKDs9iyeegHXrShRFQ2OL8qiocJvpb98ePvyw2DBJSUn8/PPPjB07lgsvvJDx48ezePFipk+fzkefvseHX43jtRHD6dChA9OmTWP+/PnceeedrFu3jv79+zN16lTuueceVqxYQePGjaldu3ahNHJycli5ciWzZs3itddeY968eYwaNYoqVasydf5yjqft5IruXVzKd8tdD/DW669RuVwkd9xxBzNmzKBv375kZWWxa9cumjRpwqRJkxg0aBAZGRk8+OCDLFq0iCZNmrjco0pjLt5UIhZsnGrcYMUeRT66Z2EyTZo0oW3btoSFhdG6dWt69uyJiNC2bVv+3bsHgCVLlnDHHXcAto3+jhw5wsmTJxk0aBCTJk0CbGdeDBo0yGUaAwYMAKBTp06kpKQAsHjxYvoPGAjA+a1aF7m54Kplf3PFpd1p27Yt8+fPZ9OmTQDcfPPN9rTzjcXWrVtp2rSpfYdcbSwCh4XrEE0Zoez0LNz0ADxhrw9blEdHR9t/h4WF2a/DwsLIzS3+iI5u3bqRlJTEoUOHmDZtGq+84vrU2fw4w8PDycnx/CjQzIwM3nj5Gf5eupxW8U0ZMWIEGRkZgO1UvoEDBzJgwABEhPj4eNaVsGem0WhCF92zsAAXX3wx48aNA2ybA9asWZPKlSsjItxwww089dRTnH/++dSoUcPjOLt3785v02znVmzbuoUNGzYU8pOZaTMMNWrU5PTp00yePNl+r1mzZoSHh/P666/bezQtWrQgOTnZ3nvJ73loQhMLDotrLEzZ6VlYmFeHj+CB+++jXbt2lC9fnu+++85+b9CgQVx44YV8++23XsX5yCOPMOhW2wR3y5YtaN26NVWqVHHyU7lKVW685U66JnSgXt06hU7LGzRoEM8++yy7du0CoFy5cowaNYpevXpRoUIFS5+up9Fo/Is2FiYSFxfndESqY4UfFxfH1D+XkacU1atXZ9q0aS7jSEhIKLQe3nHbcsdtymvWrGlv9cfExPDJmLGcyhYyjvzLoOuvo3HjxoXiH/rcK7z79n9dblH+zDPP8Mwzzzi5XX755WzduhWlFI8++miRp/ZpzEIfaFE2sN4z1MailHL27Fmu79WTjMwswsOEUaNGERUVVeJ4v/zyS7777juysrLo0KEDDz74oB+k1bij4JknnoUxQRCNqYiFlzJoY1FKqVSpEnMWLOHQqUynM7hLypNPPsmTTz7pl7g8x3qtrFBAdzBCD2Xhsl7qJ7j1lgaBx18696U1rcHjLoV+NayM9cp+qTYWMTExHDlyRBuMAKKU4siRI8TElLwno5+bj2i9lQKs9wxL9TBUgwYNSE1N5dChQ36J78DJDLJzFRyPJjK85Hb2wPF08hSEn4whzIRW9In0bE5l5JBRLoLDMYUnsA+fziQjO4/co1EefzfiCTExMTRo0MBv8WnO4dUX3NZrnGrcoOcsgkRkZKT9a2N/8Oi7C0g+fIY/n76MZrEVSxzfgP+bQ3p2LptHXkP5KP8/ijdnbWHMor28eG1LHuzQrND9e75ZyV/bDjH27gSuaFl4G5Fgo4ehzqE1oQk2pg5DiUgvEdkmIkki8oKL+0+JyGYRWS8if4pIY4d7uSKyzvibbqacnmJWx1CPGmjcoYuIJtiY1rMQkXDgM+AqIBVYJSLTlVKbHbytBRKUUmdF5GHgHSB/A6R0pVR7s+TTaMo6Vl55o7EeZvYsOgNJSqlkpVQWMBHo7+hBKfWXUuqscbkcsPRAtx4K0AQLXfY0wcZMY1Ef2OtwnWq4FcV9wGyH6xgRSRSR5SJyvasAIjLE8JPor0lsjfXQw3Tn0KrQBAtLTHCLyO1AAnCZg3NjpVSaiDQF5ovIBqXUTsdwSqkxwBiAhIQE/R5pSi3ezPXrFyH0sWIDycyeRRrQ0OG6geHmhIhcCbwM9FNKZea7K6XSjP/JwAKgg4myeoRpE9wmxaspPVix8tCULcw0FquAeBFpIiJRwGDAaVWTiHQAvsBmKA46uFcTkWjjd02gO+A4Ma4pA+hxet/wVG/aAFkXK64aN20YSimVIyJDgblAODBWKbVJREYCiUqp6cD/gIrAz8aa+j1KqX7A+cAXIpKHzaC9VWAVVVCw4PMr1ei67Bx6GKpsYUVDbuqchVJqFjCrgNurDr+vLCLcUqCtmbJpNKGIN1ugWPlrYI1rrNijyKdU7w2lCW0s/N4EHCtXIpqygTYWXmDeF9wW7HNqLIUuIppgo42FRqPRaNyijYUX6JEATbDQw1CaYKONhcby6BGYc2hdaIKFNhYaTQjgzcomPb8R+ljxGWpj4QX6C25NsNA7xGqCjTYWPqCHjwODHqf3Da230MeKz1AbCx/QbbzAYMWueLAwYxhK69e6WPHZaGPhBRY09hpNkVixdaopHis/Mm0sNJZFV3aFsWKLU1M20MbCC/QZ3BqNpqyijYUP6AavRqMpa2hj4QO6I6AJOCa0UPRyXI03aGPhBbpHERz0RosaTfDRxkKjCSE8MZvauIY+Vuz1aWPhBaY9PuuVC41Go3FCGwsf0MNRgUEvnfUN0YoLeax4yqE2Fj6gOwKBQY+mnMObqsPTYSitX+uih6FCHOvZeo2maHR5DT2s3CnUxkJjWaz84gQLPXmtCRbaWHiBfk01wULPQ2iCjTYWFsCK45MajUbjiKnGQkR6icg2EUkSkRdc3H9KRDaLyHoR+VNEGjvcu0tEdhh/d5kpp0ZTFtFNFI03mGYsRCQc+Ay4FmgF3CIirQp4WwskKKXaAZOBd4yw1YHhQBegMzBcRKqZJaun6IGA4KArNY0m+JjZs+gMJCmlkpVSWcBEoL+jB6XUX0qps8blcqCB8fsa4A+l1FGl1DHgD6CXibJqNKUGbVxDHyuuYzDTWNQH9jpcpxpuRXEfMNvHsKZy8FQGXy5KdvsSJh86zfgVe5zc1u09zsz1+1z6T8/OBeCLRckcPp1pd8/KyeOjeTvIMO5P/+df1qcedxlHRnYuH/+5g+zcPAC++juZ/ScyAPh+WYobic3lbFYOH/+5gxxDNkc2pp2g14eLOH42y208AsxY/y/r9rrWQT7fLtnFK9M2sG7vcf47ewtzN+1nadJhJz+/rktjY9oJ/tp2sNC9fHl/WZPmPnMO/L3jEI+OW8POQ6cBOJWRzafzd5Cb5583fs7G/azZfcytv7V7jjmVtaImxZVSjF6wk2+X7Cp074Up6/l0/g6X4X4rphx6y/YDp/g58dwrvnjHYRZuP1TieP/Ze5zf/vm3yPuHTmUyZtFOlFKs3n2UORv3O93fc+QsPyzf7VO6M9YXnW5R/LImlc3/nmTMop0cPJVh/xhv478nnfwppfh84U6negJw0qHZRAQspWIQkduBBOAyL8MNAYYANGrUyATJbDw2fi0rdh1166/fp0s4nZnDrV3OyXL9Z0sAuK7ddUWGG71gJxvTTvDDfV0A+HH5bj6Ytx0RGNYznmET1gKQ8lbhOEb9lcTH85OoWj6Sy5rH8p+ZW/jtn3/54f4uZGQXrqQDyYfzdjBmUTJ1qsRwc0JDp3t9PlkMwIu/bGD07Z2KjUcBQ8cXrQOAY2eyGPHbZgB+XO5ssB3DPD5xXZH38uX1lju+XgnAvC0H2Pafa3lz1hYmrNxLs9iKXNu2rtfxFeShH1d75O+GUUsBGN7XNtpb1DLbVSnHeHvO1kLux85kMXGVrfIZekV8ofuPFVMOveXqDxYBMNAoF7d/vcIvcfc33re+F9RzeX/YhLUsSz7CxefFcuPoZYXSHPjFUg6czGRgpwbERIZ7nW6fdq7TLYqnfvrH/nv+1oPc0MHWJt6yz9lY/JN6grdmb2XZziN8d29nu/uzk9fbdWg2ZvYs0gDHXDQw3JwQkSuBl4F+SqlMb8IqpcYopRKUUgmxsbF+E7wgJzNyPPJ3OtMzf+7C5vc48v8Xx9ksm5/M7Dyyc22Vw6mMHFRw7QQAZ4w8ZeYULUxJdOZInh/67SWVJT+fpzNtzyTLRY/KCmQXIVeuFcc+/MypzGyAInt9x85mB1IcJ4orf/m9c3+9L75gprFYBcSLSBMRiQIGA9MdPYhIB+ALbIbioMOtucDVIlLNmNi+2nDTeIKeidegv80IRay4J1Q+pg1DKaVyRGQotko+HBirlNokIiOBRKXUdOB/QEXgZ6Ng71FK9VNKHRWR17EZHICRSin340CaUoWVX5xgUQYa/xqLYuqchVJqFjCrgNurDr+vLCbsWGCsedJpNBqNxlP0F9weEDr78YSKnNYkZB6zSZT1/AP6FSoGbSx8wOqDI1aXT6OxOnq6pzDaWPiAbnwEFt3i1WiCjzYWpQhdqZYU6yvQk00nfS0HekNLrYPi0MaiFFLalkyWsuxoNEVj4bKujYUJhM6EeGig1akJNEErcxYu69pYeEDBgmNV42/hchYSlHmjVNbzjy4DxaGNhQ9YvTxZ1ZhpNBo3WPjl1cbCBHTrxE9Y+MUJFrpsaYKFNhYajUajcYs2Fh4QKsvpdKuzZJR1/YVq9v353EJVB4FAGwsfsProiF5qqimOsm4UrYrVn4s2Fj7g7pla/JmHHFqfmkBj9Yo7GGhj4QGhUnBCZbjMqoSC/jwpi+68FNXzDJVybibB/EbK6iMC2liUQkrbORClKzeBw+qVj6YwVn5k2liYgP6C279obfqGLoa+E6xeppUfmcfGQkQuFpF7jN+xItLEPLGsjZWtf2lC69k3tN40ZuCRsRCR4cDzwIuGUyTwo1lCWY2C1t6q1l+3JEtGadGfr9kIhTkbswlFDQRqJMPTnsUNQD/gDIBS6l+gkllChTrBLnB6rFoDuhyUhGA1HHx5ZIGS1VNjkaVs5ksBiEgF80TSaDRFoVv/mmDhqbH4SUS+AKqKyAPAPOBL88SyFqEyYR0iYlqWsq6+UC0/fv2COwR1ECiRIzzxpJR6V0SuAk4CLYBXlVJ/mCqZhdG9e41GU9ZwayxEJByYp5S6HCizBsIRt19wh2DrRKPRnCOUXmHbyIf5TVi3w1BKqVwgT0SqmC6NRQmVgqPHs0tGaTHyvg6blpLslzkC9dw8nbM4DWwQka9F5OP8P3eBRKSXiGwTkSQRecHF/UtFZI2I5IjITQXu5YrIOuNvuodyaih9Z3BrzuGNHShtX/KXBaz87no0ZwH8Yvx5jDF89RlwFZAKrBKR6UqpzQ7e9gB3A8+4iCJdKdXemzStgm7h+5dQWWBgNcpKOTQjn8Eqc76kGyhRPZ3g/k5EooDmhtM2pVS2m2CdgSSlVDKAiEwE+gN2Y6GUSjHu5Xkpd1Cxru0/R2moKKzcyrIyWm8aM/D0C+4ewA5sPYVRwHYRudRNsPrAXofrVMPNU2JEJFFElovI9V6E8z8hUu+WtgZ4oFt3pcHAQgnmLEpbASojBKrcejoM9R5wtVJqG4CINAcmAJ3MEgxorJRKE5GmwHwR2aCU2unoQUSGAEMAGjVqZKIozlh9NZRuV2pAz1mUhGC9wr70Cq32BXdkvqEAUEptx7Y/VHGkAQ0drhsYbh6hlEoz/icDC4AOLvyMUUolKKUSYmNjPY26TBBsg+UP9HBKYUrBY9WEKJ4ai0QR+UpEehh/XwKJbsKsAuJFpIkx3zEY8GhVk4hUE5Fo43dNoDsOcx2BRr+gZYQy/qBDtYERqnKHGp4ai4exVdbDjL/NhluRKKVygKHAXGAL8JNSapOIjBSRfgAicqGIpAIDgS9EZJMR/HxsBuof4C/grQKrqDTFoBvkGo3G33g6ZxEBfKSUeh/sy2Kj3QVSSs0CZhVwe9Xh9ypsw1MFwy0F2noom0ZT1jsFGj8TSr0Vq81Z/AmUc7guh20zwTJBwVUiVi1IjnJZVESvCPxqqLKNVcu1pngCtRrKU2MRo5Q6nX9h/C5vjkjWJVSGd0JFTo0PePMFty4HIYeVH5mnxuKMiHTMvxCRBCDdHJGsi6ctL91CCzylVecl6V15GzRUvzMxReogqcKXZC31BTfwBPCziPxrXNcFBpkjksYf6A+sSilWbnpqSjXF9iyM1Up1jInolsAkIBuYA+wKgHyWIFSq3VBtGRZFoHMTEgbWRBFDIfuawlhl19kvgCzjdzfgJWxbfhwDxpgol8Wx9lulv9zVQNFzFtooWBOlrN1xdDcMFa6UOmr8HgSMUUpNAaaIyDpzRQtdrNDCD74EgcUKOg8EZuYzGBpUSlnyS/1glSff5iyssRoqXETyDUpPYL7DPU/nO0KeUGmJhYqcHhPg/FhRfWY806J7HFbUgHtCVe6CiPi4RbkJsrjCXYU/AVgoIoexrX76G0BEzgNOmCybxkcs2FDT+Ak9xKgJFsUaC6XUGyLyJ7bVT7+rc2YvDHjMbOFClVLS0AktyojOS98wlDUbN8F6h0N66axSarkLt+3miBMahIIxCAUZPaU05UWjCVU8/SivTBMqk6ehIaXnBFrvoWCUPJHR13yEQv41LrDY3lAaLwj2O2fBXr3pBFvnZmGV3bHMmkT2R6xmSBa08uRDwlbbG0oTYoRKb8iKyyatjCfq0ioNXaz83mpj4QGh0j0PxSWExckc6OyEgvaCNQwVgkWrzGC1Lco1Dlj+vdFNSw1FD0fqit+aWP25aGNhAqHYwg91yorKzcxmcXGbla5V35VgyeVLslbZG0pD6FREqsiL0CTgw1AWfNBmyOTLnlFW1E0+FhbNK0RCe7sPTQiiB6FKL/rZaoKFNhYm4Imdt3JLLRSx8ioSf2LuMFQxiw1MS9OaBEsuPQylCThWfQkL4snS2bJiCDQaK6ONhQ9YtVNgVbmKo9ilswGUIxjpmYWvxlUvnQ1N9NLZEMbMtfCeUBZXzpbWyiyQ2SrWWJgkiX+em/9lC95Ggr5sUa4nuMs8ZaHOL81fcJfmvGnMwcqNHlONhYj0EpFtIpIkIi+4uH+piKwRkRwRuanAvbtEZIfxd5eZcrojaGuuSxDCyoXOkeK/4A7tT7jNkN+TOH3+grvYvaF8i1MTAEJ9GEpEwrGd130t0Aq4RURaFfC2B7gbGF8gbHVgONAF6AwMF5FqZsla2tDt2bJNft2hz+AOLaz+XMzsWXQGkpRSyUqpLGAi0N/Rg1IqRSm1HsgrEPYa4A+l1FGl1DHgD6CXibJ6hdsxQk/mLPwjisZA6/McobRFuVVXuoXUGdx+l8I1YlZX3xhW6qWUut+4vgPoopQa6sLvt8AMpdRk4/oZIEYp9R/j+v+AdKXUu0Wll5CQoBITE/0iu1KK/87eyuyN+2hVtzJzNx1wut+8dkXevrEdn/21k3lbbPd+fqgbAz9fBsCP93Xh9q9XFBn/9e3rsTjpMIdPZxW6d0vnhjSoVp7/zd1G/arlSDuebr/3wCVNEBHGLEp2m4dlL15Bt//Od3KLq1GePu3q8elfSbSpX5mNaSft9768M4EHvrfp759Xr+aCkb8DcE/3OF7t04oVu44yeMxyLomvyYa0E6x55SrGr9zDK9M28lLvltx9UROavzKb92++gAEdG/Dtkl2M+G1zIbla1qnE1v2nCrlVKRfJil1HAagUE0GYCLl5itOZOYXiGNCxPrEVo/liUTIXxlUj9Vg6+05kFKmLx644j0/mJxXSJ8CWkb3YkHaC2Rv38c2SFKd7fS+ox6a0EyQfPuPk3jS2AnuPnqV25RhSjznH16lxNVbvPuYyz4uevZyrPljIyP6tSTuewcd/7uDi82pStXwk9auVo1r5KNo3rMrcTfvtstzYsQFT1qQ6xRcmkKegavlImteuxOALG/LUT/+4zHuNClEcOWMrZwuf7cGpjBxGTN9EYgEZb+nciBXJR+x5bV2vMjsOniYrJ4+WdSoRX7sSv/3zr1P1XfLrAAAgAElEQVSYJ66M58N5O3i4RzOW7jxCmEBkeBgrdx0lPMz2/O6+KI6nrm5OuxG/U7V8JO8NvICh49eSnp0L2N6l7QdOF5K7Y6OqvHNTO9bsPs7W/adYsP0gkWFhpBw5Q4s6lVifajvVOfnN3kxbl1Yo/1e0rEW9qjEIwg/Ld9OhUVXW7jnuUkdTH7mIG0YtdXKrWTGaP5+6jAtG/s6nt3agVd3KfDhvB6/1a025qHBa/t8cmteuSERYGPd0j+PZyesBuK5dXXYePG0v469cdz73X9KUhdsPcdfYlU5pzHvqMq58f6H9uuA7Wb9qOZrVqsii7Ydcyu3Il3cmcFWr2m79uUJEViulEtz6C2VjISJDgCEAjRo16rR7926/yJ567CwXv/2XX+LyhWevacH/5m4rURyujIWn3NSpAZNXn6ugFj17Odd98jenMs5V3HOeuIReH/5tv/701g4MHb8WgJS3riPuhZk+Sh5YXFUUZtGgWrlCxiWQtKpbma37T5IX4Ebzwz2aMXrBTq/DDb38PD79K6lYP7OGXULvj/8u1o+v3H1RHN8uTQFsxnTCyj28f/MFZOXk8cIvGzyOp6j3oWDjpaCx8JaUt67zKZynxsLMYag0oKHDdQPDzW9hlVJjlFIJSqmE2NhYnwXVOJNXoAEhgpOhcIXVx1uLIpArlnIDXUsXIE+pgBuK/HR9IdhDVI4N6UyjJ6SU/4Z9gl0evMVMY7EKiBeRJiISBQwGpnsYdi5wtYhUMya2rzbcNBqNRhMETDMWSqkcYCi2Sn4L8JNSapOIjBSRfgAicqGIpAIDgS9EZJMR9ijwOjaDswoYabhpPKRELf3QavBoNJoAEGFm5EqpWcCsAm6vOvxehW2IyVXYscBYM+XTaDSaYFFwmM3qQ7n6C25NISxeZjUaTRDQxqKUUqJRKKs3cTSaUojVd4fRxkKj0Wg0btHGQlMI3a/QaMwn1Drw2liUUvRQkkaj8SfaWGgKoe2MRqMpiDYWGo1Go3GLNhaaQuiOhUZjPqH2nmljUUopyVBSWZrvKEt51WhKgjYWGo1GEwQKtlOs3m7RxkJTCIuXWb9SlvKq0ZQEbSw0Go25aItcKtDGQlOYMvRyW73rryk7WL0samOhKUSwD50JLGUpr8FBa7goVDFX1kMbC41Go9G4RRuLUkrJls76Tw6rU5byGiz08mTPsLqetLHQaDSmYvE6UOMh2lhoClGWXu4ylFWNxQi190wbi1JK2Zqk1nhCsConXRJLB9pYaApRlgxNqLXuQhGtY8+wup60sdAUwuqF1p9YfVLRnwSrEVCWGh/eUFArVteTNhaaQnhSZMtQHes3gq2zoA1D+ZhusPXliIVECRraWLjASoXUV0pDHgKBVpP5lKXeW0mwupq0sdD4hEiwJQg9gq2zYNVFvqYbbH05YiFRgoapxkJEeonINhFJEpEXXNyPFpFJxv0VIhJnuMeJSLqIrDP+PjdTTo0zobZ1ckkozXmzClrHrinY47K6miLMilhEwoHPgKuAVGCViExXSm128HYfcEwpdZ6IDAbeBgYZ93YqpdqbJV9xlIbCXbIsFCzEpUAhRVCa81aQYA0HlSUdlwSrD9eZ2bPoDCQppZKVUlnARKB/AT/9ge+M35OBniJW6nyWTTwpsxYv15Yk2DoLVvJ5ITrBrYr4bUb8ZqXhT8w0FvWBvQ7XqYabSz9KqRzgBFDDuNdERNaKyEIRucREOQuhW0LOBPulNZXSnDeLUKrLTxnCtGGoErIPaKSUOiIinYBpItJaKXXS0ZOIDAGGADRq1CgIYpZddP/Pe4Kus6BV2r4lHHR9OWAhUYKGmT2LNKChw3UDw82lHxGJAKoAR5RSmUqpIwBKqdXATqB5wQSUUmOUUglKqYTY2Fi/CV4aWkIlGf8Mte5xSSjNebMKofo+BVxui+vJTGOxCogXkSYiEgUMBqYX8DMduMv4fRMwXymlRCTWmCBHRJoC8UCyibJqNKWeoC2dtXglGCwKrToMjhgeY9owlFIqR0SGAnOBcGCsUmqTiIwEEpVS04GvgR9EJAk4is2gAFwKjBSRbCAPeEgpddQsWQvJHqiELEqhJX0u3vbSUgEEMh/B1lmwVtvk+Zhu0PXlUBOUkuJeIkyds1BKzQJmFXB71eF3BjDQRbgpwBQzZSsOqy9h84SS5KBsDUP5J3cNj++nR3IiP3S4zlqD7RYgVMtPoKsBb+udyhmnuTJpBb+0viIgZc6qE9waTchQIfMsf39xPwCv/fEFve/5mK21mhTyF2wbEmrDUMHWlyMWEgUAUXms/8g2EPP+zA949coHgetMTVNv9+GCUG0J+Qv9Bbd3jJv0sv13GIoBG+eXPNJSRKguRTdb6pJ8wX319uVO10OX/eQHiYpHGwuNT5RmA+IN12xfSvt9O5zcuu1ZD0pR48xxonKygyRZYYL2zHRZ8Tu9ty2x/95VrS5D+z9vepp6GMoFpaEiLEkeXExnu3AJjJI6793I//35JasatGbeeZ1ZGuffHWBKmosvpr4JwDuX3smobjeT8nYf2h7YyeRxz5GQtoWDFarReegPtrQsNGEbSMyc4DYzT47pB0Jz3qip3slDLG3UjltvedM8gQqgexYuCfxLVefkYZocLfgZiv/omLaFRZ/fx6fT3qJ7yjrC83IRlUfr/UlM/+4JBq+bY/erlCL29FGic7KMa9PEKpZKmWf4afwLtD2wk3tXT2f8pFdIebsPKz+9g4TUTcWGfXzxeGZ8+zgVMs8W68/bScV6Jw8yfN4XfDrtLRoe3w/A4fJVGN31JgA+6Wbb2iwhbQsAtc4co/mhFK/S8Afhebm0/3cbc75+lH6bF/LXmAdovbuwzqqkn6LHzkRisjM8irdn0grWf2Azinet/s2jMGYWH3PLprkFvySHH7U5sJN9lWv6VyA36J6FBWh2ZC9/fvUwAOvqxkPyJYytfQVHKlT1Wxq9ti2l0YkDNDpxgD7bFvNj+2tZ0bANn/z2PwDa7f+UeicP8VXnGwjPqcqqz+4kKyyC5s9OcxlfIAxI0yOpLt1rnTnG5HHP0+S56Sgp3N5pcjSNJ5eMB2yV9sKmnUokR2RuNvet+pX9lWrw4Yz37O59ti0G4JHrX7TL8d4lt/PgiilE5eVw423vMGXcc/w+digfdxvEXxf3ZX+Y/55pUYTn5bLzf+e2YfvYeMZD5nzNnIFvOPn95+NbANhaszG97vuMiplneXDFFL7r1IfDFao5+R2w8U/en/mB/fq1eV/wXae+buWx0gR3zTPH6LtlEd906kentC3027KQfZVi+bzLjfYEe29dTGqVWjh+UxzsCW5Rebw38wNqnjnOllpNKJeTSXK1grsnmYs2Fi4IdEv61T+/tP9uv28H/LyD1YwFYF3d5tx580hOxlT0MlYFSjFh4kusq9uCIaumsqdKbRqdOADA7etmc/u62U4hhi2bxLBlkzhdvjIAUXk5nHd4T0D7WZG52ex49wYntwcGvMILC75hVNebWVuvBfevmsqt/8zl+QXf8laPe5xqlbijaXT4d6v9+rufh7O8YRsevv5FqqWfIisikk9/fZsaZ08QnpdLXmo/aDCgWJn6bV7ECwu/dXlvdb2WrGzQ+pyDCAmP/UjsmWPsrNGQRXEduDRlLcOWTeLR5T8T/8xU8sLCvVeMB0TnZNFjZyJfTHMemjhcvgo1z56gYvoZROXZDVt+zxGg5eHdPLZkApftWkNC2hZqnT7K870fd4rH0VDk88iyn0hs0IqWB3dx9+rf6HvXh5yJLg/YjPauavU8Lj/1TxzkcPkqZEZGe5Frz6mUeYa5Xz9KjfSTHC1Xmaf//tH+PkTk5TCn+UWcjYph1K9v2QJ8D/8XGc0tg9+EDuZXzEXVO3eu/o2R876wX1+ashaACe17mS6TI9pYBJEmR9N4fuG3XLZrDdtrNCI3LIyPut/C59P+a/fTft921n80mKWN2rGpdlPe6nEPuWHh9N28kLTKtVjT4HynODulbqbP1r8Ju7MVXfZupNueDXTbswGAxXHteanXYzy6dBLP/v2DPUzTZ39lxai7iD1zHIBtzS+g07q/AWh9YKfZarATlpfLZ7++Xch9RcM29Hzg3MvyYfdbufWfuTy08hfK5WSyq1p90iOjOVK+Kl/98nqh8F33bmTtJ7e5TnTcWP7XZg8v9hpKTrjr16HX9iVO15c8+BX9Ny+gSsZpPut2c6Em8MmYinbjPvyqh/jrywcBCFd5fDjjPX5qexWLm3QoWhE+cm/irzy/8Dv79Ted+vLalba0p33/FO33bafXtqXMbnkxAFckrQRs8y3PLfqeGzb9RdNj/wIwaMMfKBGic7Konn6SFQ3bAHA8piKdH/2BquknWTnqLp5b9L2TDAM3zOPbhH5cuWOF/Vk89sMqt7Jfs32pff7n/Ytv4+OLBrsJ4T3vzPqIGum27eU+MnqIKxq2ocvejTzz94888/ePLGzS0SlM+exMfv3haSaV+z9+qdbF7zK5Izwv18lQ5PPK1Y9wtHyVgMqijYULAtGSjj+0m75bFtFr+zIAPrloEL+1ugyAGW9+SbMP3mBO84vswykX7VnPRXvW83t8V0bMG0Prg7bdT1o8NcXeEovOyWLKuOcAOPZpHN9MHuOU5gcX2yrMLzsPYFmjduSGhXG0fBXywsLp9vC3tDqYTGZEFLHdEli/MYX1Hw3moxnvkd57IXLlq05DPmb0vnruXMXVO2xLAofc8DKv/jmGtfVaFupVHaxUw/77rjUzXcZ1LKYS4Xm5VM46S2Z4JNG551YlHY+pSNWM0/brgRvnMXDjPJ7o8zTTWl9OWF4u0354mo21m/HJRYO5KmklSdUb8Hi/Z8kKi2Bv1Tp86mFltqtaPRY3voDd1epy27o59NuyiH5bFnHJg19xpHwVzkaV81g/7mh5MAWAtXVbUCXjFO9fcrv93gMDXmHVZ3cy+te34Fc4ULE6tU/bNkVY2LQTFbLSeXT5z07xDV7/u/33ZbvWAHDzrW+RFRHJwUo1yJUwwlUeAAcrVKPWmWP0SF7NvPguTkb7kzsu5BNgYZOOPHvt4/bn1+JQCtti4yiXlcGoaW/Z/T+1eBxPLR7HvJMvUU/Oo9Hx/axo2Map/PXdvJDX/xhNevizQPHDjDXPHKNcdibXbl8KQHpENCdiKpAr4Qzt9zy/f/0I1TJO2fOZI2E0f2YqT1U7ydCX7gBg0JjXeePxicU/gCIocl7MgyXqP056xf77w+638EOH62h6NJVVhvEOJNpYBIjqZ09wd+J0Ov67lQ7/bqOCw4Tix90G8Xvzbvbr3V17MPTeugCM6jaQh5dP5qnF4wD4ebzzgYOvzfuC0V1vYne1evYJV4Bqoz8B4HRUOdo88ZNT6zcrIrJQjyQnPIL1dW17NdZEcTKmIukR0ZTLyaTcmkS6xa8vdiVSSWxH2307uOWfOdz6z1wAEob+wOEK1Zx0UpC452fwzqwPuXnDvEL3Rne5iSWNL7C33hsf+5frti7mRExF7lo9g3sGjmDJ5/cyN74r0VMm06NdAwA+nPEeAzbOJzo3m3b7k2i3P8luzD+4+DY21W7mfeZEuH2wbZ7gvNOH6JK0GsD+EV/P+0ezs0bDIoN7Q/PDu/mraSfuGfhaoXuHKlbnwz6P8MSMUQB2Q7GhdjM21W5GC4dJ+Gvu/ZTeW5fw4MopxDgMVd1029tsj42zXy+Oa89lu9bQ++6P2Vy7KYtH30OPXat5YOUvdj/r6sbblxZftmsNK0fdRcunJnPrurm8Ov9Lnu/1GG/PsZXVZY3akly9PrcZiy2u/PxNrjTi+W+Pu/mp7VUMWzqRuc272efaqr45nOb3fuoklyMx2RkkfnqH/fqVqx/hxw69nfw8f+0w2u3fYf9WYWOd88gLC2dPfFvinp/B5TtX8c3k13hk2U9wR3eX6fiT29fMpEdyIlfutPXI0irF0vOB0WRExgD4dS7TG6Q0bG0BkJCQoBITE/0S17b9p7jmw0V+ieu1P0YX2foF2BIbx7X3furk9uw1Lfjf3G1ObpUzTvPH14/YX/LPug50agk+MOAVvvzlPwA8eP1LvFM+lSrjv2dEzyF8m9DPK5m7n1eDJUlHiM7OpOmxNGZ/M4yprXrQ4MRBttaKY07zi3jhjfvp++U5fX98SweGTbCNpaa8dR1xLxSdZ0f+O/tjbnFowf4e35UhA14pJsQ5wvJyiT+ylx01GhKu8uiyZyMb6pzHiXKV3IatdvYEZ6PKMfr+ixj2xSJem/c5NxbxMd3uqnW47MGvPJKpOJpG5VI9eSvvzvyQuOP77O5xz8/wKb6onGwaH/uXHbGNeXj5zzy/8DvbEE73W1z6r1clhuhdOxk17b/UPn2U7PAIhvV9lhWN2lL/xEG+nvwa01tdxqhuN9sCKEWdU0fICQ+nxtkTbCtQIYfn5VL97AkOVawOYJchn3aPT+RkTEU+2fEbHeb9QlKNRvTYtbrI/Nxx80j+btKR+icOct3Wv3l081zOpGdR79Rhl/7z52Iclyc/sOIXnl78I5tqNaX1wWQnY/fHeZ156IaXyS1izqjtvh28M/sj7hr4Ggcr1eDmhAb8lJgKSpHyjm0i/1jLtsz+dgYvTd1QZD4Kkvxmb5q+NKuQe6XoCE5l5tiv61ctx6HDJ9n+nvOcXb4xdkfKW759wS0iq5VSCe786Z6FC/yxdvuzaf/lum1LXN576ronORVdgXb7dvB9R88e8MmYilxx/+fctWYG49v3IjM8yslY5BuKo+UqM7d5N567tx2fpwmT215ZVJRFkt9+yIyMZkutpmS0asMNmxcAcGHaZu5YO4vMld8QNeBdsiIijTDe6+yKpJV2QzG1VQ/2VK3LV52v9zh8Xli4vQLLI9yreYBjxnivUnA6ujxPX/eUk7HYWrMxWRGRtNufxGs9h3gcb3GcjalAcoPW9HjwSxoe32/vXVyavJpFXqzYCs/L5YnF42lzIInLk50r30ntrio27K7q9Qs1TgDSqtSi132fOTuKsN9YnllwZRRAbli43VAAjL+gl91YXH/He/bhw5nXP8Bj8X0RlceCMUNo7NADzueG299lbf2WdlnGdLmR8OefY/SCnfzfn19yX+KvTv631WzEfTcNZ866sdRavoT3ZrzH0fJVeGCVbfVep3+3klj/fBIbtOJYuUqM6TzA5co5RzbUjXfSjb1Ii7Dw2lu5bPZ4qm3dQKVdO1xHUARFvRmFls4qRd1ThwDIDI9gausreK3nENKjYrxKzyy0sfAjNc8cY8noe53Gxwuypl4LfmnTE4A/4rt6Ff+Z6PLnWn3AhY/+wKW71vDeLNsqlWMxlbjswS9BhLyq1RjdtdAejT5xfNBt1Bn+opNb9K6dPLX4R9667G6f1ziOnTISONeiDDavXPUw5x/axZcX3sC/lWsRk5NJrdNHSfLTMJGjmvZWrcM1937K3LFD+f7n4XR4bJzdgBXHkBVTeGnBNy7v/XFeFw5UKnrtvdljCCfKVaL501OJycl0mmfKb3wpCaPn/Z/T5sBORCnqnzzI40sm8ND1L7GzZmEd51fWb1x+L7+2uoxcCSOpZiPKZWdwvJxtxd6/L4+ked+e3LjpL6ew22s04s3L72VN/fMLRusTM257gp8iG/LZ9Lfp9p9noI+tcfbOrA/pv3kBFz38rVfDQx3TtnCodkMyJIZlo+8m9sxxZna8miWx8QDcPXAkyxq384vs/kIbCxd43UhWijCVx32rfnUyFKmVY3nwhpfpnLqJOc0vYl9l/x3QBHCoYjWmtO3JlLY9C4tUgngL5v/UVddSZ/iLbKrVlG8S+pFSrS6Txz3PQyumMLPFxWyoG+91Gt12/2P//Xec/1cGeYpjXn8s0MvLioj0YcmyZ2kBbIuNY0rry7lx01+s/eQ2mj77a5HLaqNysvl68mtcsnudk/sH3W8lPTKaP5t1dlnhFpe+GWRFRNp7m/k4nsGdEx7BunotAFhbvyUzzr+0yLjyjUxeWLh9Pg0gMyLK/vtsu3NlZ2f1+jQ7msaV940iqWbJT850VFdORCSzWnbn9OpG1Ny0jpRNffjywuvtc2arP7UtJrjw0R84VNG5F+bY647IzeG9mR/Qf8tCAEZ1vcm+CvG6Nb9zHbaednL1eiWW399oY1FC8ifQKmalA3AqqhwXPfItFTLT7V34TXXOC6aIXlNwGC67cRO6PfwNp6IrcNpYQ5/8zUSa3jOY375/klFdb6JOrV6E5cV69A1BVE42EybaNt97/Yr7rbW9aIB5tvcT9lZx8v/68+R1TzG19eWFdNL88G67obj3xldZ1qgdVTJOs79SDcvrz1QjJcI9Nw3nsaUTGdbvOVKr1PZb1IU21JQwVo54nysesH2tnz/k5ciqz+4odg7q9d9H2Q0FwCPLJwNwqEJVjlauSYt9SeRIWLE9xGCht/twgceFWymm/vAMFbPSWV/nPN67+DYeuuFlTkVXsBuKYOHPF1Sh2Fc51m4oAM5ccG7Y6JHlk+ny9P1cnuxmgYFSiMrjgxnv2p2+9eALYDMJ9vKOvLBwbh30H9IjbMufP5j5Pjds+otrti8l5e0+3LBxPuF5uTywcioAP7W9kvnNLiQ9KsZWxrwwFMHb/dXHdD3ZG0op/mp2IQPueM+vhqIoDl/QmYVvjrZfL258AVfde26uJzM8ksjcbDqmbuHTaW/R4MQBVHo6Ebk5tDqQbJ+jO++ZaSyI7wzYlhRfOPRH7np0NI/3eZr2Pi7RNRvds3BBUS9VeF4uw5ZMoHx2BhPbXcMvPz5Dlcwz7K1Sm353fmCpFl5JKgZPtijPqVGTUV1v4ra1s5nSpid3r5vJ11Ne5/lej8EPR6mUGc2p6AqIyqPXtqVkRUTy+u+jnVa2tHj6lyJXpgQKK6wGXBrXnnZPTOTrySO5NGUtH8x8337vg5nvO12/3vMBS5UzTzBTxabuO1XEBpp7evQi7rnfqH36CAcq2np2bZ+YRO+ti3l7zidctHs93/08HDC2hPn8PpIc4rjptrfJCY9gUqfrqHzmBO8Z38MogV9bX25ijkqGNhZecMeamTy+1Gb1HbugN9zxruVeYLPrQAW8c9ndvHPZ3QDcvmMhUSdP2NbMz4EHu93Mdx37MHHCizRzsUHitfd87DT2XBYorohkh0dy56DX6bt5of0bAoAcCSPC+PDtkge/4lR0BZ/TD5Zd9DXZ4JvxcxR6dCJOQ0Wnoiuw0RhuzjcUAG9ddjfPdq1D+Nu2jw5fvvoREo3tYRbFd2Z245LtWxZItLFwgauXqt7JgzQ/vBuwfQvQIW0rsWePc8mDX7lcVhhs/LtFufv4l733FZc9MJBXrnqY/xxYwtBlP1HjzHGaHU0jDyHMiHV5wzaM6TyALbXcrxsPBIGskDx5Jr+1uoyFTTtxb+KvHC1Xme879uGd2R8xpU1P9latU7L0SxTad3zfotx9OFMNoIstypUqWo+bajfjtZ4P0G/zIrLDw3nohpc5Wr4KT468mt9+W8buqnUZ5/BBYOGls/4U3v9oY+EB5x9MZvY3w+zXQwa8QkRuDnVOHwnIOGnA8aHQHu7Q1T6x95/MaFi/nlvW/86J6Ap0fvR7Gpw4SExulm9fQZcSPB0aPBlTkQ8vPreX1XO9nzBLpIBg9UqwKHwR+5uE/nyT0N/ZMSyMJ/o+6xeZgok2Fh5wqbEvDkBKVds2HDnhEZY2FP6czHT9shcT/3//yx8zljG7xUXMaX4RmZHRbpd1BotQrch8IeSGoTwKGNhMKbzXo6f+rV4UtbHwgIqZ6eQhxD87zVLHZBZHyYahvA/sFCImhgdu/D/fBSillCXD5IgVFhH4gtlyh9pZ99pYuCD/oVXMPEvDE/sZvH4u2eHh5IaFkx4V3NU7wcDlqpBCBd3iJb1IQlVuXwitvPoyd2YW+RPcSvmvrFtsTYxbtLFwQX7l+OCKKTy2bBIAJ0uwCiUYlKhn4UGLJ9Qm54oikHIHW0XBekbmnsFtHsrFb+VDv7vIHcoLuQe7hBSPqR/liUgvEdkmIkki8oKL+9EiMsm4v0JE4hzuvWi4bxORa8yU0yVK2Q3FofJVefnqRwIuQkko0XcWAU6vrBCqBrWkhGq+zV9+7pyA1fVkWs9CRMKBz4CrgFRglYhMV0ptdvB2H3BMKXWeiAwG3gYGiUgrYDDQGqgHzBOR5kqpXLPkdUTl5vGlcXjL4sYX2M8jCCX8+wW3+/itXtCLIkTF9omgfb/t8wfcQV46W0R6Xk9wl5JSZmbPojOQpJRKVkplAROBAmvK6A/kb4A/GegpImK4T1RKZSqldgFJRnwBIeLgAa4yjpz84OIijuMsxfgyJls6XgezKZtaCtXKMjSlNg/TDj8SkZuAXkqp+43rO4AuSqmhDn42Gn5SjeudQBdgBLBcKfWj4f41MFspNbmo9Hw9/OhE6n6OJTifyBaRk0WDI//ycP8X7OcVB5KIMCEnr2TPpWH1cuw9mu4XeRpUK0fqMee46lctR9rxc25Vy0dy/KxtpVh8rYrsOHiaUKBO5Rj2n8xw71ETcKqUi+REevGrDwuWQ7OJrRTNibPZZOXmeRymWWwFdh46Y6JUNsw+/CikNxIUkSEikigiiYcOHfItjsgIjjZu5vR3sNn5TGlzBcsbtTXSsfmtWv7c1stdm1anSrlz123qV6Zjo6r2eyXhqla27zda1a1sd6sUHeGUXnGEhwlt6587G6GaIXedyjHUr+r6zOf2Dc/txd+rtfOXwu0aVKFny1oAREfYiswFDavY4woPEy5qZjtXuWnNCsTXrkilaP+PcDapeW6RQZ3KRR8IU7dKDDGR54p2UXkG6Ni4Kuc76NkxnaY1K9jzW83h2des6Ns2Jde2sem1duXoIv3kP/O4GuWL9FOQGhXOyRMedm6JTVSE8+t98Xk17WXUW8JcrNxpUM2m15Z1Cp9M6JjH/HwDNK/t2QmgW/oAAAj4SURBVJbv5SLD6X5eDSrHFF+OLmhYxV52wwsIWclF2OgCOikfFc4FDV3r5GrjPWxUvbw9DxfGVePKVrXsfvLLfrNY1wtgwsOEFnUquUyjYD1xSbzvm49eeb7533yZuRoqDXD8EquB4ebKT6qIRABVgCMehkUpNQYYA7aehS9CVq5dk44rCp/j3BG40ZcINRqNphRiZs9iFRAvIk1EJArbhPX0An6mA3cZv28C5ivbuNh0YLCxWqoJEA+sNFFWjUaj0RSDaT0LpVSOiAwF5gLhwFil1CYRGQkkKqWmA18DP4hIEnAUm0HB8PcTsBnIAR4N1EoojUaj0RTGtAnuQOPrBLdGo9GUZcrEBLdGo9FoAoM2FhqNRqNxizYWGo1Go3GLNhYajUajcYs2FhqNRqNxS6lZDSUih4DdJYiiJnDYT+L4Ey2Xd2i5vEPL5R2lUa7GSqlYd55KjbEoKSKS6MnysUCj5fIOLZd3aLm8oyzLpYehNBqNRuMWbSw0Go1G4xZtLM4xJtgCFIGWyzu0XN6h5fKOMiuXnrPQaDQajVt0z0Kj0Wg0binzxkJEeonINhFJEpEXApx2QxH5S0Q2i8gmEXnccB8hImkiss746+0Q5kVD1m0ico2JsqWIyAYj/UTDrbqI/CEiO4z/1Qx3EZGPDbnWi0hHk2Rq4aCTdSJyUkSeCIa+RGSsiBw0TnvMd/NaPyJyl+F/h4jc5SotP8j1PxHZaqQ9VUSqGu5xIpLuoLfPHcJ0Mp5/kiG7i+OP/CKb18/O3+9sEXJNcpApRUTWGe4B0VkxdUPwyphSqsz+Yds6fSfQFIgC/gFaBTD9ukBH43clYDvQCtuxss+48N/KkDEaaGLIHm6SbClAzQJu7wAvGL9fAN42fvcGZgMCdAVWBOjZ7QcaB0NfwKXYzsja6Kt+gOpAsvG/mvG7mglyXQ1EGL/fdpArztFfgXhWGrKKIfu1JunMq2dnxjvrSq4C998DXg2kzoqpG4JWxsp6z6IzkKSUSlZKZQETgf6BSlwptU8ptcb4fQrYAtQvJkh/YKJSKlMptQtIwpaHQNEf+M74/R1wvYP798rGcqCqiNQ1WZaewE6lVHEfYpqmL6XUImxnsBRMzxv9XAP8oZQ6qpQ6BvwB9PK3XEqp35VSOcblcmwnTxaJIVtlpdRyZatxvnfIi19lK4ainp3f39ni5DJ6BzcDE4qLw986K6ZuCFoZK+vGoj6w1+E6leIra9MQkTigA7DCcBpqdCfH5nc1Cay8CvhdRFaLyBDDrbZSap/xez+Qf/BvMPQ4GOcXONj6Au/1Ewy93YutBZpPExFZKyILReQSw62+IUug5PLm2QVaZ5cAB5RSOxzcAqqzAnVD0MpYWTcWlkBEKgJTgCeUUieB0UAzoD2wD1s3ONBcrJTqCFwLPCoilzreNFpPQVlKJ7ZjevsBPxtOVtCXE8HUT1GIyMvYTp4cZzjtAxoppToATwHjRaRygMWy3LMrwC04N0oCqjMXdYOdQJexsm4s0oCGDtcNDLeAISKR2ArDOKXULwBKqQNKqVylVB7wJeeGTgImr1Iqzfh/EJhqyHAgf3jJ+H8w0HIZXAusUUodMGQMur4MvNVPwOQTkbuBPsBtRiWDMcRzxPi9GttcQHNDBsehKjPLmbfPLpA6iwAGAJMc5A2YzlzVDQSxjJV1Y7EKiBeRJkZrdTAwPVCJG+OhXwNblFLvO7g7jvffAOSv0pgODBaRaBFpAsRjm1Tzt1wVRKRS/m9sE6QbjfTzV1PcBfzqINedxoqMrsAJh66yGTi19oKtLwe81c9c4GoRqWYMv1xtuPkVEekFPAf0U0qddXCPFZFw43dTbPpJNmQ7KSJdjTJ6p0Ne/C2bt88ukO/slcBWpZR9eClQOiuqbiCYZczX2frS8odtFcF2bC2ElwOc9sXYupHrgXXGX2/gB2CD4T4dqOsQ5mVD1m34YYVKEXI1xbbK5B9gU75egBrAn8AOYB5Q3XAX4DNDrg1Agok6qwAcAao4uAVcX9iM1T4gG9s48H2+6AfbHEKS8XePSXIlYRu3zi9jnxt+bzSe7zpgDdDXIZ4EbBX3TuBTjA94TZDN62fn73fWlVyG+7fAQwX8BkRnFF03BK2M6S+4NRqNRuOWsj4MpdFoNBoP0MZCo9FoNG7RxkKj0Wg0btHGQqPRaDRu0cZCo9FoNG7RxkKjAUQkV5x3tC12N1MReUhE7vRDuikiUtOHcNeIyGti24V0tvsQGk3JiAi2ABqNRUhXSrX31LNS6nP3vkzlEuAv4//iIMuiKQPonoVGUwxGy/8dsZ1TsFJEzjPcR4jIM8bvYWI7d2C9iEw03KqLyDTDbbmItDPca4jI72I7o+ArbB9T5ad1u5HGOhH5Iv9L4QLyDBLb2QrDgA+xbZFxj4gEbOcBTdlEGwuNxka5AsNQgxzunVBKtcX2Ve6HLsK+AHRQSrUDHjLcXgPWGm4vYduyGmA4sFgp1RrbnluNAETkfGAQ0N3o4eQCtxVMSCk1CdsOpBsNmTYYafcrSeY1GnfoYSiNxkZxw1ATHP5/4OL+emCciEwDphluF2PbGgKl1HyjR1EZ20E7Awz3mSJyzPDfE+gErLJtC0Q5zm0SV5Dm2A6xAaigbOcdaDSmoo2FRuMeVcTvfK7DZgT6Ai+LSFsf0hDgO6XUi8V6sh1xWxOIEJHNQF1jWOoxpdTfPqSr0XiEHobSaNwzyOH/MscbIhIGNFRK/QU8D1QBKgJ/YwwjiUgP4LCynUewCLjVcL8W21GXYNsc7iYRqWXcqy4ijQsKopRKAGZiOxntHWwb6bXXhkJjNrpnodHYKGe00POZo5TKXz5bTUTWA5nYtkd3JBz4UUSqYOsdfKzU/7d3hzYIxVAUQO9jJMaCFRCwACNg2YQNQBM0AoUt4pNgfniGBHOObZrU3bRN7huPqtomObz3PfOpld4lOVbVOckpyS1JxhiXqtpkmk64yNSAuk4yNzZ2memDe5VkP7MOP6d1Fr6oqmumuuf7v88C/+QZCoCWmwUALTcLAFrCAoCWsACgJSwAaAkLAFrCAoDWC1f+KWycTJI9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the scores\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "plt.plot(np.arange(len(scores)), scores, label='MADDPG')\n",
    "plt.plot(np.arange(len(scores)), avgs, c='r', label='moving avg')\n",
    "plt.ylabel('Score')\n",
    "plt.xlabel('Episode #')\n",
    "plt.legend(loc='upper left');\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episodes 0000-0010\tMax Reward: 0.000\tMoving Average: 0.000\n",
      "Episodes 0010-0020\tMax Reward: 0.000\tMoving Average: 0.000\n",
      "Episodes 0020-0030\tMax Reward: 0.000\tMoving Average: 0.000\n",
      "Episodes 0030-0040\tMax Reward: 0.100\tMoving Average: 0.003\n",
      "Episodes 0040-0050\tMax Reward: 0.100\tMoving Average: 0.006\n",
      "Episodes 0050-0060\tMax Reward: 0.200\tMoving Average: 0.010\n",
      "Episodes 0060-0070\tMax Reward: 0.200\tMoving Average: 0.013\n",
      "Episodes 0070-0080\tMax Reward: 0.100\tMoving Average: 0.014\n",
      "Episodes 0080-0090\tMax Reward: 0.200\tMoving Average: 0.018\n",
      "Episodes 0090-0100\tMax Reward: 0.200\tMoving Average: 0.019\n",
      "Episodes 0100-0110\tMax Reward: 0.000\tMoving Average: 0.019\n",
      "Episodes 0110-0120\tMax Reward: 0.100\tMoving Average: 0.021\n",
      "Episodes 0120-0130\tMax Reward: 0.100\tMoving Average: 0.023\n",
      "Episodes 0130-0140\tMax Reward: 0.000\tMoving Average: 0.022\n",
      "Episodes 0140-0150\tMax Reward: 0.300\tMoving Average: 0.026\n",
      "Episodes 0150-0160\tMax Reward: 0.200\tMoving Average: 0.028\n",
      "Episodes 0160-0170\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 0170-0180\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 0180-0190\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 0190-0200\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 0200-0210\tMax Reward: 0.200\tMoving Average: 0.029\n",
      "Episodes 0210-0220\tMax Reward: 0.200\tMoving Average: 0.031\n",
      "Episodes 0220-0230\tMax Reward: 0.200\tMoving Average: 0.032\n",
      "Episodes 0230-0240\tMax Reward: 0.100\tMoving Average: 0.035\n",
      "Episodes 0240-0250\tMax Reward: 0.100\tMoving Average: 0.032\n",
      "Episodes 0250-0260\tMax Reward: 0.100\tMoving Average: 0.030\n",
      "Episodes 0260-0270\tMax Reward: 0.100\tMoving Average: 0.029\n",
      "Episodes 0270-0280\tMax Reward: 0.200\tMoving Average: 0.033\n",
      "Episodes 0280-0290\tMax Reward: 0.000\tMoving Average: 0.029\n",
      "Episodes 0290-0300\tMax Reward: 0.100\tMoving Average: 0.030\n",
      "Episodes 0300-0310\tMax Reward: 0.200\tMoving Average: 0.030\n",
      "Episodes 0310-0320\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 0320-0330\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 0330-0340\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 0340-0350\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 0350-0360\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 0360-0370\tMax Reward: 0.300\tMoving Average: 0.030\n",
      "Episodes 0370-0380\tMax Reward: 0.200\tMoving Average: 0.027\n",
      "Episodes 0380-0390\tMax Reward: 0.100\tMoving Average: 0.029\n",
      "Episodes 0390-0400\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 0400-0410\tMax Reward: 0.200\tMoving Average: 0.032\n",
      "Episodes 0410-0420\tMax Reward: 0.100\tMoving Average: 0.033\n",
      "Episodes 0420-0430\tMax Reward: 0.100\tMoving Average: 0.032\n",
      "Episodes 0430-0440\tMax Reward: 0.200\tMoving Average: 0.037\n",
      "Episodes 0440-0450\tMax Reward: 0.100\tMoving Average: 0.036\n",
      "Episodes 0450-0460\tMax Reward: 0.200\tMoving Average: 0.038\n",
      "Episodes 0460-0470\tMax Reward: 0.100\tMoving Average: 0.036\n",
      "Episodes 0470-0480\tMax Reward: 0.200\tMoving Average: 0.038\n",
      "Episodes 0480-0490\tMax Reward: 0.100\tMoving Average: 0.039\n",
      "Episodes 0490-0500\tMax Reward: 0.100\tMoving Average: 0.036\n",
      "Episodes 0500-0510\tMax Reward: 0.100\tMoving Average: 0.033\n",
      "Episodes 0510-0520\tMax Reward: 0.100\tMoving Average: 0.034\n",
      "Episodes 0520-0530\tMax Reward: 0.300\tMoving Average: 0.036\n",
      "Episodes 0530-0540\tMax Reward: 0.200\tMoving Average: 0.034\n",
      "Episodes 0540-0550\tMax Reward: 0.000\tMoving Average: 0.033\n",
      "Episodes 0550-0560\tMax Reward: 0.200\tMoving Average: 0.032\n",
      "Episodes 0560-0570\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 0570-0580\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 0580-0590\tMax Reward: 0.200\tMoving Average: 0.026\n",
      "Episodes 0590-0600\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 0600-0610\tMax Reward: 0.100\tMoving Average: 0.029\n",
      "Episodes 0610-0620\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 0620-0630\tMax Reward: 0.400\tMoving Average: 0.029\n",
      "Episodes 0630-0640\tMax Reward: 0.200\tMoving Average: 0.027\n",
      "Episodes 0640-0650\tMax Reward: 0.100\tMoving Average: 0.029\n",
      "Episodes 0650-0660\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 0660-0670\tMax Reward: 0.000\tMoving Average: 0.026\n",
      "Episodes 0670-0680\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 0680-0690\tMax Reward: 0.100\tMoving Average: 0.021\n",
      "Episodes 0690-0700\tMax Reward: 0.000\tMoving Average: 0.020\n",
      "Episodes 0700-0710\tMax Reward: 0.200\tMoving Average: 0.020\n",
      "Episodes 0710-0720\tMax Reward: 0.100\tMoving Average: 0.020\n",
      "Episodes 0720-0730\tMax Reward: 0.000\tMoving Average: 0.015\n",
      "Episodes 0730-0740\tMax Reward: 0.100\tMoving Average: 0.015\n",
      "Episodes 0740-0750\tMax Reward: 0.100\tMoving Average: 0.014\n",
      "Episodes 0750-0760\tMax Reward: 0.200\tMoving Average: 0.015\n",
      "Episodes 0760-0770\tMax Reward: 0.100\tMoving Average: 0.019\n",
      "Episodes 0770-0780\tMax Reward: 0.200\tMoving Average: 0.023\n",
      "Episodes 0780-0790\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 0790-0800\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 0800-0810\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 0810-0820\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 0820-0830\tMax Reward: 0.100\tMoving Average: 0.032\n",
      "Episodes 0830-0840\tMax Reward: 0.100\tMoving Average: 0.033\n",
      "Episodes 0840-0850\tMax Reward: 0.200\tMoving Average: 0.035\n",
      "Episodes 0850-0860\tMax Reward: 0.000\tMoving Average: 0.032\n",
      "Episodes 0860-0870\tMax Reward: 0.100\tMoving Average: 0.030\n",
      "Episodes 0870-0880\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 0880-0890\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 0890-0900\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 0900-0910\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 0910-0920\tMax Reward: 0.200\tMoving Average: 0.022\n",
      "Episodes 0920-0930\tMax Reward: 0.200\tMoving Average: 0.025\n",
      "Episodes 0930-0940\tMax Reward: 0.100\tMoving Average: 0.023\n",
      "Episodes 0940-0950\tMax Reward: 0.100\tMoving Average: 0.022\n",
      "Episodes 0950-0960\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 0960-0970\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 0970-0980\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 0980-0990\tMax Reward: 0.100\tMoving Average: 0.023\n",
      "Episodes 0990-1000\tMax Reward: 0.100\tMoving Average: 0.022\n",
      "Episodes 1000-1010\tMax Reward: 0.100\tMoving Average: 0.021\n",
      "Episodes 1010-1020\tMax Reward: 0.100\tMoving Average: 0.021\n",
      "Episodes 1020-1030\tMax Reward: 0.200\tMoving Average: 0.020\n",
      "Episodes 1030-1040\tMax Reward: 0.100\tMoving Average: 0.020\n",
      "Episodes 1040-1050\tMax Reward: 0.200\tMoving Average: 0.022\n",
      "Episodes 1050-1060\tMax Reward: 0.100\tMoving Average: 0.022\n",
      "Episodes 1060-1070\tMax Reward: 0.100\tMoving Average: 0.023\n",
      "Episodes 1070-1080\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 1080-1090\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1090-1100\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1100-1110\tMax Reward: 0.100\tMoving Average: 0.029\n",
      "Episodes 1110-1120\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 1120-1130\tMax Reward: 0.200\tMoving Average: 0.036\n",
      "Episodes 1130-1140\tMax Reward: 0.100\tMoving Average: 0.036\n",
      "Episodes 1140-1150\tMax Reward: 0.100\tMoving Average: 0.034\n",
      "Episodes 1150-1160\tMax Reward: 0.100\tMoving Average: 0.034\n",
      "Episodes 1160-1170\tMax Reward: 0.200\tMoving Average: 0.037\n",
      "Episodes 1170-1180\tMax Reward: 0.100\tMoving Average: 0.037\n",
      "Episodes 1180-1190\tMax Reward: 0.200\tMoving Average: 0.038\n",
      "Episodes 1190-1200\tMax Reward: 0.100\tMoving Average: 0.037\n",
      "Episodes 1200-1210\tMax Reward: 0.200\tMoving Average: 0.035\n",
      "Episodes 1210-1220\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 1220-1230\tMax Reward: 0.000\tMoving Average: 0.023\n",
      "Episodes 1230-1240\tMax Reward: 0.100\tMoving Average: 0.023\n",
      "Episodes 1240-1250\tMax Reward: 0.200\tMoving Average: 0.027\n",
      "Episodes 1250-1260\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1260-1270\tMax Reward: 0.200\tMoving Average: 0.024\n",
      "Episodes 1270-1280\tMax Reward: 0.100\tMoving Average: 0.023\n",
      "Episodes 1280-1290\tMax Reward: 0.100\tMoving Average: 0.020\n",
      "Episodes 1290-1300\tMax Reward: 0.100\tMoving Average: 0.021\n",
      "Episodes 1300-1310\tMax Reward: 0.100\tMoving Average: 0.021\n",
      "Episodes 1310-1320\tMax Reward: 0.100\tMoving Average: 0.021\n",
      "Episodes 1320-1330\tMax Reward: 0.100\tMoving Average: 0.023\n",
      "Episodes 1330-1340\tMax Reward: 0.100\tMoving Average: 0.023\n",
      "Episodes 1340-1350\tMax Reward: 0.100\tMoving Average: 0.018\n",
      "Episodes 1350-1360\tMax Reward: 0.200\tMoving Average: 0.018\n",
      "Episodes 1360-1370\tMax Reward: 0.100\tMoving Average: 0.018\n",
      "Episodes 1370-1380\tMax Reward: 0.100\tMoving Average: 0.021\n",
      "Episodes 1380-1390\tMax Reward: 0.100\tMoving Average: 0.022\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episodes 1390-1400\tMax Reward: 0.100\tMoving Average: 0.022\n",
      "Episodes 1400-1410\tMax Reward: 0.100\tMoving Average: 0.021\n",
      "Episodes 1410-1420\tMax Reward: 0.200\tMoving Average: 0.024\n",
      "Episodes 1420-1430\tMax Reward: 0.200\tMoving Average: 0.025\n",
      "Episodes 1430-1440\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1440-1450\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1450-1460\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1460-1470\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1470-1480\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 1480-1490\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 1490-1500\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 1500-1510\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1510-1520\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1520-1530\tMax Reward: 0.200\tMoving Average: 0.025\n",
      "Episodes 1530-1540\tMax Reward: 0.200\tMoving Average: 0.025\n",
      "Episodes 1540-1550\tMax Reward: 0.300\tMoving Average: 0.028\n",
      "Episodes 1550-1560\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1560-1570\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1570-1580\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 1580-1590\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1590-1600\tMax Reward: 0.000\tMoving Average: 0.024\n",
      "Episodes 1600-1610\tMax Reward: 0.300\tMoving Average: 0.027\n",
      "Episodes 1610-1620\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1620-1630\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1630-1640\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1640-1650\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 1650-1660\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 1660-1670\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 1670-1680\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1680-1690\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1690-1700\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1700-1710\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 1710-1720\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 1720-1730\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 1730-1740\tMax Reward: 0.200\tMoving Average: 0.025\n",
      "Episodes 1740-1750\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 1750-1760\tMax Reward: 0.200\tMoving Average: 0.027\n",
      "Episodes 1760-1770\tMax Reward: 0.200\tMoving Average: 0.029\n",
      "Episodes 1770-1780\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1780-1790\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 1790-1800\tMax Reward: 0.200\tMoving Average: 0.028\n",
      "Episodes 1800-1810\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1810-1820\tMax Reward: 0.100\tMoving Average: 0.031\n",
      "Episodes 1820-1830\tMax Reward: 0.000\tMoving Average: 0.029\n",
      "Episodes 1830-1840\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1840-1850\tMax Reward: 0.200\tMoving Average: 0.030\n",
      "Episodes 1850-1860\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1860-1870\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1870-1880\tMax Reward: 0.200\tMoving Average: 0.029\n",
      "Episodes 1880-1890\tMax Reward: 0.100\tMoving Average: 0.032\n",
      "Episodes 1890-1900\tMax Reward: 0.200\tMoving Average: 0.029\n",
      "Episodes 1900-1910\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1910-1920\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1920-1930\tMax Reward: 0.100\tMoving Average: 0.027\n",
      "Episodes 1930-1940\tMax Reward: 0.100\tMoving Average: 0.028\n",
      "Episodes 1940-1950\tMax Reward: 0.100\tMoving Average: 0.026\n",
      "Episodes 1950-1960\tMax Reward: 0.100\tMoving Average: 0.025\n",
      "Episodes 1960-1970\tMax Reward: 0.100\tMoving Average: 0.024\n",
      "Episodes 1970-1980\tMax Reward: 0.100\tMoving Average: 0.022\n",
      "Episodes 1980-1990\tMax Reward: 0.100\tMoving Average: 0.020\n",
      "Episodes 1990-2000\tMax Reward: 0.100\tMoving Average: 0.021\n"
     ]
    }
   ],
   "source": [
    "# run the training loop\n",
    "\n",
    "# GAMMA = 0.99            # discount factor\n",
    "# TAU = 1e-3              # for soft update of target parameters\n",
    "# OU_SIGMA = 0.4          # Ornstein-Uhlenbeck noise parameter, volatility\n",
    "# OU_THETA = 0.2          # Ornstein-Uhlenbeck noise parameter, speed of mean reversion\n",
    "# EPS = 1.0               # explore->exploit noise process added to act step\n",
    "# EPS_DECAY = 4e-3        # decay rate for noise process\n",
    "\n",
    "scores, avgs = maddpg()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xd8FGX+wPHPNyGNFkoiIgESMIg0KaHYPUVEUZoFPFQsP8sJ1tMTT08Qy9m9U7Hg6amIIDbkBBtNQQUBQaQTIEDopBDS2/P7YyfLJtlkN8lOdpN8369XXtl95pmZ78zuzneeeaaIMQallFKqMkH+DkAppVTg02ShlFLKI00WSimlPNJkoZRSyiNNFkoppTzSZKGUUsojTRZKKaU80mShlFLKI00WSimlPGrk7wB8JSoqysTGxvo7DKWUqlPWrFlz1BgT7alevUkWsbGxrF692t9hKKVUnSIiu72pp4ehlFJKeaTJQimllEeaLJRSSnlUb/os3CkoKCA5OZnc3Fx/h9KghIeHExMTQ0hIiL9DUUr5SL1OFsnJyTRr1ozY2FhExN/hNAjGGFJSUkhOTiYuLs7f4SilfMTWw1AiMlREtopIoohMqqTelSJiRCTBpexha7ytInJJdeafm5tL69atNVHUIhGhdevW2ppTqp6xrWUhIsHANOBiIBlYJSLzjDGbytRrBtwDrHQp6waMBboDpwALRaSLMaaoGnFUfyFUteg6V6r+sbNlMQBINMbsNMbkA7OBEW7qPQE8C7juio4AZhtj8owxu4BEa3qqHtmfnsOSLYf9HYZSygt2Jot2wF6X98lWmZOI9AXaG2PmV3Vca/zbRGS1iKw+cuSIb6L2MRHhuuuuc74vLCwkOjqayy+/vFS9kSNHMmjQoFJlU6ZMoV27dvTu3Zv4+HhGjx7Npk0nGmYXXHABp512Gr169aJr165MnDiR9PR05/Dg4GB69+5Njx49uPrqq8nOzgbg0KFD/PnPf6ZTp07069ePM888ky+++MKOxa/U5a8u56b3VtX6fJVSVee3U2dFJAh4CfhrdadhjJlujEkwxiRER3u8Wt0vmjRpwoYNG8jJyQHg+++/p1270nkvPT2dNWvWcOzYMXbu3Flq2H333ce6devYvn07Y8aM4cILL8Q1Mc6cOZP169ezfv16wsLCGDHiROMtIiKCdevWsWHDBkJDQ3nzzTcxxjBy5EjOO+88du7cyZo1a5g9ezbJyck2rgX3UrPya32eSqnqsTNZ7APau7yPscpKNAN6AEtFJAkYBMyzOrk9jVunXHbZZcyf72g8zZo1i2uvvbbU8M8//5wrrriCsWPHMnv27AqnM2bMGIYMGcJHH31UblhoaCjPPfcce/bs4ffffy83/NxzzyUxMZHFixcTGhrKHXfc4RzWsWNH7rrrruounlKqAbDz1NlVQLyIxOHY0I8F/lwy0BhzDIgqeS8iS4EHjDGrRSQH+EhEXsLRwR0P/FqTYB7/30Y27c+oySTK6XZKcyZf0d1jvbFjxzJ16lQuv/xy1q9fz80338yyZcucw2fNmsVjjz1GmzZtuPLKK/n73/9e4bT69u3Lli1b3A4LDg7mjDPOYMuWLZxxxhnO8sLCQr7++muGDh3Kxo0b6du3bxWWUimlbGxZGGMKgYnAt8BmYI4xZqOITBWR4R7G3QjMATYB3wATqnMmVKDo1asXSUlJzJo1i8suu6zUsEOHDrF9+3bOOeccunTpQkhICBs2bKhwWsaYSuflOjwnJ4fevXuTkJBAhw4duOWWW8rVnzBhAmeccQb9+/ev4lIppRoSWy/KM8YsABaUKXusgroXlHn/FPCUr2LxpgVgp+HDh/PAAw+wdOlSUlJSnOVz5swhLS3NeQFbRkYGs2bN4qmn3C/62rVrSUhIcDusqKiIP/74g9NPPx040Wfhqnv37nz22WfO99OmTePo0aMVTlMppUDvDVVrbr75ZiZPnkzPnj1Llc+aNYtvvvmGpKQkkpKSnB3O7nz22Wd899135fo8wHFrk4cffpj27dvTq1evCuO48MILyc3N5Y033nCWlZwlpZRSFanXt/sIJDExMdx9992lypKSkti9e3epU2bj4uKIjIxk5UrHNYovv/wyH374IVlZWfTo0YPFixfjeubXuHHjCAsLIy8vj8GDB/Pll19WGoeIMHfuXO677z6ee+45oqOjadKkCc8++6wPl1YpVd+Ip2PgdUVCQoIp+/CjzZs3Ow/JqNrlzbqPneQ4QyzpmWG1EZJSyg0RWWOM8XgcWg9DKaWU8kiThVJKKY80WSillPJIk4VSSimPNFkopZTySJOFUkopjzRZ1EFvvvkmH3zwgb/DUEo1IHpRXh3kesdYpZSqDdqysFFSUhJdu3blxhtvpEuXLowbN46FCxdy9tlnEx8fz6+/Om6km5qaysiRI+nVqxeDBg1i/fr1FBcXExsbW+phRvHx8Rw6dIgpU6bwwgsvAI4HID300EMMGDCALl26OO9mm52dzTXXXEO3bt0YNWoUAwcOpOxFiwBTp06lf//+9OjRg9tuuw1jDFu2bGHAgBMPJkxKSnLepmTBggV07dqVfv36cffdd5d7iJNSqn5qOC2Le++FMjfVq7HeveFf/6q0SmJiIp988gnvvvsu/fv356OPPmL58uXMmzePp59+mrlz5zJ58mT69OnD3LlzWbx4MTfccAPr1q1jxIgRfPHFF9x0002sXLmSjh070qZNm3LzKCws5Ndff2XBggU8/vjjLFy4kNdff52WLVuyadMmNmzYQO/evd3GN3HiRB57zHFvx+uvv56vvvqKK664gvz8fHbt2kVcXBwff/wxY8aMITc3l9tvv50ff/yRuLg4t/eoUkrVT9qysFlcXBw9e/YkKCiI7t27c9FFFyEi9OzZk6SkJACWL1/O9ddfDzhu9JeSkkJGRgZjxozh448/BmD27NmMGTPG7TxGjx4NQL9+/UpNc+zYsQD06NGjwpsLLlmyhIEDB9KzZ08WL17Mxo0bAbjmmmuc8y5JFlu2bKFTp07OO+RqslCq4Wg4LQsPLQC7hIWFOV8HBQU53wcFBVFYWFjpuGeeeSaJiYkcOXKEuXPn8uijj1Y6j+DgYI/TdJWbm8udd97J6tWrad++PVOmTCE3NxdwPJXv6quvZvTo0YgI8fHx5W53rpRqOGxtWYjIUBHZKiKJIjLJzfA7ROQPEVknIstFpJtVHisiOVb5OhF50844/e3cc89l5syZACxdupSoqCiaN2+OiDBq1Cjuv/9+Tj/9dFq3bu31NM8++2zmzJkDwKZNm/jjjz/K1SlJDFFRUWRmZvLpp586h3Xu3Jng4GCeeOIJZ4vmtNNOY+fOnc7WS0nLQylV/9nWshCRYGAacDGQDKwSkXnGmE0u1T4yxrxp1R8OvAQMtYbtMMa4P9Bez0yZMoWbb76ZXr160bhxY95//33nsDFjxtC/f3/ee++9Kk3zzjvvZPz48XTr1o2uXbvSvXt3IiMjS9Vp0aIFt956Kz169ODkk08u97S8MWPG8OCDD7Jr1y7A8TCl119/naFDh9KkSRN9up5SDYhttygXkTOBKcaYS6z3DwMYY/5ZQf1rgRuMMZeKSCzwlTGmh7fz01uUl1ZUVERBQQHh4eHs2LGDwYMHs3XrVkJDQ2s03czMTJo2bYoxhgkTJhAfH899991Xrp7eolypusHbW5Tb2WfRDtjr8j4ZGFi2kohMAO4HQoELXQbFichaIAN41BizzMZY653s7Gz+9Kc/UVBQgDGG119/vcaJAuDtt9/m/fffJz8/nz59+nD77bf7IFqlVKDzewe3MWYaME1E/gw8CowHDgAdjDEpItIPmCsi3Y0xGa7jishtwG0AHTp0qOXIA1uzZs3cXldRU/fdd5/bloRSqn6zs4N7H9De5X2MVVaR2cBIAGNMnjEmxXq9BtgBdCk7gjFmujEmwRiT4Pqo0TJ1qhe9qjZd50rVP3Ymi1VAvIjEiUgoMBaY51pBROJd3g4Dtlvl0VYHOSLSCYgHdlY1gPDwcFJSUnTjVYuMMaSkpBAeHu7vUJRSPmTbYShjTKGITAS+BYKBd40xG0VkKrDaGDMPmCgig4ECIA3HISiA84CpIlIAFAN3GGNSqxpDTEwMycnJHDlyxBeLpLwUHh5OTEyMv8NQSvmQrX0WxpgFwIIyZY+5vL6ngvE+Az6r6fxDQkKcVxsrpZSqPr3dh1JKKY80WSillPJIk4VSSimPNFkopZTySJOFUkopjzRZKKWU8kiThVJKKY80WSillPJIk4VSSimPNFkopZTySJOFUkopjzRZKKWU8kiThVJKKY80WSillPJIk4VSSimPNFkov9MnGSoV+GxNFiIyVES2ikiiiExyM/wOEflDRNaJyHIR6eYy7GFrvK0icomdcSqllKqcbcnCeob2NOBSoBtwrWsysHxkjOlpjOkNPAe8ZI3bDcczu7sDQ4HXS57JrZRSqvbZ2bIYACQaY3YaY/KB2cAI1wrGmAyXt02AkuMRI4DZxpg8Y8wuINGanlJKKT+wM1m0A/a6vE+2ykoRkQkisgNHy+LuqowbyIwxxE6azz+/3uzXOJ74ahOxk+bbNv2lWw8TO2k+Ww5meK6sVA39vjed2EnzWbkzpcI6qVn5xE6az4xfkmotrobA7x3cxphpxpjOwEPAo1UZV0RuE5HVIrL6yJEj9gRYTcVWG+ntH3f6NY53lu+ydfrfbjwEwJrdadWehvZvK28tTzwKwNJtFf/e96XlAPDx6r0V1lFVZ2ey2Ae0d3kfY5VVZDYwsirjGmOmG2MSjDEJ0dHRNQzXHrodVErVB3Ymi1VAvIjEiUgojg7rea4VRCTe5e0wYLv1eh4wVkTCRCQOiAd+tTFWpZRSlWhk14SNMYUiMhH4FggG3jXGbBSRqcBqY8w8YKKIDAYKgDRgvDXuRhGZA2wCCoEJxpgiu2JVSilVOduSBYAxZgGwoEzZYy6v76lk3KeAp+yLzl56oZlSqj7xewe3UppWlQp8miyUUkp5pMnCJrq3rJSqTzRZKKWU8kiThU20f1spVZ9oslBKKeWRJgubBUoLI5BP5Q3k2JRSDposbGK0i1spVY9oslBKKeWRJgubBNqRlUCLRylVt2iyUEop5ZEmC+V32uhRKvBpslBKKeWRJosGQvfelVI1ocnCJtqhrJSqTzRZKKWU8sjWZCEiQ0Vkq4gkisgkN8PvF5FNIrJeRBaJSEeXYUUiss76m1d2XFV/aCtMqcBn25PyRCQYmAZcDCQDq0RknjFmk0u1tUCCMSZbRP4CPAeMsYblGGN62xWf3QLtCm7HLTXE32EopeooO1sWA4BEY8xOY0w+MBsY4VrBGLPEGJNtvV0BxNgYj1JKqWqyM1m0A/a6vE+2yipyC/C1y/twEVktIitEZKS7EUTkNqvO6iNHjtQ8YqWUUm7ZdhiqKkTkOiABON+luKMxZp+IdAIWi8gfxpgdruMZY6YD0wESEhIC6riPHodXStUndrYs9gHtXd7HWGWliMhg4BFguDEmr6TcGLPP+r8TWAr0sTHWei+Qc1eg9e8opcqzM1msAuJFJE5EQoGxQKmzmkSkD/AWjkRx2KW8pYiEWa+jgLMB147xgKebP6VUfWLbYShjTKGITAS+BYKBd40xG0VkKrDaGDMPeB5oCnwiIgB7jDHDgdOBt0SkGEdCe6bMWVRKKaVqka19FsaYBcCCMmWPubweXMF4PwM97YxNKaWU9/QKbpsE2qNCAywcpVQdo8lC+Z0mMqUCnyYLm+j2TylVn2iyaCD09FSlVE1oslBKKeWRJgub6HF4pVR9oslCKaWUR14nCxE5R0Rusl5Hi0icfWHVAwHWstCWjlKqJrxKFiIyGXgIeNgqCgE+tCsopZRSgcXblsUoYDiQBWCM2Q80sysopZRSgcXbZJFvHJckGwARaWJfSPWDnqqqlKpPvE0Wc0TkLaCFiNwKLATeti8s1ZBof4pSgc+rGwkaY14QkYuBDOA04DFjzPe2RlbH6QZQKVWfeEwWIhIMLDTG/AnQBKGUUg2Qx8NQxpgioFhEImshHqWUUgHI2z6LTOAPEXlHRF4p+fM0kogMFZGtIpIoIpPcDL9fRDaJyHoRWSQiHV2GjReR7dbfeO8XKTAE2lEoPSymlKoJbx9+9Ln15zXr8NU04GIgGVglIvPKPPFuLZBgjMkWkb8AzwFjRKQVMBlIwLHdXWONm1aVGFTdoGeOKRX4vO3gft96jnYXq2irMabAw2gDgERjzE4AEZkNjMDlWdrGmCUu9VcA11mvLwG+N8akWuN+DwwFZnkTbyAItIcfKaVUTXh7BfcFwHYcLYXXgW0icp6H0doBe13eJ1tlFbkF+Lqa49qmsKiYZ77ewuHjufxzwWaOZTty5Mb9x/jXwm38c8FmCoqKK53GG0t3sONIZrny9Ox8HvtyA5f+exnPfbOFVxZtrzTJ/GfZTv73+37n+9SsfJ75egtFxZ4T09MLNpNfeCLO5LRs/rVwm3N+RcWGgU8vZP76A2TnF/LU/E3kFhQBMHPlbn7bU3mj7un5m8nOL6SgqNixnnJK70t8u/Eg05Ykuh13w74M3v85yeMyAGTmFfL0gs3kFRaVG7Y3NZt/LzyxDmf8ksTve9M9TrO42PDCt1s5nJHrse6xnAKeXrCZ57/dwr70HK9iLmt9cjof/JLE2j1pfLhid5XG/Wr9fpZsPVyu3BjDS99vY1VSaqnPFWD6jzvYfuh4qfr/WbaTzQcyAMf38J9fb6bQw/e4rBkrdrOukvVbEtO+9Bxe/G4rh7xYv+6kZ+fz8Od/eL2+9x/LseZfrdmVsmz7Eb5ct8/5fl96Di9/v43vNx3imw0HATiUkcuL322t8Lf7665U5qw6sTkzxvDqou3sTslylr3/cxJ/JB+recA28vYw1IvAEGPMVgAR6YJjL7+fL4IQketwHHI6v4rj3QbcBtChQwdfhFLOws2HefOHHbz5ww7AsbF45speDHtlubNO17bNGNUnpsJpPPvNFv6zbCdr/nFxqfInvtrMZ78lAzh/uKP6tKN9q8Zup/Pk/M0AXHHGKQBMnreR//2+n34dW3JxtzaVLseMFbvp2rYZ4wY6uoVun7GGjfszuLzXKZx6UlN+2HaYQxl5TPjoN+6+KJ63l+0iqmkYt5/fmUe+2ABA0jPDKpx+Vn4Rby7dQWxUE976cScZuYX8c/SJx6jfPmMNABP+dGq5ca956xcAxp8VW+kyALyyaDvTf9xJTMsIbjizdP1b3l/FtkOZjOrTjg6tG/OPLzd6jBtgzZ40XluSyO/J6cy4ZWCldZ/9ZgsfrdwDwPLEFL6ccLbHmMsa/tpPpd5fN6hjBTXLm/jRWqD8Mm0+cJxXFm3nlUXbAZyfa1Gx4ekFW/j3wu1snDrUWb/ku5T0zDDn97Bnu0gu73WK17H8Y27l34uyMa3ZncZHtw7yevolvt5wkFm/7qFpWDAtGod6rH+XtY5KflM1cf07vwIwordjX/WOGWv4Y9+JjXrSM8O4d/Y6ftmZwoVdT6JPh5blplHy/b6mf3sADmXk8eL32/h87T6WPHAB4Pgtl0wvUHnbwR1SkigAjDHbcNwfqjL7gPYu72OsslJEZDDwCDDcGJNXlXGNMdONMQnGmITo6GivFqSqyu6157vZ+3K3Q1Z2H6NkL93TtKoiz5qmNy2LsvVynPE4ygqKTgwraSkVejndEvlFxjmOp9ZWdZW0jgqLyseWW+AYVtU+kJL1klfoOWbX1lmem8/UX4rL7dWWfp9TSawl30Nvv0fVjcmb9etOSYunsmVwVbI8dhwIdvc7zrVaud6uvpL14m5agczbZLFaRP4jIhdYf28Dqz2MswqIF5E4q79jLDDPtYKI9AHewpEoXNvW3wJDRKSliLQEhlhlyufE3wEoW+jnagdpwKvV28NQfwEmAHdb75fh6LuokDGmUEQm4tjIBwPvGmM2ishUYLUxZh7wPNAU+EQcn8IeY8xwY0yqiDyBI+EATC3p7K4rAr5/u0x8vohXz2pSqv7yNlk0Av5tjHkJnKfFhnkayRizAFhQpuwxl9eDKxn3XeBdL+NT1dSQ95TqM/1c7SENuMXm7WGoRUCEy/sIHDcTVEoFMD2FW/mKt8ki3BjjPPfTeu3+lB0FBP4hmfLR+eQ4lFKqnvI2WWSJSN+SNyKSAFTvJHMVUBpuo7p+K/lc3eVvbW1UX0M+vOdtn8W9ODqhS64IawuMsSekeiLAf49lNxi+6eBWdYHmClUdlbYsRKS/iJxsjFkFdAU+BgqAb4BdtRCfspk05F2lekw/V+Vrng5DvQXkW6/PBP6O45YfacB0G+NSSvmAu1aENixUdXg6DBXscn3DGGC6MeYz4DMRWWdvaHVboP8gK7/et5rT1OMbdYJ+Tqo6PLUsgkWkJKFcBCx2GeZtf4cKYPXpYEWgn4FWm050cBvr/wm6lqqvIR/e87TBnwX8ICJHcZz9tAxARE4FAvsWiUoppXym0mRhjHlKRBbhOPvpO3Oi/RoE3GV3cHVZoLf0y8bnk7Oh/LzMDfnqWk9c14y/PydVN3k8lGSMWeGmbJs94aja1oBb1fVayefqvoNbs0V1NeSfi7cX5akqCvQfZNn4fBFvYC+xKqEtC1UdmiwauPp06CbQE3RtKvu56prxjYbcEtdkoZRSyiNNFjYJ9Ka+dnA3LNrBrWpKk0UD15Cb1fWZdnDboyH/XmxNFiIyVES2ikiiiExyM/w8EflNRApF5Koyw4pEZJ31N6/suIEu0H+O5VoWvphmwC+1Am1ZqOqx7Sps62l604CLgWRglYjMM8Zscqm2B7gReMDNJHKMMb3tik/VP5qsKqZXcPtGQz7UaectOwYAicaYnQAiMhsYATiThTEmyRpWbGMcSjVYmkCVr9h5GKodsNflfbJV5q1wEVktIitEZKRvQ7NfXbtZmy/i9fciN+S9Pk9Kd3DXre+mCgyBfDPAjsaYfSLSCVgsIn8YY3a4VhCR24DbADp06OCPGOu8+tRhp3vRJ5Tt4NbDUL5Rn34vVWVny2If0N7lfYxV5hVjzD7r/05gKdDHTZ3pxpgEY0xCdHR0zaL1sUDfedO9y4ZLP3pVHXYmi1VAvIjEiUgoMBbw6qwmEWkpImHW6yjgbFz6OpTv6Iajfqr0c9XPvNoacMPCvmRhjCkEJgLfApuBOcaYjSIyVUSGg/OxrcnA1cBbIrLRGv10YLWI/A4sAZ4pcxaVUsoLmheUr9jaZ2GMWQAsKFP2mMvrVTgOT5Ud72egp52xNXTlnpTnkyu4/btp0g7uipXq4NYUoqpBr+BW9YZuBCtWqoNbV1P1NeAebk0WNgn0H2T5K7j1FuX1kbvWnn5Oqjo0WfiAvw+/BBQ/rooaH4ZqYB9jXdtJDoSfmS9XWSAsT1Vosmjg6toXVnnH3fUVJ4bph66qTpOFD4ibXbRAP35e7kl5vrpFeR3bWy2lLsfuQekO7rqprrWEPKlry6PJooELpKRW0z3eQFoWfyv/2FyX19abOratCgh1bQPvS5osbBLoLX1bHn6kG+uAo8+zCHx15bCgJgsfqCsfdq3wsCrsXFfawV017g6fBrJA+JlpB7dqsALpC1vTWHSP+QTnunTbw12bkaiy7LggtjZosvAB9x3cga3cF9YX09QO7joh0L+bFaljDSGP6tryaLJo4Gp7w1HZXlRd3YgFokrvI6gd3NXmi0N3dfWwtSYLmwT6F6J8B7dewV0fuTs0p4frAktd+TQ0WfhAoCeGWuWpg9vGWWsHd9XUtcMggfAz0w5uFTBq+wsUSIlOr7PwnZJ16fbUWV1NfmVHq742aLLwgbrYwV02Qu3gpm7HXgWB/910r661hDypa8ujySLA1Pbece13cFc8R1/GUlf21uxSeQe3Y6g+/6Pq6toG3pdsTRYiMlREtopIoohMcjP8PBH5TUQKReSqMsPGi8h262+8nXHaIdC3VeXi80m8gbHQgb7ua5P7GwnWehiqEnXl47AtWYhIMDANuBToBlwrIt3KVNsD3Ah8VGbcVsBkYCAwAJgsIi3tirU6XPcwGvpebCnawV131LG95ED4mfmyNRYIy1MVdj5WdQCQaIzZCSAis4ERgPNZ2saYJGtYcZlxLwG+N8akWsO/B4YCs2yMt5Qjx/MwxpCUklWu/JsNB8t90Ov2pnPqSU05ejyP0EZBFBSVXqSs/CK2HMxg15EsMnILOKtzFOnZ+eXmm5ZdwL70o+TkFxHdLIzM3EIGdmrNtkPHnXWS07KJCAnm9+R0APamZrNx/zGMgZZNQskvLKbYGI4czys17TW70xjS7WT2pWeTkuWY94H0XEKDg9iw/5iz3sFjuQD87/f9dGvb3Fm+dk8aLRuHEhkRQnpOAY2ChG82HCg1Xv4pjuX+bU8ahzJyyckvIqegyFln19EsNu4/xhkxLcot+6qkVM7s1JpdR7M4OTKcxqGNOJZdQEZuAe1bNcYYw+IthwFIzyngaGYexcYQEhTEzqNZHM10LO+G/cfYfyzHOd2fEo/SumkoR4/nc+BYDpERIbRqEkq/ji35bU8ay7cfddbdk5KNCOQVFpFbUIwxJ163axnBvrQT0y02ho37j1FcDPuP5RDTMoLO0U3ZuP8Y/Tq2Iq+wiP3puew4nEmPdpHkFRaRnX9iXZRYsuUwwUFCu5YRZOQUUFRsiIwIYceRTOKimrLzSCbNI0Kcy1fymUeEBrNxfwbFxYbdZb6n6dn5HDmeR3CQY+NmDKRk5tGqSSh7U7MByC8sZn1yOjuPZAKw/VAmu1OyCG0UxOGMPOKim5B0NIt2LSIIEmH/sRw6tm7C5gMZ9G5/4vPbeSSTdXvTiY1qQpPQRrRrGcGKHSmEhZTeFz2amcfWg8fJyi+kXYsINu3PoH2rxuxPzyG6WRghwY5YjxzPJ6ppKFn5RURGhJBXeOK3lJFb4Ph+7U5jze40+nZowZ7UbA4ey6Vj6yacHBnurGsMHM7I5UhmHhEhwew6mkWn6KYcysjllMgIUrPzKTYGAfIKi2nTPJyComJnOogIDXZO61h2AXvTstl2+MTvECC3oIjjuYUAfLV+P62ahLLt0HE6Rzfl1JNb3h8EAAAgAElEQVSakuvy3d9yMIMmoY3YuD/DuSzfbzpEbOvGzjqbD2TQKEg4qVk4x3IKSE7LpkPrxhw4lkvbyHC2H87kgi7RZOQUkpHr+A3kFRbTo10kTcNsfUo2YtdesXVYaagx5v+s99cDA40xE93UfQ/4yhjzqfX+ASDcGPOk9f4fQI4x5oUy490G3AbQoUOHfrt37/ZZ/LGT5ntdt0e75mzYl1GqrHN0E3YcyapgjKpp3yqCvak5nivWA3+/rCtPL9jCJd3b8Nb1CYx6/SfW7kkn6ZlhvPz9Nv69aLvP5nXDmR354JcT35kBca34dVeqT6b9xri+zF23j283HvLJ9KprzaOD6ffkQuf7kvVbU75cV966blAHPlyxp1TZ9YM6MmPFic8w8alLOfWRr2stpvO7RPPDtiNuh22aegndHvvW5/N87qpe/O3T9aXKzo2PYsYtA6s1PRFZY4xJ8FSvTndwG2OmG2MSjDEJ0dHRfoujbKIAfJYogAaTKACWJ6YAODeya/ekO4ct3XrYp/P6fpN9G/IN+4/5PVG4s3KnbzbwtZ0oKvLdpoOl3hfV8rGdihIFQF5B2QMmvuH6myixzKV1bBc7k8U+oL3L+xirzO5xVR1WW2dLOebl4wkGoAawiKUE0mcaQKH4hJ3JYhUQLyJxIhIKjAXmeTnut8AQEWlpdWwPscqUsk99+3XXM4GUCPzLPyvCtmRhjCkEJuLYyG8G5hhjNorIVBEZDiAi/UUkGbgaeEtENlrjpgJP4Eg4q4CpJZ3dquHy9ck7Dfmc+foqkD5Tu/qD/ZU0be0+N8YsABaUKXvM5fUqHIeY3I37LvCunfGpwOPPvcf6eLuQ+rQ37k0iCKTltSsUfy1jne7gVvVPsR9/7YG0oVF1n13fJ3/t1GiyUAGlsmShHdxVVx9bS5UJpM+0vq17TRYqoPj3MJQKZIGUCPxJD0MpReUbbLs7uPW2LXVfQO3N23YYyj80WaiA4s8NdgBtZnynHi2UdnBb09WWhVJQHEA/dhX4yt7YL5C+PoGUuHxBk4UKKLV5BXf5eds8A2W7QDqUaNchMT0bSilqt2VR7vGWtTfrWlOflsntI2Lr1RJ6SQ9DKeXfDm5tWtR9gfQJ2nedhX9oslABxa+HoXw5rQDZagVKHL6gHdzWdP20kJosVECp6KI8Y0xAbQhUYCj35LoA+o4EUv+JL2iyUAGlot+XMfYfn/blb7t+bSbqjkDqw9DDUErZyL8d3L6beaDsVAbSxrOmtIPbQa+zUIqKm+4G3/9Iym5ofNuyaHgbsUAQKEnaTtqyUIrKDkP5vs+iXMvCpz3cPpxWDdSnjadXHdz2h+E12w5DaQe3UpV0cGPDXWc9vFeBxZttZCB1Kte31qWtyUJEhorIVhFJFJFJboaHicjH1vCVIhJrlceKSI6IrLP+3rQzThU4Kvp5GeP7DYGd25X6tZmoOwJpvde3Dm7bnpQnIsHANOBiIBlYJSLzjDGbXKrdAqQZY04VkbHAs8AYa9gOY0xvu+JTgcm/Dz/yZQd3YGy2AiMK3wikR6b6VT3s4B4AJBpjdhpj8oHZwIgydUYA71uvPwUuEtGvRINWUZ+FLb8Q+351AZIrGpxAWu+2XZRXD+8N1Q7Y6/I+2SpzW8cYUwgcA1pbw+JEZK2I/CAi57qbgYjcJiKrRWT1kSNHfBu98ouKL8oLrA1BXREoLRy7lL/rbOAsb31b94HawX0A6GCM6QPcD3wkIs3LVjLGTDfGJBhjEqKjo2s9SOV7lf28fL0hsPNsqPq1mQgMXn0+AbTi9XkW3tsHtHd5H2OVua0jIo2ASCDFGJNnjEkBMMasAXYAXWyMVQWI2mxZlJ1Xfbwor74rd62Mn+Jwx75TZ+2Zrid2JotVQLyIxIlIKDAWmFemzjxgvPX6KmCxMcaISLTVQY6IdALigZ02xqoCRHGx/+ZdHy/Kq09JS3szHepdn4XVBzER+BbYDMwxxmwUkakiMtyq9g7QWkQScRxuKjm99jxgvYisw9HxfYcxJtWuWJV7YQV5NMnLBqBRUSEvzH+ZLz74K52P7vUwpu8ZG34idl5n4c+N9J2/zGHCzx/7LwA/CqzkaNPDj/y0jLadOgtgjFkALChT9pjL61zgajfjfQZ8Zmdsyg1jaJ6XRUZ4U7oe3sU3/70LgPuH3cdL8192Vntz7tOMuv5FMsMa0zw3k78veZfmuZk8feEtJEe2qVEIlR+GCqgtgV81z80kKzSCoqBgZ1l4QS5X/7GQv/34AQAD924g6Nayjfn6LVBadBBoiavmbE0WdZW/N0od0/Zz+8rPeHPgVexp2bbW5vvS/JcYvXEJh5q2ok1mqku5I1EUI+Q1CiU+ZS8b/nUNXe//lMu3LGPs+u8AODkzhdHXv0hYQR55IWHViqHC231gwxXcLhOMzDlOUGH1Yq5N3Q/tYP579zjff9bjQpKbtyEtohlTFk0vVfe8pLVwans63/IGO1u3w0igns/iHe+u4LY/Dm/Zd+qsf2iycMMvXzhjiMk4THBxET9Mvw2AzNDGPH3hLc4qwcVFNM3L5lhEM5/MslFRIVFZ6RwPa0zzvCxGb1wCQIG1t/pZ9z9xpVUG0Olv82h/7BDL3vo/AJ787g2u2rDIObzv/q0sfPsOTk1N5vtTB3DrlY9RVZU9z8LXv5JiY2iWl0Vs6n7mzvgrmRFN6XfnBxQG1/xnYcsOhzGlEgXAlRsWl6v2aY+LeOjSu7ljxac8uGwGi975C5ujY7n05td8H5Oflb1LcQDlinrXwa3Jwg1/fBYTfpnDg8tmlCq7dNvPnJSVyn/7Def3U07j8e/f5Lp1XzPCOgTUIuc4Izb9wL/PvpaUJi0qnPbJGUd5dPF/eO788exp2RYxxfTZt5XPZz5Yru74qx/nh079nO8fH3w7ozYu4eNeF4MIe1ucTJ+7ZrL21XHORPHiOeNoVFzMPT/P4tTUZAAuTvyVCT9/zLSzxpSbRwkxxURnpnG4WWtnWW3corzN8aM8vPS/ROVncU7iamd5ZHYGiS+MZG6383nwsnspCA6p9jx8vRhiirl3+Szn+8tufIV5799LI+M4I2B3i5OZMGISG04+1Vln2lljuHf3j4Ts2c3pR5J4+ptXWdT1CR9HVnvcdXAHUkuiRHBxER3SDxKUmgLAgL0bOGv377TOPsbb/UedOFpgDA8sm0FkbiZPXvh/5DUK9XIO/lloTRa1LCb9IEeatKRpfo5zAz9qw2JnotjZ8hR+7ngGqRHNuXzLMi7b8hMjN/3ATx17cfbu9QB8OeOvpabZZ/8Wrrjx327n1zQvmxVv3AhA+2OHuOnqKdz5yxz+b/WXbuv/1PGMUu8zwpvyfr8rSpWlNY7k7ise5O6fZvFlt/N59exrAfi5Yy/SI5px10+zuXzrch5cNoMf4/ryR9v4UuNHZaXR5ngK9/40i4sTV3L3FQ8y7/TzaHv8KG3y4GhoVLktgy8OQ4UV5hNz7BBffnA/TfNznOWzew1h7Smncd/KOZycdoiRm35g5KYfuGPkw3xz2tk1nGv1BRUX0aQgl0IJZvPLVznLJwx/iE1tOnHq3zz3R+xb+gvv3zqZyYve5s+/f8vq9Hs8jlOXVdaiCykqILSwgKywxhXWOW/nGoZu+4Upg28nv1HFOwthhfk0z80itXFzZ79ReEEu4YX5/GPxfxwtvreh8y1vMOejE7fFG7VxCf0mfsh9y2dy+ZZlxGQ4LibumHaAvw+dSLO8LLa37kBhcCOCiotoXJBHZiXx1ibx9/F5X0lISDCrV6/2XNELhUXFnPrI1zWeTvPcTCJzM/nLik8ZtmUZkXlZzmH5QY3oc/dHDNuynOe+eQWAa8c+zS8de5WaxoM/vM+EFZ94nNeSTv148k//R36jEPZGtnFubO//cQZ3/1LxmTHxD3xBREEeN6/+khl9hlXaQqmKvsmbnS2XMdf+k5UdegKOM6y2vnRlpeM+d94NfNrjIn6ddgOxk+YD8PvkIYx4bTlJKdnViiekqIDtL4xyvv+kx2D+c+F1tDq8j1+sBBkX1YT2v/7IB59MdtYrkiCCTTHfdDmTGX2G8VOsd7cru35QR2as2F2tWEvi/ebdiXRO3UcxQpCVKh8Zcicz+1zm9XQW//V8LnzxB25cPY8pi6aTFNuVC8a8QOusdJrlZbEv8qQataCqqmX2McCx4zL+t6+YPmA0eyJPJic0vFS9pnnZPLz0XT7rfhG/xZzOyRlHGXp+Nz5dkcTkhdMJMkVEFOSxv20cTw0cw8NL/suadqfz6FsPce5Ly4jOTGX45h8ZuHcDmaERTBt0DYve+QsAfe+aSWpEcxCh/94NjF3/Hd/FD2Lc2q8d/TzAfcPu54seFxJcXERc6j52t2xLQXAIjYoKmfblM1yyfQUA8087mwkjHya8IJc1r15Hk4LcGq+jpBZtuW7ME7z9+ZOcfiSp1E5LRH4uZ/XqwKIth8uP98ywas1PRNYYYxI81tNkUV5BUTHxPkgWv752PSdlpXlV982BV/LMBTe5HXbJtp+ZtPS//BDXj5fOvY6E5E383LEXXY7uoXFBLrNn/b3cOKff9yk5oeG8NvcZLt+6nGmDri6VdP591rV80mtwjc9eqnSZvniKodt+AeDhSyZy/q41/GnHKsKKCr2bwJAhXBN1IcfDGvPx1Gu4YtYmdlcjWVywYzVvzn2a8MJ8Z1n8A18Q1jiCzLwTscS2buxMRm988TSXbvu51HQONW3FWX/5L0VBwbTNOELj/Fx2tI4BEWKOHeKRxe/wa/vu9N+7kdQrx/BocecqxwpwdtI63v78CRoX5DnLchuFMviW10lucXKVprXor+dz0Ys/gDF8MvMheh9KpMc9s1nz6jia5ueQH9SI+V3P4YO+l7O+bXyps6t8JSI/l9fmPctFO1ZVWOfasU+xs1U7+u7bwhWbf+Qyl3W/NK4fF+xaA8B/EkZU2CoGKGxzMsOGPcq3706sNKZFnfuz+aQ4Jv4yp8I6g/7yHk9+N43BO1aREdaEa8c+xZl71vPokndL1VvQ5axS8QKMG/Mk0wrW0+Jzx/RvumoyP8b1ZcfzJ26P9/y51/NJz8EIhpWv31hpvDN7D+Vg09b8dflMlp17BdefdXu5OposvOTLZJFfWEyXR6uXLIKLi+hydDf7mp/E+n+PdZZfNe5Z4o/u5dsuZ5IXHML89+4hNv0AANdfM5VlcX2rHe85u9by4Zx/VDh8Zfse/HnsU3Q7tJPdLdvSe/9WfozrWytXOS1961bncpbYG9mGc2//D1HZ6YQVFnDT6i9plZPBtEHXMGTfem5fOoMWuZmlxik6vRt/uvE19qR6nywu2LGa8b/9jz/tdGxovjrtHJ47fzz5wSEcbB5F49BgsvOLnPU7tm7sTEZhhfmM+f1brlv7NWmNm7MtqgPXr3WcBX7D1Y+Xan2sbXsafQ5sLTf/qRfeytzuF5DaONJZFp2ZypGmrcrVDSkqoNeB7Xw282/OskWd+9MsL4u84FBuvnpytVoAC+8/j8Ev/QjAkG2/MP2Lpyqt/+aA0Y6dFuu70SklmeTINuQ3CqF1Vjotco4DkNTqFLeJJTLnOLkhYYQV5hNSVMhVGxby8NL3ytU70rgF0dnpVV6eEnsj2/DyOX8udUq3Oy+f/Wfu++kjAPY1i6bd8YrvITdh+EPMP/1ctj8/gpDiogrrgWPnbllsH2Z+/Gip8puumszm6DgONo/iuxExZF05hp2t2/HXYfcDEH9kNyM2/8DK9j1K/eYj8nPpmH6AlMaRvPfJFHa1PIXv4wfS+8A2blrzv3Lz73vXTOf3Krwgl8KgRiQ+X/Y+rd7RZFEDeYVFnPboN1UaJyorjTN3r+fxhW/RKifDWf6vs6/lP/1HlTvuGFJUwBtfPM3qmO68OeiqspOrso5p+0mLaE5+cCO2vHRievlBjbjlqsdqlIxqIjozlS9mPEBMhqPZPP7qx1kW25viCvZgm4U34niuY28/6fwguOzEIZc77nyNxRHtaJ2dTlhhPkmtyt6XsrSkZy93vp44/G98dfp5pYZHhASTU3Bio9ChVeMKk1Gr7GP89uq4Sue3o1UMOSFhdEpNLtUqKJQgZ0c0ODYoSzr3J6wgj7Hrv+PxhW+Vm9aV455jTUy3Sufnje/vO4+LX3Yki4j83FJ9H3eMfJgJv8yh56EdpcYZfMvrjuErP3OexLC9dXviU05cjPnUBTfz9sDRzve3rvyckZuW0v1wxTdaeOCye9nQpjNN87NZHdPdWT5p6X+5Y+WJy6q+jR9Ey5wM/nbpPcSm7ee9Tx8H4INHXqPnu6+yNbojky69G4CO5DD4129Y0bEXG0/qxKbfXqXxQsep3LeNeoTvupzJqUf3EHPsED/F9ubF+S8TWlRA+/RDZIQ34bbRjxKZm0lOozDnIdhOKcm8NP8leh/YBsA/Lr6DYVuWE52VTufUZBZ0OYs7Rzla8x3T9jvPXux1z2wywps6l2P+3ecw7JXlFa4Pb4QX5DI48Vd6HEzkrD3r2R7VwXkG3IvnjOPDPpex9tVxfNPlTIZu/dnD1NzTZFED3iaLRkWFDE5cSWhRAa/874VSw34/OZ7MsAjuHPGwz0519VajokIG7N1Am8xUvux2foUb5toSXpDLCwv+zXfxg5jX7fxK6zYLa8Rx69BQ0jPDuOymV+l9YCtPfzutXN3trdtz8S2vu20hXbL1Z96a+zRwYqNRLq6QIHILTmzEK0sWAK2z0lnz2nUAvDVgNO8mDOe8Xb8xbt3XHGwWxR0j/+6MZWJH4YE7Kz4skBrRvNRORYkpF93GwviBPjs8+N195zHEShYANzQ9Tsqqdext0Yb1bR23WwsrzKd9+kHO37mGfyx5x+tpvzlgNOfs/p2orDROzqz4Bgv/N/ofLIwfWOm0wgry6HkokdXtupX7PONS99E0L5szRl3Ehyv2lBrWsnEIadkFzveL/no+T/3lec5NWsvjg8sfqrFDu2OHyQoNJz2i9L1Ov7rrHC5/tWbJwp2tL19FWL6bvpFqbsu9TRZ6NpQb3qzz83auKXUoAuD1QVfxZbcL2Boda09gXioMbsTPXnbE1obckHAmjnjIq7plV/2mNp3YEt2RSXddwf7b76brvm3OYfEpe2mVk+FsjgcXF9HjYCLnJq3lgWUfAjD8hpecG8Vy8yp3jn7lH3xKkxb0umc27TIOs/mkTgB80msIn/QaUr5um/bEPvQVCckbKSaI4qAg2hxP4eX5L9K4IK9Uovgxtg//TRjOLx16khsSXm5aNVF2GffFdGZRZumdl7xGoSRGdSAxqgPjf/uKDscOkdsolPuH3U9Sy1Noe/wIQcawNaoje1q25YnvXuf6tQu449fPS00nNaI5Y679J3es/JSWOcd5/vwbaJ6bxcr2PTzGmRcSVqq14WqX1YI8w+3QsstrWHzqABafOsCL2r6xL/KkWpsXwOTH3qfJou+ciT0/qBE3XT2FmTbPV5NFNXRIO+BMFEUSxG2jH2FvZBu2+TlJ1AfuLsorDgqm4PwLuGXiG5y+ain7m0czOHEl9y+fyW+vjmNut/P5oM/lXJy4kr+s/NQ53uhxz1eYKKB6tyjPCG9a6lBDxRwTK7sBHNSxF88v+Ber23VjZYcebI2OrcL59VVX1dtfXHndC5yaspdVMd2cFyduatOpVJ3HLr6DvOCQUh3N69rGM/IGR//BXy8vfWq3r7j7fMpel1Mb1+l4y66DNvvbxvLjgFG8M2CU58o+pMnCjYo+5JCiAhrn5/Lj9FsBSh27VL5R4RXc1v+SwxlpEc24c8UnhBfmO6+LcDX0plfZclJclebl07vOVjCtjPCm3D76UfcDbVDVu/geadqSI01bVlrHSBBPXnQrT174f3ROSWZHVPtK69up7GF0fz6Wtyy77lPlr64DTRZeapqXzYZ/XeN8f7hJS00UNvD2d3CgeTS97/6IIdtXcPX6hZy7ex0/d+jFl93OZ2X7Hh47vyGwbg1RJ4nUaqLw5uS9AMoV9Y4mCzdK7REYw50rPnHeyRPgWFgTzvrLf/0QWf1X0W/d3V1nc0PCmdftAuZ1u6B687JxyxIoG61AugtrbQiU9Q6BFYsvaLJww/VD7nokqVSi6D/hA7fnySvfqGgDbsfzLMrdhM6Hv+5A2UiX78SvX8ouT2AdhrJpuvXwSXmIyFAR2SoiiSIyyc3wMBH52Bq+UkRiXYY9bJVvFZFL7IyzrJLPIri4iKe/ddyp86VzxtH5wS81Udiswg5KGx6r6mYWvptWgGyzysYRSBvTqnIXup3PUa8pu1qu9e5JedZjUacBlwLdgGtFpOxVRrcAacaYU4GXgWetcbvheAxrd2Ao8HrJY1ZrgzGGNseP8t6cyfTd77gy952EEbbcCkGVVlkHt90/Et8+VjUw2NmJHwgCu4PbpunWw5bFACDRGLPTGJMPzAbKXo8+Anjfev0pcJGIiFU+2xiTZ4zZBSRa06s1n858iHN3rwNg2Ph/VXqnSuU7AfRbVwHGqw5u+8NosOzss2gHuD6sORkoexmns44xplBEjgGtrfIVZcb1fHpLNRxLPkhaQtmrew2xxw5xsGkrbrhmql4/4ScXv3TidNix01dwKCOvkto1dzCj5ncMLfHpmmSfTasmJs76rdT7H7ZVfG+kQPfRyj3lyrLyS9/DaeJHv5Wr4y93z1pry3R/3pFiy3Q9qdPPWRSR20RktYisPnKkej8CCWlEasfOZf5O5X+nn8e4sU95lSgGn+64gjOmZYSz7JxTozyO1zSsfK5uHl6+zHVavWIiiW4WRodWvmnphDaq+lfAXYwlEjq6P0e/Z7tIopp6vvhsSLc2hIcEMTCuFfFtmtIrJpLWTUI5vW0zLu524hYYrZqE0ijIsavZonH1brF9aY/Sd3A9v0t0lcY/IyayfFn7Fs5p9+3gm9u9uzq5eThndnI8LKqy70CnqCaIONY7OG6jEtooiAu7en+1cbsWju9zaHCQ87sa29oxz/Nc1lWT0NKHZ+OimgDQuknFn7enz+ykZicecVuyDJd0P5l+Zb5fl3Q/8Z0IDhJ6lflMWlUSg7daNA4hqumJeFxbOGe0b0F0s/KP4z2jfQt6xUQ64+3dvvx3wd3vv4Tr/Mq6pHubcuv2moSYCuv7ip0ti32A60nYMVaZuzrJItIIiARSvBwXY8x0YDo47g1VnSCbt4mi78qF5cr7AleUr66UUg2SnS2LVUC8iMSJSCiODuuyj/aaB4y3Xl8FLDaOHqt5wFjrbKk4IB741cZYlVJKVcK2loXVBzER+BYIBt41xmwUkanAamPMPOAdYIaIJAKpOBIKVr05wCagEJhgjKn8BvNKKaVso7coV0qpBszbW5TX6Q5upZRStUOThVJKKY80WSillPJIk4VSSimPNFkopZTyqN6cDSUiR4DdNZhEFHDUR+H4ksZVNRpX1WhcVVMf4+pojPF4+4J6kyxqSkRWe3P6WG3TuKpG46oajatqGnJcehhKKaWUR5oslFJKeaTJ4oTp/g6gAhpX1WhcVaNxVU2DjUv7LJRSSnmkLQullFIeNfhkISJDRWSriCSKyKRannd7EVkiIptEZKOI3GOVTxGRfSKyzvq7zGWch61Yt4rIJTbGliQif1jzX22VtRKR70Vku/W/pVUuIvKKFdd6EelrU0ynuayTdSKSISL3+mN9ici7InJYRDa4lFV5/YjIeKv+dhEZ725ePojreRHZYs37CxFpYZXHikiOy3p702Wcftbnn2jF7sVDTasVW5U/O1//ZiuI62OXmJJEZJ1VXivrrJJtg/++Y8aYBvuH49bpO4BOQCjwO9CtFuffFuhrvW4GbAO6AVOAB9zU72bFGAbEWbEH2xRbEhBVpuw5YJL1ehLwrPX6MuBrQIBBwMpa+uwOAh39sb6A83A8I2tDddcP0ArYaf1vab1uaUNcQ4BG1utnXeKKda1XZjq/WrGKFfulNq2zKn12dvxm3cVVZviLwGO1uc4q2Tb47TvW0FsWA4BEY8xOY0w+MBsYUVszN8YcMMb8Zr0+Dmym8meNjwBmG2PyjDG7gEQcy1BbRgDvW6/fB0a6lH9gHFYALUSkrc2xXATsMMZUdiGmbevLGPMjjmewlJ1fVdbPJcD3xphUY0wa8D0w1NdxGWO+M8YUWm9X4HjyZIWs2JobY1YYxxbnA5dl8Wlslajos/P5b7ayuKzWwTXArMqm4et1Vsm2wW/fsYaeLNoBe13eJ1P5xto2IhIL9AFWWkUTrebkuyVNTWo3XgN8JyJrROQ2q6yNMeaA9fogUPIAZH+sx7GU/gH7e31B1dePP9bbzTj2QEvEichaEflBRM61ytpZsdRWXFX57Gp7nZ0LHDLGbHcpq9V1Vmbb4LfvWENPFgFBRJoCnwH3GmMygDeAzkBv4ACOZnBtO8cY0xe4FJggIue5DrT2nvxyKp04HtM7HPjEKgqE9VWKP9dPRUTkERxPnpxpFR0AOhhj+gD3Ax+JSPNaDivgPrsyrqX0TkmtrjM32wan2v6ONfRksQ9o7/I+xiqrNSISguPLMNMY8zmAMeaQMabIGFMMvM2JQye1Fq8xZp/1/zDwhRXDoZLDS9b/w7Udl+VS4DdjzCErRr+vL0tV10+txSciNwKXA+OsjQzWIZ4U6/UaHH0BXawYXA9V2fk9q+pnV5vrrBEwGvjYJd5aW2futg348TvW0JPFKiBeROKsvdWxwLzamrl1PPQdYLMx5iWXctfj/aOAkrM05gFjRSRMROKAeBydar6Oq4mINCt5jaODdIM1/5KzKcYDX7rEdYN1RsYg4JhLU9kOpfb2/L2+XFR1/XwLDBGRltbhlyFWmU+JyFDgb8BwY0y2S3m0iARbrzvhWD87rdgyRGSQ9R29wWVZfB1bVT+72vzNDga2GGOch5dqa51VtG3An9+x6vbW15c/HGcRbMOxh/BILc/7HBzNyMfMalgAAAM+SURBVPXAOuvvMmAG8IdVPg9o6zLOI1asW/HBGSoVxNUJx1kmvwMbS9YL0BpYBGwHFgKtrHIBpllx/QEk2LjOmgApQKRLWa2vLxzJ6gBQgOM48C3VWT84+hASrb+bbIorEcdx65Lv2JtW3Sutz3cd8Btwhct0EnBsuHcAr2FdwGtDbFX+7Hz9m3UXl1X+HnBHmbq1ss6oeNvgt++YXsGtlFLKo4Z+GEoppZQXNFkopZTySJOFUkopjzRZKKWU8kiThVJKKY80WSgFiEiRlL6jbaV3MxWRO0TkBh/MN0lEoqox3iUi8rg47kL6tecxlKqZRv4OQKkAkWOM6e1tZWPMm55r2epcYIn1f7mfY1ENgLYslKqEtef/nDieU/CriJxqlU8RkQes13eL47kD60VktlXWSkTmWmUrRKSXVd5aRL4TxzMK/oPjYqqSeV1nzWOdiLxVcqVwmXjGiOPZCncD/8Jxi4ybRKTW7jygGiZNFko5RJQ5DDXGZdgxY0xPHFfl/svNuJOAPsaYXsAdVtnjwFqr7O84blkNMBlYbozpjuOeWx0AROR0YAxwttXCKQLGlZ2RMeZjHHcg3WDF9Ic17+E1WXilPNHDUEo5VHYYapbL/5fdDF8PzBSRucBcq+wcHLeGwBiz2GpRNMfxoJ3RVvl8EUmz6l8E9ANWOW4LRAQnbhJXVhccD7EBaGIczztQylaaLJTyzFTwusQwHEngCuAREelZjXkI8L4x5uFKKzkecRsFNBKRTUBb67DUXcaYZdWYr1Je0cNQSnk2xuX/L64DRCQIaG+MWQI8BEQCTYFlWIeRROQC4KhxPI/gR+DPVvmlOB51CY6bw10lIidZw1qJSMeygRhjEoD5OJ6M9hyOG+n11kSh7KYtC6UcIqw99BLfGGNKTp9tKSLrgTwct0d3FQx8KCKROFoHrxhj0kVkCvCuNV42J24r/TgwS0Q2Aj8DewCMMZtE5FEcTycMwnEH1AmAu8fG9sXRwX0n8JKb4Ur5nN51VqlKiEgSjts9H/V3LEr5kx6GUkop5ZG2LJRSSnmkLQullFIeabJQSinlkSYLpZRSHmmyUEop5ZEmC6WUUh5pslBKKeXR/wMsit57bPSSZAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the scores\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "plt.plot(np.arange(len(scores)), scores, label='MADDPG')\n",
    "plt.plot(np.arange(len(scores)), avgs, c='r', label='moving avg')\n",
    "plt.ylabel('Score')\n",
    "plt.xlabel('Episode #')\n",
    "plt.legend(loc='upper left');\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "drlnd",
   "language": "python",
   "name": "drlnd"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}