Reinforcement_Learning/Actor_Critic_Method/Actor_Critic_Method_KHW.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "minimal-recipe",
   "metadata": {},
   "source": [
    "# Actor Critic Method with CartPole"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dirty-roommate",
   "metadata": {},
   "source": [
    "## Reinforcement Learning"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "outdoor-airline",
   "metadata": {},
   "source": [
    "<img src='../img/RL01.png' width='600'>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "entertaining-powell",
   "metadata": {},
   "source": [
    "- policy : $\\pi(a|s)=P(a|s), \\forall s, \\forall a$\n",
    "\n",
    "- value function : $v_{\\pi}(s)=\\sum P(z)R(z)=\\sum_{a\\in\\mathbb{A}(s)}P(a|s)(r+v_{\\pi}(s')), \\forall s \\in \\mathbb{S}$\n",
    "\n",
    "- reward : $R(z)=r_{t+1}+\\gamma r_{t+2} + \\gamma^2r_{t+3}+\\cdots=\\sum_{k=1}^{\\infty}\\gamma^{k-1}r_{t+k}$\n",
    "\n",
    "- Q-Value : $Q_{\\pi}(s,a)=E_{\\pi}[R_{t+1}+\\gamma R_{t+2}+\\gamma^2R_{t+3}+\\cdots|S_t=s,A_t=a]$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eleven-presentation",
   "metadata": {},
   "source": [
    "### Actor Critic"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bacterial-picnic",
   "metadata": {},
   "source": [
    "Agent가 action을 취하고 생성된 state를 두 가지로 출력"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "compound-potato",
   "metadata": {},
   "source": [
    "1. Actor : 상태가 주어졌을 때 행동을 결정\n",
    "\n",
    "2. Critic : 상태의 가치를 평가"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ongoing-gregory",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "prerequisite-breed",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T01:12:20.787195Z",
     "start_time": "2021-03-21T01:12:11.680005Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading package lists... Done\n",
      "Building dependency tree       \n",
      "Reading state information... Done\n",
      "The following additional packages will be installed:\n",
      "  freeglut3 libglu1-mesa libpython-stdlib libpython2.7-minimal\n",
      "  libpython2.7-stdlib libxi6 python python-minimal python2.7 python2.7-minimal\n",
      "Suggested packages:\n",
      "  python-doc python-tk python-numpy libgle3 python2.7-doc binfmt-support\n",
      "The following NEW packages will be installed:\n",
      "  freeglut3 libglu1-mesa libpython-stdlib libpython2.7-minimal\n",
      "  libpython2.7-stdlib libxi6 python python-minimal python-opengl python2.7\n",
      "  python2.7-minimal\n",
      "0 upgraded, 11 newly installed, 0 to remove and 46 not upgraded.\n",
      "Need to get 4734 kB of archives.\n",
      "After this operation, 23.1 MB of additional disk space will be used.\n",
      "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython2.7-minimal amd64 2.7.17-1~18.04ubuntu1.6 [335 kB]\n",
      "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python2.7-minimal amd64 2.7.17-1~18.04ubuntu1.6 [1291 kB]\n",
      "Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-minimal amd64 2.7.15~rc1-1 [28.1 kB]\n",
      "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython2.7-stdlib amd64 2.7.17-1~18.04ubuntu1.6 [1917 kB]\n",
      "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python2.7 amd64 2.7.17-1~18.04ubuntu1.6 [248 kB]\n",
      "Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 libpython-stdlib amd64 2.7.15~rc1-1 [7620 B]\n",
      "Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python amd64 2.7.15~rc1-1 [140 kB]\n",
      "Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 libxi6 amd64 2:1.7.9-1 [29.2 kB]\n",
      "Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 freeglut3 amd64 2.8.1-3 [73.6 kB]\n",
      "Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libglu1-mesa amd64 9.0.0-2.1build1 [168 kB]\n",
      "Get:11 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-opengl all 3.1.0+dfsg-1 [496 kB]\n",
      "Fetched 4734 kB in 3s (1616 kB/s)      \n",
      "debconf: delaying package configuration, since apt-utils is not installed\n",
      "Selecting previously unselected package libpython2.7-minimal:amd64.\n",
      "(Reading database ... 17815 files and directories currently installed.)\n",
      "Preparing to unpack .../0-libpython2.7-minimal_2.7.17-1~18.04ubuntu1.6_amd64.deb ...\n",
      "Unpacking libpython2.7-minimal:amd64 (2.7.17-1~18.04ubuntu1.6) ...\n",
      "Selecting previously unselected package python2.7-minimal.\n",
      "Preparing to unpack .../1-python2.7-minimal_2.7.17-1~18.04ubuntu1.6_amd64.deb ...\n",
      "Unpacking python2.7-minimal (2.7.17-1~18.04ubuntu1.6) ...\n",
      "Selecting previously unselected package python-minimal.\n",
      "Preparing to unpack .../2-python-minimal_2.7.15~rc1-1_amd64.deb ...\n",
      "Unpacking python-minimal (2.7.15~rc1-1) ...\n",
      "Selecting previously unselected package libpython2.7-stdlib:amd64.\n",
      "Preparing to unpack .../3-libpython2.7-stdlib_2.7.17-1~18.04ubuntu1.6_amd64.deb ...\n",
      "Unpacking libpython2.7-stdlib:amd64 (2.7.17-1~18.04ubuntu1.6) ...\n",
      "Selecting previously unselected package python2.7.\n",
      "Preparing to unpack .../4-python2.7_2.7.17-1~18.04ubuntu1.6_amd64.deb ...\n",
      "Unpacking python2.7 (2.7.17-1~18.04ubuntu1.6) ...\n",
      "Selecting previously unselected package libpython-stdlib:amd64.\n",
      "Preparing to unpack .../5-libpython-stdlib_2.7.15~rc1-1_amd64.deb ...\n",
      "Unpacking libpython-stdlib:amd64 (2.7.15~rc1-1) ...\n",
      "Setting up libpython2.7-minimal:amd64 (2.7.17-1~18.04ubuntu1.6) ...\n",
      "Setting up python2.7-minimal (2.7.17-1~18.04ubuntu1.6) ...\n",
      "Linking and byte-compiling packages for runtime python2.7...\n",
      "Setting up python-minimal (2.7.15~rc1-1) ...\n",
      "Selecting previously unselected package python.\n",
      "(Reading database ... 18564 files and directories currently installed.)\n",
      "Preparing to unpack .../python_2.7.15~rc1-1_amd64.deb ...\n",
      "Unpacking python (2.7.15~rc1-1) ...\n",
      "Selecting previously unselected package libxi6:amd64.\n",
      "Preparing to unpack .../libxi6_2%3a1.7.9-1_amd64.deb ...\n",
      "Unpacking libxi6:amd64 (2:1.7.9-1) ...\n",
      "Selecting previously unselected package freeglut3:amd64.\n",
      "Preparing to unpack .../freeglut3_2.8.1-3_amd64.deb ...\n",
      "Unpacking freeglut3:amd64 (2.8.1-3) ...\n",
      "Selecting previously unselected package libglu1-mesa:amd64.\n",
      "Preparing to unpack .../libglu1-mesa_9.0.0-2.1build1_amd64.deb ...\n",
      "Unpacking libglu1-mesa:amd64 (9.0.0-2.1build1) ...\n",
      "Selecting previously unselected package python-opengl.\n",
      "Preparing to unpack .../python-opengl_3.1.0+dfsg-1_all.deb ...\n",
      "Unpacking python-opengl (3.1.0+dfsg-1) ...\n",
      "Setting up libxi6:amd64 (2:1.7.9-1) ...\n",
      "Setting up freeglut3:amd64 (2.8.1-3) ...\n",
      "Setting up libglu1-mesa:amd64 (9.0.0-2.1build1) ...\n",
      "Setting up libpython2.7-stdlib:amd64 (2.7.17-1~18.04ubuntu1.6) ...\n",
      "Setting up python2.7 (2.7.17-1~18.04ubuntu1.6) ...\n",
      "Setting up libpython-stdlib:amd64 (2.7.15~rc1-1) ...\n",
      "Setting up python (2.7.15~rc1-1) ...\n",
      "Setting up python-opengl (3.1.0+dfsg-1) ...\n",
      "Processing triggers for libc-bin (2.27-3ubuntu1.2) ...\n",
      "Processing triggers for mime-support (3.60ubuntu1) ...\n"
     ]
    }
   ],
   "source": [
    "!pip install gym\n",
    "!apt-get install python-opengl -y\n",
    "!apt install xvfb -y\n",
    "!pip install pyvirtualdisplay\n",
    "!pip install piglet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "advance-provider",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T19:22:42.840824Z",
     "start_time": "2021-03-21T19:22:42.044868Z"
    }
   },
   "outputs": [],
   "source": [
    "import gym\n",
    "from gym import wrappers\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import random\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "from torch.distributions import Categorical\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "smooth-minimum",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T19:22:43.914645Z",
     "start_time": "2021-03-21T19:22:43.862689Z"
    }
   },
   "outputs": [],
   "source": [
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "necessary-treatment",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T19:22:44.849411Z",
     "start_time": "2021-03-21T19:22:44.235028Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bash: ../xvfb: No such file or directory\n",
      "env: DISPLAY=:1\n"
     ]
    }
   ],
   "source": [
    "# 게임 이미지를 그리는 가상 디스플레이 생성\n",
    "# Colab이나 Jupyter 같은 환경에서만 필요. 로컬은 필요 없음\n",
    "import os\n",
    "if type(os.environ.get(\"DISPLAY\")) is not str or len(os.environ.get(\"DISPLAY\"))==0:\n",
    "    !bash ../xvfb start\n",
    "    %env DISPLAY=:1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "british-brook",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T19:22:45.416723Z",
     "start_time": "2021-03-21T19:22:45.400555Z"
    }
   },
   "outputs": [],
   "source": [
    "# Configuration parameters for the whole setup\n",
    "seed = 42\n",
    "gamma = 0.99  # 과거 보상에 대한 감가율\n",
    "alpha = 0.01  # learning rate\n",
    "max_steps_per_episode = 10000\n",
    "env = gym.make(\"CartPole-v0\")  # Create the environment\n",
    "env.seed(seed)\n",
    "eps = np.finfo(np.float32).eps.item()  # float으로 표현 가능한 가장 작은 값"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "assured-confusion",
   "metadata": {},
   "source": [
    "- CartPole"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "anticipated-glory",
   "metadata": {},
   "source": [
    "<img src='../img/RL03.png' width='300'>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hazardous-digit",
   "metadata": {},
   "source": [
    "## Actor Critic 네트워크 구현"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "renewable-place",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T00:53:39.087592Z",
     "start_time": "2021-03-21T00:53:39.073567Z"
    }
   },
   "source": [
    "1. Actor : state를 입력으로 받아 action에 대한 확률 값을 반환, $\\pi(s,a)$\n",
    "\n",
    "2. critic : state를 입력으로 받아 향후 총 보상의 추정치를 반환, $V(s)$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "forty-mouth",
   "metadata": {},
   "source": [
    "<img src='../img/RL02.png' width='300'>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "applied-wound",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T19:22:47.606289Z",
     "start_time": "2021-03-21T19:22:47.596921Z"
    }
   },
   "outputs": [],
   "source": [
    "num_inputs = 4  # state의 크기\n",
    "num_actions = 2 # action -> 좌, 우로 이동\n",
    "num_hidden = 128  # hidden layer node 수\n",
    "\n",
    "class ActorCritic(nn.Module):\n",
    "    def __init__(self, num_inputs, num_actions, num_hidden, gamma=0.99, alpha=0.01):\n",
    "        super(ActorCritic, self).__init__()\n",
    "        self.num_inputs = num_inputs\n",
    "        self.num_actions = num_actions\n",
    "        self.gamma = gamma\n",
    "        self.alpha = alpha\n",
    "\n",
    "        self.common_layer = nn.Sequential(\n",
    "            nn.Linear(self.num_inputs, num_hidden),\n",
    "            nn.ReLU()\n",
    "        )\n",
    "        self.action_layer = nn.Linear(num_hidden, num_actions)\n",
    "        self.critic_layer = nn.Linear(num_hidden, 1)\n",
    "\n",
    "    def forward(self, input):\n",
    "        common = self.common_layer(input)\n",
    "        action_prob = F.softmax(self.action_layer(common), dim=-1)\n",
    "        critic_value = self.critic_layer(common)\n",
    "\n",
    "        return action_prob, critic_value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "checked-requirement",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T19:22:48.482767Z",
     "start_time": "2021-03-21T19:22:48.476442Z"
    }
   },
   "outputs": [],
   "source": [
    "model = ActorCritic(num_inputs, num_actions, num_hidden, gamma)\n",
    "optimizer = optim.Adam(model.parameters(), lr=alpha)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "processed-broadcast",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T00:54:38.508265Z",
     "start_time": "2021-03-21T00:54:38.499869Z"
    }
   },
   "source": [
    "- Running reward : $\\mathrm{Running \\; reward}_{t}=0.05*\\mathrm{episode \\; reward}_{t}+(1-0.05)*\\mathrm{Running \\;reward}_{t-1}$\n",
    "\n",
    "- Sum of reward : $R(z)=r_{t+1}+\\gamma r_{t+2} + \\gamma^2r_{t+3}+\\cdots=\\sum_{k=1}^{\\infty}\\gamma^{k-1}r_{t+k}$\n",
    "\n",
    "- actor loss : $\\pi(s_t,a_t)$에 $R_{t}$와 $V(s_t)$의 차이를 곱한 값\n",
    "\n",
    "- critic loss : $R_{t}$와 $V(s_t)$의 잔차 이용, Huber loss(평균 제곱 오차 함수와 절대 값 함수의 조합) 사용\n",
    "\n",
    "Huber loss\n",
    "\n",
    "일정한 범위(\n",
    "δ)를 정해서 그 안에 있으면 오차를 제곱하고, 그 밖에 있으면 오차의 절대값을 구하는 것\n",
    "\n",
    "$$L_{\\delta}(e)=\\left\\{\\begin{matrix}\n",
    " \\frac{1}{2}e^2&\\textrm{for}\\left |  e\\right | \\leq \\delta\\\\ \n",
    "\\delta(\\left | e \\right |- \\frac{1}{2}\\delta), & \\mathrm{otherwise}\n",
    "\\end{matrix}\\right.$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "special-yellow",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T19:24:40.839128Z",
     "start_time": "2021-03-21T19:24:28.607528Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "running reward: 76.09 at episode 10\n",
      "running reward: 118.85 at episode 20\n",
      "running reward: 144.52 at episode 30\n",
      "running reward: 153.17 at episode 40\n",
      "running reward: 154.83 at episode 50\n",
      "running reward: 160.32 at episode 60\n",
      "running reward: 168.81 at episode 70\n",
      "running reward: 181.33 at episode 80\n",
      "running reward: 188.82 at episode 90\n",
      "running reward: 193.31 at episode 100\n",
      "Solved at episode 106!\n"
     ]
    }
   ],
   "source": [
    "action_probs_history = []\n",
    "critic_value_history = []\n",
    "rewards_history = []\n",
    "running_reward = 0\n",
    "episode_count = 0\n",
    "\n",
    "while True:\n",
    "    state = env.reset()\n",
    "    episode_reward = 0\n",
    "\n",
    "    for timestep in range(1, max_steps_per_episode):\n",
    "        state = torch.FloatTensor(state)\n",
    "        action_probs, critic_value = model(state)\n",
    "        critic_value_history.append(critic_value)\n",
    "        m = Categorical(action_probs)\n",
    "        action = m.sample()\n",
    "        action_probs_history.append(m.log_prob(action))\n",
    "\n",
    "        # env.step(action)'s ouput: state, reward, done, info\n",
    "        state, reward, done, _ = env.step(action.item())  \n",
    "        rewards_history.append(reward)\n",
    "        episode_reward += reward\n",
    "\n",
    "        if done:\n",
    "            break\n",
    "    \n",
    "    running_reward = 0.05 * episode_reward + (1 - 0.05) * running_reward\n",
    "    \n",
    "    returns = []\n",
    "    discounted_sum = 0\n",
    "    for r in rewards_history[::-1]:\n",
    "        discounted_sum = r + gamma * discounted_sum\n",
    "        returns.insert(0, discounted_sum)\n",
    "    \n",
    "    # Normalize\n",
    "    returns_amount = len(returns)\n",
    "    returns = torch.tensor(returns)\n",
    "    returns = (returns - returns.mean()) / (returns.std() + eps)\n",
    "    returns.resize_(returns_amount, 1)\n",
    "\n",
    "    # Calculating loss values to update our network\n",
    "    history = zip(action_probs_history, critic_value_history, returns)\n",
    "    actor_losses = []\n",
    "    critic_losses = []\n",
    "    for log_prob, value, ret in history:\n",
    "        diff = ret - value.item()\n",
    "        actor_losses.append(-log_prob * diff)  # actor loss\n",
    "        critic_losses.append(\n",
    "            F.smooth_l1_loss(value, ret.clone().detach())\n",
    "        )\n",
    "\n",
    "    optimizer.zero_grad()\n",
    "\n",
    "    # Backpropagation\n",
    "    loss_value = torch.stack(actor_losses).sum() + torch.stack(critic_losses).sum()\n",
    "    loss_value.backward()\n",
    "    optimizer.step()\n",
    "\n",
    "    # Clear the loss and reward history\n",
    "    action_probs_history.clear()\n",
    "    critic_value_history.clear()\n",
    "    rewards_history.clear()\n",
    "\n",
    "    # Log details\n",
    "    episode_count += 1\n",
    "    if episode_count % 10 == 0:\n",
    "        template = \"running reward: {:.2f} at episode {}\"\n",
    "        print(template.format(running_reward, episode_count))\n",
    "\n",
    "    if running_reward > 195:  # Condition to consider the task solved\n",
    "        print(\"Solved at episode {}!\".format(episode_count))\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "micro-madonna",
   "metadata": {},
   "source": [
    "----"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fourth-proposal",
   "metadata": {},
   "source": [
    "## Visualization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "material-milwaukee",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T19:27:28.896606Z",
     "start_time": "2021-03-21T19:27:28.543797Z"
    }
   },
   "outputs": [],
   "source": [
    "# Render an episode and save as a GIF file\n",
    "\n",
    "from IPython import display as ipythondisplay\n",
    "from PIL import Image\n",
    "from pyvirtualdisplay import Display\n",
    "\n",
    "\n",
    "display = Display(visible=0, size=(400, 300))\n",
    "display.start()\n",
    "\n",
    "\n",
    "def render_episode(env, model, max_steps): \n",
    "    screen = env.render(mode='rgb_array')\n",
    "    im = Image.fromarray(screen)\n",
    "\n",
    "    images = [im]\n",
    "  \n",
    "    state = env.reset()\n",
    "    for i in range(1, max_steps + 1):\n",
    "        state = torch.FloatTensor(state)\n",
    "        action_probs, _ = model(state)\n",
    "        m = Categorical(action_probs)\n",
    "        action = m.sample()\n",
    "\n",
    "        state, _, done, _ = env.step(action.item())\n",
    "        state = torch.FloatTensor(state)\n",
    "\n",
    "        # Render screen every 10 steps\n",
    "        if i % 10 == 0:\n",
    "            screen = env.render(mode='rgb_array')\n",
    "            images.append(Image.fromarray(screen))\n",
    "  \n",
    "        if done:\n",
    "            break\n",
    "  \n",
    "    return images\n",
    "\n",
    "\n",
    "# Save GIF image\n",
    "images = render_episode(env, model, max_steps_per_episode)\n",
    "image_file = 'cartpole-v0.gif'\n",
    "# loop=0: loop forever, duration=1: play each frame for 1ms\n",
    "images[0].save(\n",
    "    image_file, save_all=True, append_images=images[1:], loop=0, duration=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "mounted-ideal",
   "metadata": {},
   "source": [
    "------"
   ]
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}