demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# About\n",
    "\n",
    "It's difficult to use functions in jupyter notebook, since we want different steps to be in different cells, so one of the main functions of this module is to emulate a function like scope of the variables - which get destroyed at the end of the experiment. Some extra magic is added to reclaim GPU and General RAM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ipyexperiments import IPyExperimentsCPU, IPyExperimentsPytorch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup and preload"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import torch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def consume_gpu_ram(n): return torch.ones((n, n)).cuda()\n",
    "def consume_cpu_ram(n): return np.ones((n, n))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`pytorch`'s CUDA machinery seems to require ~0.5GB GPU RAM for this particular card, and ~2GB of RAM upon its first use, and it's not shared between processes. So if you use pytorch w/ CUDA and you have the same GPU - its real capacity is 0.5GB smaller from the get going, and multiply that by the number of concurrent processes. Newer GPUs may consume easily up to 1.5GB in CUDA kernels.\n",
    "\n",
    "Because of that, in order to get the numbers right, it can be a good idea to pre-load it by allocating a tiny tensor on `cuda`. If we don't - the first experiment' stats will be misleading/incorrect."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#z = consume_gpu_ram(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But, if you use `IPyExperiments` - it performs this preloading for you already, when the backend is loaded (see below)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Experiment with no GPU\n",
    "\n",
    "Let's consume a big chunk of non-GPU RAM and reclaim it at the end of the experiment.\n",
    "\n",
    "In this experiment we use the `cpu` backend, so GPU RAM will not be managed, regardless of whether there is a GPU that can be used or not. This mode is primarily used for configurations without GPU."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "*** Experiment started with the CPU-only backend\n",
      "\n",
      "\n",
      "*** Current state:\n",
      "RAM:     Used     Free    Total        Util\n",
      "CPU:    1,055   86,223  128,696 MB   0.82% \n",
      "\n",
      "\n",
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.000\n",
      "･ CPU:          0          0      1,055 MB |\n"
     ]
    }
   ],
   "source": [
    "exp1 = IPyExperimentsCPU()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.364\n",
      "･ CPU:      2,048          0      3,104 MB |\n"
     ]
    }
   ],
   "source": [
    "x1 = consume_cpu_ram(2**14) # about 2GB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:01.201\n",
      "･ CPU:      2,047          0      5,151 MB |\n"
     ]
    }
   ],
   "source": [
    "x2 = consume_cpu_ram(2**14) # about 2GB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.000\n",
      "･ CPU:          0          0      5,151 MB |\n",
      "\n",
      "IPyExperimentsCPU: Finishing\n",
      "\n",
      "*** Experiment finished in 00:00:01 (elapsed wallclock time)\n",
      "\n",
      "*** Newly defined local variables:\n",
      "Deleted: x1, x2\n",
      "\n",
      "*** Experiment memory:\n",
      "RAM: Consumed       Reclaimed\n",
      "CPU:    4,096    4,095 MB ( 99.99%)\n",
      "\n",
      "*** Current state:\n",
      "RAM:     Used     Free    Total        Util\n",
      "CPU:    1,056   87,879  128,696 MB   0.82% \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "del exp1 # finish experiment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## GPU Experiment: consume general and GPU RAM\n",
    "\n",
    "Let's consume a big chunk of each, general and GPU RAM and reclaim both of them, at the end of the experiment.\n",
    "\n",
    "This time we wil use the GPU backed `pytorch`, so both GPU and general RAM will be managed. This is the default backed, so if you don't pass this argument, it'll default to `pytorch`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "*** Experiment started with the Pytorch backend\n",
      "Device: ID 0, NVIDIA GeForce RTX 3090 (8119 RAM)\n",
      "\n",
      "\n",
      "*** Current state:\n",
      "RAM:     Used     Free    Total        Util\n",
      "CPU:    4,842   84,641  128,696 MB   3.76% \n",
      "GPU:      795    7,324    8,119 MB   9.79% \n",
      "\n",
      "\n",
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.000\n",
      "･ CPU:          1          0      4,843 MB |\n",
      "･ GPU:          0          0        795 MB |\n"
     ]
    }
   ],
   "source": [
    "exp2 = IPyExperimentsPytorch()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.223\n",
      "･ CPU:      2,048          0      6,891 MB |\n",
      "･ GPU:          0          0        795 MB |\n"
     ]
    }
   ],
   "source": [
    "x1 = consume_cpu_ram(2**14) # about 2GB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.293\n",
      "･ CPU:          0      1,023      6,892 MB |\n",
      "･ GPU:          0          0        795 MB |\n"
     ]
    }
   ],
   "source": [
    "x2 = consume_gpu_ram(2**14) # about 1GB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.000\n",
      "･ CPU:          0          0      6,892 MB |\n",
      "･ GPU:          0          0        795 MB |\n",
      "\n",
      "IPyExperimentsPytorch: Finishing\n",
      "\n",
      "*** Experiment finished in 00:00:03 (elapsed wallclock time)\n",
      "\n",
      "*** Newly defined local variables:\n",
      "Deleted: x1, x2\n",
      "\n",
      "*** Experiment memory:\n",
      "RAM: Consumed       Reclaimed\n",
      "CPU:    2,049    2,048 MB ( 99.92%)\n",
      "GPU:        0        0 MB (100.00%)\n",
      "\n",
      "*** Current state:\n",
      "RAM:     Used     Free    Total        Util\n",
      "CPU:    4,844   84,712  128,696 MB   3.76% \n",
      "GPU:      795    7,324    8,119 MB   9.79% \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "del exp2 # finish experiment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get experiment data, preserve some vars\n",
    "\n",
    "Here we demonstate features that help with using this framework programmatically. i.e. getting the functions to return experiment data during and at the end of the experiment, rather than just printing it. You can then use it to programmatically refine the hyper parameters before rerunning the experiment.\n",
    "\n",
    "This experiment also demonstrates how to save some of the local variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "*** Experiment started with the Pytorch backend\n",
      "Device: ID 0, NVIDIA GeForce RTX 3090 (8119 RAM)\n",
      "\n",
      "\n",
      "*** Current state:\n",
      "RAM:     Used     Free    Total        Util\n",
      "CPU:    4,844   84,711  128,696 MB   3.76% \n",
      "GPU:      795    7,324    8,119 MB   9.79% \n",
      "\n",
      "\n",
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.000\n",
      "･ CPU:          0          0      4,844 MB |\n",
      "･ GPU:          0          0        795 MB |\n"
     ]
    }
   ],
   "source": [
    "exp3 = IPyExperimentsPytorch()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.222\n",
      "･ CPU:      2,048          0      6,892 MB |\n",
      "･ GPU:          0          0        795 MB |\n"
     ]
    }
   ],
   "source": [
    "x1 = consume_cpu_ram(2**14) # about 2GB"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run an intermediary report of how much of the resources was consumed, and how much is available, returning the data as numbers. (none would be reclaimed yet, so it'll be zeros, but the return value is there for consistency)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "IPyExperimentMemory(consumed=2147491840, reclaimed=0, available=86656245760) IPyExperimentMemory(consumed=0, reclaimed=0, available=7680294912)\n",
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.001\n",
      "･ CPU:          0          0      6,892 MB |\n",
      "･ GPU:          0          0        795 MB |\n"
     ]
    }
   ],
   "source": [
    "cpu_data = exp3.data.cpu\n",
    "gpu_data = exp3.data.gpu\n",
    "print(cpu_data, gpu_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's preserve these variables, so that they remain available after the experiment is finished and the rest of the local variables get deleted. \n",
    "\n",
    "Note, that you need to pass the names of the variables and not the variables themselves."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.000\n",
      "･ CPU:          0          0      6,892 MB |\n",
      "･ GPU:          0          0        795 MB |\n"
     ]
    }
   ],
   "source": [
    "exp3.keep_var_names('cpu_data', 'gpu_data')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.280\n",
      "･ CPU:          0      1,023      6,892 MB |\n",
      "･ GPU:          0          0        795 MB |\n"
     ]
    }
   ],
   "source": [
    "x2 = consume_gpu_ram(2**14) # about 1GB"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run another intermediary report."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "IPyExperimentMemory(consumed=2147516416, reclaimed=0, available=86661820416)\n",
      "IPyExperimentMemory(consumed=0, reclaimed=0, available=7680294912)\n",
      "2147516416\n",
      "7680294912\n",
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.001\n",
      "･ CPU:          0          0      6,892 MB |\n",
      "･ GPU:          0          0        795 MB |\n"
     ]
    }
   ],
   "source": [
    "cpu_data = exp3.data.cpu\n",
    "gpu_data = exp3.data.gpu\n",
    "print(cpu_data)\n",
    "print(gpu_data)\n",
    "print(cpu_data.consumed)\n",
    "print(gpu_data.available)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Complete the experiment, delete local vars, reclaim memory, and run the final report of how much of the resources was consumed, and how much is available, and how much was reclaimed, returning the data as numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.000\n",
      "･ CPU:          0          0      6,892 MB |\n",
      "･ GPU:          0          0        795 MB |\n",
      "\n",
      "IPyExperimentsPytorch: Finishing\n",
      "\n",
      "*** Experiment finished in 00:00:00 (elapsed wallclock time)\n",
      "\n",
      "*** Newly defined local variables:\n",
      "Deleted: x1, x2\n",
      "Kept:    cpu_data, gpu_data\n",
      "\n",
      "*** Experiment memory:\n",
      "RAM: Consumed       Reclaimed\n",
      "CPU:    2,048    2,047 MB (100.00%)\n",
      "GPU:        0        0 MB (100.00%)\n",
      "\n",
      "*** Current state:\n",
      "RAM:     Used     Free    Total        Util\n",
      "CPU:    4,844   84,697  128,696 MB   3.76% \n",
      "GPU:      795    7,324    8,119 MB   9.79% \n",
      "\n",
      "\n",
      "\n",
      "Numerical data:\n",
      " IPyExperimentMemory(consumed=2147516416, reclaimed=2147446784, available=88812236800) IPyExperimentMemory(consumed=0, reclaimed=0, available=7680294912)\n"
     ]
    }
   ],
   "source": [
    "data = exp3.finish() # finish experiment\n",
    "# or:\n",
    "# _ = exp3.finish()\n",
    "# data = exp3.data\n",
    "cpu_data_final = data.cpu\n",
    "gpu_data_final= data.gpu\n",
    "\n",
    "print(\"\\nNumerical data:\\n\", cpu_data_final, gpu_data_final)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And let's test that we can still access the variables we asked not to destroy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Half-way data:\n",
      " IPyExperimentMemory(consumed=2147516416, reclaimed=0, available=86661820416) IPyExperimentMemory(consumed=0, reclaimed=0, available=7680294912)\n"
     ]
    }
   ],
   "source": [
    "print(\"\\nHalf-way data:\\n\", cpu_data, gpu_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using the context manager\n",
    "\n",
    "If you want to put all cells into one, you could simplify the experiment even further by using its context manager."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "*** Experiment started with the Pytorch backend\n",
      "Device: ID 0, NVIDIA GeForce RTX 3090 (8119 RAM)\n",
      "\n",
      "\n",
      "*** Current state:\n",
      "RAM:     Used     Free    Total        Util\n",
      "CPU:    4,844   84,709  128,696 MB   3.76% \n",
      "GPU:      795    7,324    8,119 MB   9.79% \n",
      "\n",
      "\n",
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.496\n",
      "･ CPU:      2,047      1,023      6,892 MB |\n",
      "･ GPU:          0          0        795 MB |\n",
      "\n",
      "IPyExperimentsPytorch: Finishing\n",
      "\n",
      "*** Experiment finished in 00:00:00 (elapsed wallclock time)\n",
      "\n",
      "*** Newly defined local variables:\n",
      "Deleted: x1, x2\n",
      "\n",
      "*** Experiment memory:\n",
      "RAM: Consumed       Reclaimed\n",
      "CPU:    2,047    2,048 MB (100.01%)\n",
      "GPU:        0        0 MB (100.00%)\n",
      "\n",
      "*** Current state:\n",
      "RAM:     Used     Free    Total        Util\n",
      "CPU:    4,844   84,712  128,696 MB   3.76% \n",
      "GPU:      795    7,324    8,119 MB   9.79% \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with IPyExperimentsPytorch(): \n",
    "    x1 = consume_cpu_ram(2**14)\n",
    "    x2 = consume_gpu_ram(2**14)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "*** Experiment started with the Pytorch backend\n",
      "Device: ID 0, NVIDIA GeForce RTX 3090 (8119 RAM)\n",
      "\n",
      "\n",
      "*** Current state:\n",
      "RAM:     Used     Free    Total        Util\n",
      "CPU:    4,844   84,711  128,696 MB   3.76% \n",
      "GPU:      795    7,324    8,119 MB   9.79% \n",
      "\n",
      "\n",
      "･ RAM:  △Consumed    △Peaked    Used Total | Exec time 0:00:00.501\n",
      "･ CPU:      2,048      1,023      6,892 MB |\n",
      "･ GPU:          0          0        795 MB |\n",
      "\n",
      "IPyExperimentsPytorch: Finishing\n",
      "\n",
      "*** Experiment finished in 00:00:00 (elapsed wallclock time)\n",
      "\n",
      "*** Newly defined local variables:\n",
      "Deleted: x1, x2\n",
      "Kept:    z\n",
      "\n",
      "*** Experiment memory:\n",
      "RAM: Consumed       Reclaimed\n",
      "CPU:    2,048    2,048 MB (100.00%)\n",
      "GPU:        0        0 MB (100.00%)\n",
      "\n",
      "*** Current state:\n",
      "RAM:     Used     Free    Total        Util\n",
      "CPU:    4,844   84,708  128,696 MB   3.76% \n",
      "GPU:      795    7,324    8,119 MB   9.79% \n",
      "\n",
      "\n",
      "some data\n"
     ]
    }
   ],
   "source": [
    "with IPyExperimentsPytorch() as exp: \n",
    "    x1 = consume_cpu_ram(2**14)\n",
    "    z = \"some data\"\n",
    "    x2 = consume_gpu_ram(2**14)\n",
    "    exp.keep_var_names('z')\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%javascript # prevent committing an unsaved notebook\n",
    "IPython.notebook.save_notebook()"
   ]
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "285px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "position": {
    "height": "323px",
    "left": "956px",
    "right": "20px",
    "top": "152px",
    "width": "350px"
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}