legacy/Install.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Installing `heptabot` on raw system\n",
    "\n",
    "**This notebook can (and should) be downloaded and run independently from the repository, as it contains all the necessary code to install `heptabot`.** The notebook was tested to install the system correctly on [vast.ai](https://vast.ai/console/create/) instances using `nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04` image. `heptabot` is supposed to be run on Linux x64 arhitecture."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we install packages to interact with web repositories where we store our code and models: `git` and `wget`. We also install `gcc`, which is missing on some systems and is needed to compile C code used in some of our dependencies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The following additional packages will be installed:\n",
      "  git-man libcurl3-gnutls liberror-perl\n",
      "Suggested packages:\n",
      "  gettext-base git-daemon-run | git-daemon-sysvinit git-doc git-el git-email\n",
      "  git-gui gitk gitweb git-cvs git-mediawiki git-svn\n",
      "The following NEW packages will be installed:\n",
      "  git git-man libcurl3-gnutls liberror-perl wget\n",
      "0 upgraded, 5 newly installed, 0 to remove and 94 not upgraded.\n",
      "Need to get 5269 kB of archives.\n",
      "After this operation, 35.5 MB of additional disk space will be used.\n",
      "debconf: delaying package configuration, since apt-utils is not installed\n",
      "\n",
      "\u001b7\u001b[0;23r\u001b8\u001b[1ASelecting previously unselected package wget.\n",
      "(Reading database ... 16889 files and directories currently installed.)\n",
      "Preparing to unpack .../wget_1.19.4-1ubuntu2.2_amd64.deb ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [  0%]\u001b[49m\u001b[39m [..........................................................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [  4%]\u001b[49m\u001b[39m [##........................................................] \u001b8Unpacking wget (1.19.4-1ubuntu2.2) ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [  8%]\u001b[49m\u001b[39m [####......................................................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 12%]\u001b[49m\u001b[39m [######....................................................] \u001b8Selecting previously unselected package libcurl3-gnutls:amd64.\n",
      "Preparing to unpack .../libcurl3-gnutls_7.58.0-2ubuntu3.12_amd64.deb ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 15%]\u001b[49m\u001b[39m [########..................................................] \u001b8Unpacking libcurl3-gnutls:amd64 (7.58.0-2ubuntu3.12) ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 19%]\u001b[49m\u001b[39m [###########...............................................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 23%]\u001b[49m\u001b[39m [#############.............................................] \u001b8Selecting previously unselected package liberror-perl.\n",
      "Preparing to unpack .../liberror-perl_0.17025-1_all.deb ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 27%]\u001b[49m\u001b[39m [###############...........................................] \u001b8Unpacking liberror-perl (0.17025-1) ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 31%]\u001b[49m\u001b[39m [#################.........................................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 35%]\u001b[49m\u001b[39m [####################......................................] \u001b8Selecting previously unselected package git-man.\n",
      "Preparing to unpack .../git-man_1%3a2.17.1-1ubuntu0.7_all.deb ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 38%]\u001b[49m\u001b[39m [######################....................................] \u001b8Unpacking git-man (1:2.17.1-1ubuntu0.7) ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 42%]\u001b[49m\u001b[39m [########################..................................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 46%]\u001b[49m\u001b[39m [##########################................................] \u001b8Selecting previously unselected package git.\n",
      "Preparing to unpack .../git_1%3a2.17.1-1ubuntu0.7_amd64.deb ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 50%]\u001b[49m\u001b[39m [#############################.............................] \u001b8Unpacking git (1:2.17.1-1ubuntu0.7) ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 54%]\u001b[49m\u001b[39m [###############################...........................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 58%]\u001b[49m\u001b[39m [#################################.........................] \u001b8Setting up git-man (1:2.17.1-1ubuntu0.7) ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 62%]\u001b[49m\u001b[39m [###################################.......................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 65%]\u001b[49m\u001b[39m [#####################################.....................] \u001b8Setting up liberror-perl (0.17025-1) ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 69%]\u001b[49m\u001b[39m [########################################..................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 73%]\u001b[49m\u001b[39m [##########################################................] \u001b8Setting up libcurl3-gnutls:amd64 (7.58.0-2ubuntu3.12) ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 77%]\u001b[49m\u001b[39m [############################################..............] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 81%]\u001b[49m\u001b[39m [##############################################............] \u001b8Setting up wget (1.19.4-1ubuntu2.2) ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 85%]\u001b[49m\u001b[39m [#################################################.........] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 88%]\u001b[49m\u001b[39m [###################################################.......] \u001b8Setting up git (1:2.17.1-1ubuntu0.7) ...\n",
      "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 92%]\u001b[49m\u001b[39m [#####################################################.....] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 96%]\u001b[49m\u001b[39m [#######################################################...] \u001b8Processing triggers for libc-bin (2.27-3ubuntu1) ...\n",
      "\n",
      "\u001b7\u001b[0;24r\u001b8\u001b[1A\u001b[J"
     ]
    }
   ],
   "source": [
    "!apt install -y git wget gcc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we get the `heptabot` source code and enter the containing directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cloning into 'heptabot'...\n",
      "remote: Enumerating objects: 147, done.\u001b[K\n",
      "remote: Counting objects: 100% (147/147), done.\u001b[K\n",
      "remote: Compressing objects: 100% (115/115), done.\u001b[K\n",
      "remote: Total 147 (delta 64), reused 105 (delta 29), pack-reused 0\u001b[K\n",
      "Receiving objects: 100% (147/147), 137.87 KiB | 1.11 MiB/s, done.\n",
      "Resolving deltas: 100% (64/64), done.\n",
      "/root/heptabot\n"
     ]
    }
   ],
   "source": [
    "!git clone https://github.com/lcl-hse/heptabot\n",
    "%cd heptabot\n",
    "!mv install/* ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To facilitate setting up the environment, this script relies on `mamba`, which is a `conda` reimplementation with faster environment solving. So now we check if `mamba` is installed, and if it isn't, we let the code install it for us."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mamba installation not found. Installing\n",
      "⏬ Downloading https://github.com/jaimergp/miniforge/releases/latest/download/Mambaforge-colab-Linux-x86_64.sh...\n",
      "📦 Installing...\n",
      "📌 Adjusting configuration...\n",
      "🩹 Patching environment...\n",
      "⏲ Done in 0:00:21\n"
     ]
    }
   ],
   "source": [
    "mamba_test = !which mamba\n",
    "if mamba_test:\n",
    "    print(\"mamba installation found. Feel free to proceed\")\n",
    "else:\n",
    "    print(\"mamba installation not found. Installing\")\n",
    "    import os\n",
    "    from install_mamba import install_mambaforge\n",
    "    prefix = os.environ.get(\"HOME\") or \"/usr/local\"\n",
    "    prefix += \"/mambaforge\"\n",
    "    install_mambaforge(prefix=prefix)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also tell our shell where the `mamba` binaries are located."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "PPATH=\"$(echo '/opt/conda/bin' | sed 's/\\//\\\\\\//g')\"\n",
    "CPATH=\"$(realpath  ~/mambaforge/condabin | sed 's/\\//\\\\\\//g')\"\n",
    "sed -i \"s/$PPATH/$CPATH:$PPATH/g\" ~/.bashrc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that `mamba` is installed, we need to connect to a new terminal session, as `mamba` is not configured correctly in the running one. Because of this, and because all the code of the following section should be executed in shell, we need to reconnect to the `Bash` kernel in jupyter. The code in the cell below installs the `Bash` kernel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting bash_kernel\n",
      "  Downloading bash_kernel-0.7.2-py2.py3-none-any.whl (15 kB)\n",
      "Requirement already satisfied: pexpect>=4.0 in /usr/local/lib/python3.6/dist-packages (from bash_kernel) (4.8.0)\n",
      "Requirement already satisfied: ptyprocess>=0.5 in /usr/local/lib/python3.6/dist-packages (from pexpect>=4.0->bash_kernel) (0.7.0)\n",
      "Installing collected packages: bash-kernel\n",
      "Successfully installed bash-kernel-0.7.2\n",
      "\u001b[33mWARNING: You are using pip version 20.1.1; however, version 21.0.1 is available.\n",
      "You should consider upgrading via the '/usr/bin/python3.real -m pip install --upgrade pip' command.\u001b[0m\n",
      "Installing IPython kernel spec\n"
     ]
    }
   ],
   "source": [
    "!pip install bash_kernel\n",
    "!python -m bash_kernel.install"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hit **F5** now, then select `Kernel` -> `Bash` after reload"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We enter the `heptabot` folder once again and set our installation code up for execution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "cd heptabot\n",
    "chmod +x setup.sh\n",
    "chmod +x start.sh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, finally, we can create the `heptabot` virtual environment and install `heptabot` there. Keep in mind that installation time depends on your network speed and computational capabilites of your system, and also that the first `heptabot` run will take some extra time, as it needs to download the model needed for `sentence_transformers` and some similar files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This script sets up a new 'heptabot' virtual environment and downloads all the necessary files.\n",
      "It depends on mamba, git and wget.\n",
      "We strongly suggest following the https://github.com/lcl-hse/heptabot/blob/cpu/notebooks/Install.ipynb notebook to avoid any unexpected problems.\n",
      "Initializing virtual environment with python 3.6.9\n",
      "  Package                          Version  Build              Channel                    Size\n",
      "────────────────────────────────────────────────────────────────────────────────────────────────\n",
      "  Install:\n",
      "────────────────────────────────────────────────────────────────────────────────────────────────\n",
      "\n",
      "\u001b[32m  argon2-cffi                  \u001b[00m     20.1.0  py36h8f6f2f9_2     conda-forge/linux-64      47 KB\n",
      "\u001b[32m  async_generator              \u001b[00m       1.10  py_0               conda-forge/noarch        18 KB\n",
      "\u001b[32m  attrs                        \u001b[00m     20.3.0  pyhd3deb0d_0       conda-forge/noarch        41 KB\n",
      "\u001b[32m  backcall                     \u001b[00m      0.2.0  pyh9f0ad1d_0       conda-forge/noarch        13 KB\n",
      "\u001b[32m  backports                    \u001b[00m        1.0  py36_1             conda-forge/linux-64       4 KB\n",
      "\u001b[32m  backports.functools_lru_cache\u001b[00m      1.6.1  py_0               conda-forge/noarch         8 KB\n",
      "\u001b[32m  bleach                       \u001b[00m      3.3.0  pyh44b312d_0       conda-forge/noarch       111 KB\n",
      "\u001b[32m  decorator                    \u001b[00m      4.4.2  py_0               conda-forge/noarch        11 KB\n",
      "\u001b[32m  defusedxml                   \u001b[00m      0.6.0  py_0               conda-forge/noarch        22 KB\n",
      "\u001b[32m  entrypoints                  \u001b[00m        0.3  py36h9f0ad1d_1002  conda-forge/linux-64      13 KB\n",
      "\u001b[32m  importlib-metadata           \u001b[00m      3.7.0  py36h5fab9bb_0     conda-forge/linux-64      23 KB\n",
      "\u001b[32m  importlib_metadata           \u001b[00m      3.7.0  hd8ed1ab_0         conda-forge/noarch         3 KB\n",
      "\u001b[32m  ipykernel                    \u001b[00m      5.5.0  py36he448a4c_1     conda-forge/linux-64     166 KB\n",
      "\u001b[32m  ipython                      \u001b[00m     7.16.1  py36he448a4c_2     conda-forge/linux-64       1 MB\n",
      "\u001b[32m  ipython_genutils             \u001b[00m      0.2.0  py36_0             conda-forge/linux-64      36 KB\n",
      "\u001b[32m  jedi                         \u001b[00m     0.17.2  py36h9f0ad1d_1     conda-forge/linux-64     946 KB\n",
      "\u001b[32m  jinja2                       \u001b[00m     2.11.3  pyh44b312d_0       conda-forge/noarch        93 KB\n",
      "\u001b[32m  jsonschema                   \u001b[00m      3.2.0  py36h9f0ad1d_1     conda-forge/linux-64      89 KB\n",
      "\u001b[32m  jupyter_client               \u001b[00m     6.1.11  pyhd8ed1ab_1       conda-forge/noarch        76 KB\n",
      "\u001b[32m  jupyter_core                 \u001b[00m      4.7.1  py36h5fab9bb_0     conda-forge/linux-64      72 KB\n",
      "\u001b[32m  jupyterlab_pygments          \u001b[00m      0.1.2  pyh9f0ad1d_0       conda-forge/noarch         8 KB\n",
      "\u001b[32m  libsodium                    \u001b[00m     1.0.18  h516909a_1         conda-forge/linux-64     366 KB\n",
      "\u001b[32m  markupsafe                   \u001b[00m      1.1.1  py36h8f6f2f9_3     conda-forge/linux-64      27 KB\n",
      "\u001b[32m  mistune                      \u001b[00m      0.8.4  py36h8f6f2f9_1003  conda-forge/linux-64      54 KB\n",
      "\u001b[32m  nb_conda                     \u001b[00m      2.2.1  py36h9f0ad1d_4     conda-forge/linux-64      36 KB\n",
      "\u001b[32m  nb_conda_kernels             \u001b[00m      2.3.1  py36h5fab9bb_0     conda-forge/linux-64      27 KB\n",
      "\u001b[32m  nbclient                     \u001b[00m      0.5.3  pyhd8ed1ab_0       conda-forge/noarch        67 KB\n",
      "\u001b[32m  nbconvert                    \u001b[00m      6.0.7  py36h5fab9bb_3     conda-forge/linux-64     533 KB\n",
      "\u001b[32m  nbformat                     \u001b[00m      5.1.2  pyhd8ed1ab_1       conda-forge/noarch        66 KB\n",
      "\u001b[32m  nest-asyncio                 \u001b[00m      1.5.1  pyhd3eb1b0_0       pkgs/main/noarch          10 KB\n",
      "\u001b[32m  notebook                     \u001b[00m      6.2.0  py36h5fab9bb_0     conda-forge/linux-64       6 MB\n",
      "\u001b[32m  packaging                    \u001b[00m       20.9  pyh44b312d_0       conda-forge/noarch        35 KB\n",
      "\u001b[32m  pandoc                       \u001b[00m     2.11.4  h7f98852_0         conda-forge/linux-64      18 MB\n",
      "\u001b[32m  pandocfilters                \u001b[00m      1.4.3  py36h06a4308_1     pkgs/main/linux-64        14 KB\n",
      "\u001b[32m  parso                        \u001b[00m      0.7.1  pyh9f0ad1d_0       conda-forge/noarch        70 KB\n",
      "\u001b[32m  pexpect                      \u001b[00m      4.8.0  py36h9f0ad1d_1     conda-forge/linux-64      79 KB\n",
      "\u001b[32m  pickleshare                  \u001b[00m      0.7.5  py36h9f0ad1d_1002  conda-forge/linux-64      13 KB\n",
      "\u001b[32m  prometheus_client            \u001b[00m      0.9.0  pyhd3deb0d_0       conda-forge/noarch        44 KB\n",
      "\u001b[32m  prompt-toolkit               \u001b[00m     3.0.16  pyha770c72_0       conda-forge/noarch       244 KB\n",
      "\u001b[32m  ptyprocess                   \u001b[00m      0.7.0  pyhd3deb0d_0       conda-forge/noarch        16 KB\n",
      "\u001b[32m  pygments                     \u001b[00m      2.8.0  pyhd8ed1ab_0       conda-forge/noarch       736 KB\n",
      "\u001b[32m  pyparsing                    \u001b[00m      2.4.7  pyh9f0ad1d_0       conda-forge/noarch        60 KB\n",
      "\u001b[32m  pyrsistent                   \u001b[00m     0.17.3  py36h8f6f2f9_2     conda-forge/linux-64      89 KB\n",
      "\u001b[32m  python-dateutil              \u001b[00m      2.8.1  py_0               conda-forge/noarch       220 KB\n",
      "\u001b[32m  pyzmq                        \u001b[00m     22.0.3  py36h7068817_1     conda-forge/linux-64     496 KB\n",
      "\u001b[32m  send2trash                   \u001b[00m      1.5.0  py_0               conda-forge/noarch        12 KB\n",
      "\u001b[32m  terminado                    \u001b[00m      0.9.2  py36h5fab9bb_0     conda-forge/linux-64      26 KB\n",
      "\u001b[32m  testpath                     \u001b[00m      0.4.4  py_0               conda-forge/noarch        85 KB\n",
      "\u001b[32m  tornado                      \u001b[00m        6.1  py36h8f6f2f9_1     conda-forge/linux-64     643 KB\n",
      "\u001b[32m  traitlets                    \u001b[00m      4.3.3  py36h9f0ad1d_1     conda-forge/linux-64     133 KB\n",
      "\u001b[32m  typing_extensions            \u001b[00m    3.7.4.3  py_0               conda-forge/noarch        25 KB\n",
      "\u001b[32m  wcwidth                      \u001b[00m      0.2.5  pyh9f0ad1d_2       conda-forge/noarch        33 KB\n",
      "\u001b[32m  webencodings                 \u001b[00m      0.5.1  py_1               conda-forge/noarch        12 KB\n",
      "\u001b[32m  zeromq                       \u001b[00m      4.3.4  h9c3ff4c_0         conda-forge/linux-64     352 KB\n",
      "\u001b[32m  zipp                         \u001b[00m      3.4.0  py_0               conda-forge/noarch        11 KB\n",
      "\n",
      "  Change:\n",
      "────────────────────────────────────────────────────────────────────────────────────────────────\n",
      "\n",
      "\u001b[31m  certifi                      \u001b[00m  2020.12.5  py36h5fab9bb_0     installed                      \n",
      "\u001b[32m  certifi                      \u001b[00m  2020.12.5  py36h5fab9bb_1     conda-forge/linux-64     143 KB\n",
      "\n",
      "  Upgrade:\n",
      "────────────────────────────────────────────────────────────────────────────────────────────────\n",
      "\n",
      "\u001b[31m  ca-certificates              \u001b[00m  2020.12.5  ha878542_0         installed                      \n",
      "\u001b[32m  ca-certificates              \u001b[00m  2021.1.19  h06a4308_0         pkgs/main/linux-64       121 KB\n",
      "\u001b[31m  openssl                      \u001b[00m     1.1.1i  h7f98852_0         installed                      \n",
      "\u001b[32m  openssl                      \u001b[00m     1.1.1j  h7f98852_0         conda-forge/linux-64       2 MB\n",
      "\n",
      "  Summary:\n",
      "\n",
      "  Install: 55 packages\n",
      "  Change: 1 packages\n",
      "  Upgrade: 2 packages\n",
      "\n",
      "  Total download: 34 MB\n",
      "\n",
      "────────────────────────────────────────────────────────────────────────────────────────────────\n",
      "\n",
      "Preparing transaction: ...working... done\n",
      "Verifying transaction: ...working... done\n",
      "Executing transaction: ...working... Enabling nb_conda_kernels...\n",
      "CONDA_PREFIX: /root/mambaforge\n",
      "Status: enabled\n",
      "\n",
      "Config option `kernel_spec_manager_class` not recognized by `EnableNBExtensionApp`.\n",
      "Enabling notebook extension nb_conda/main...\n",
      "      - Validating: \u001b[32mOK\u001b[0m\n",
      "Enabling tree extension nb_conda/tree...\n",
      "      - Validating: \u001b[32mOK\u001b[0m\n",
      "Config option `kernel_spec_manager_class` not recognized by `EnableServerExtensionApp`.\n",
      "Enabling: nb_conda\n",
      "- Writing config: /root/mambaforge/etc/jupyter\n",
      "    - Validating...\n",
      "      nb_conda 2.2.1 \u001b[32mOK\u001b[0m\n",
      "\n",
      "done\n",
      "  Package             Version  Build               Channel                    Size\n",
      "────────────────────────────────────────────────────────────────────────────────────\n",
      "  Install:\n",
      "────────────────────────────────────────────────────────────────────────────────────\n",
      "\n",
      "\u001b[32m  _libgcc_mutex   \u001b[00m        0.1  conda_forge         conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  _openmp_mutex   \u001b[00m        4.5  1_gnu               conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  ca-certificates \u001b[00m  2020.12.5  ha878542_0          conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  certifi         \u001b[00m  2020.12.5  py36h5fab9bb_1      conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  ld_impl_linux-64\u001b[00m     2.35.1  hea4e1c9_2          conda-forge/linux-64     618 KB\n",
      "\u001b[32m  libffi          \u001b[00m      3.2.1  he1b5a44_1007       conda-forge/linux-64      47 KB\n",
      "\u001b[32m  libgcc-ng       \u001b[00m      9.3.0  h2828fa1_18         conda-forge/linux-64       8 MB\n",
      "\u001b[32m  libgomp         \u001b[00m      9.3.0  h2828fa1_18         conda-forge/linux-64     376 KB\n",
      "\u001b[32m  libstdcxx-ng    \u001b[00m      9.3.0  h6de172a_18         conda-forge/linux-64       4 MB\n",
      "\u001b[32m  ncurses         \u001b[00m        6.2  h58526e2_4          conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  openssl         \u001b[00m     1.1.1j  h7f98852_0          conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  pip             \u001b[00m     21.0.1  pyhd8ed1ab_0        conda-forge/noarch         1 MB\n",
      "\u001b[32m  python          \u001b[00m      3.6.9  h9d8adfe_0_cpython  conda-forge/linux-64      34 MB\n",
      "\u001b[32m  python_abi      \u001b[00m        3.6  1_cp36m             conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  readline        \u001b[00m        8.0  he28a2e2_2          conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  setuptools      \u001b[00m     49.6.0  py36h5fab9bb_3      conda-forge/linux-64     936 KB\n",
      "\u001b[32m  sqlite          \u001b[00m     3.34.0  h74cdb3f_0          conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  tk              \u001b[00m     8.6.10  hed695b0_1          conda-forge/linux-64       3 MB\n",
      "\u001b[32m  wheel           \u001b[00m     0.36.2  pyhd3deb0d_0        conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  xz              \u001b[00m      5.2.5  h516909a_1          conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  zlib            \u001b[00m     1.2.11  h516909a_1010       conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\n",
      "  Summary:\n",
      "\n",
      "  Install: 21 packages\n",
      "\n",
      "  Total download: 52 MB\n",
      "\n",
      "────────────────────────────────────────────────────────────────────────────────────\n",
      "\n",
      "Preparing transaction: ...working... done\n",
      "Verifying transaction: ...working... done\n",
      "Executing transaction: ...working... done\n",
      "no change     /root/mambaforge/condabin/conda\n",
      "no change     /root/mambaforge/bin/conda\n",
      "no change     /root/mambaforge/bin/conda-env\n",
      "no change     /root/mambaforge/bin/activate\n",
      "no change     /root/mambaforge/bin/deactivate\n",
      "no change     /root/mambaforge/etc/profile.d/conda.sh\n",
      "no change     /root/mambaforge/etc/fish/conf.d/conda.fish\n",
      "no change     /root/mambaforge/shell/condabin/Conda.psm1\n",
      "no change     /root/mambaforge/shell/condabin/conda-hook.ps1\n",
      "no change     /root/mambaforge/lib/python3.6/site-packages/xontrib/conda.xsh\n",
      "no change     /root/mambaforge/etc/profile.d/conda.csh\n",
      "modified      /root/.bashrc\n",
      "\n",
      "==> For changes to take effect, close and re-open your current shell. <==\n",
      "\n",
      "\n",
      "Installing requirements\n",
      "  Package                           Version  Build                            Channel                    Size\n",
      "───────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
      "  Install:\n",
      "───────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
      "\n",
      "\u001b[32m  argon2-cffi                  \u001b[00m      20.1.0  py36h8f6f2f9_2                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  async_generator              \u001b[00m        1.10  py_0                             conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  attrs                        \u001b[00m      20.3.0  pyhd3deb0d_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  backcall                     \u001b[00m       0.2.0  pyh9f0ad1d_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  backports                    \u001b[00m         1.0  py36_1                           conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  backports.functools_lru_cache\u001b[00m       1.6.1  py_0                             conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  blas                         \u001b[00m         1.1  openblas                         conda-forge/linux-64       1 KB\n",
      "\u001b[32m  bleach                       \u001b[00m       3.3.0  pyh44b312d_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  brotlipy                     \u001b[00m       0.7.0  py36he6145b8_1001                conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  cffi                         \u001b[00m      1.14.4  py36h211aa47_0                   conda-forge/linux-64     224 KB\n",
      "\u001b[32m  chardet                      \u001b[00m       4.0.0  py36h5fab9bb_1                   conda-forge/linux-64     219 KB\n",
      "\u001b[32m  click                        \u001b[00m       7.1.2  pyh9f0ad1d_0                     conda-forge/noarch        64 KB\n",
      "\u001b[32m  cryptography                 \u001b[00m       3.4.4  py36hc39840e_0                   conda-forge/linux-64     631 KB\n",
      "\u001b[32m  cymem                        \u001b[00m      1.31.2  py36_0                           conda-forge/linux-64      67 KB\n",
      "\u001b[32m  cytoolz                      \u001b[00m       0.8.2  py36_0                           conda-forge/linux-64       1 MB\n",
      "\u001b[32m  decorator                    \u001b[00m       4.4.2  py_0                             conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  defusedxml                   \u001b[00m       0.6.0  py_0                             conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  diff-match-patch             \u001b[00m    20200713  pyh9f0ad1d_0                     conda-forge/noarch        38 KB\n",
      "\u001b[32m  dill                         \u001b[00m       0.2.9  py36_0                           conda-forge/linux-64     113 KB\n",
      "\u001b[32m  entrypoints                  \u001b[00m         0.3  py36h9f0ad1d_1002                conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  flask                        \u001b[00m       1.1.2  pyh9f0ad1d_0                     conda-forge/noarch        70 KB\n",
      "\u001b[32m  ftfy                         \u001b[00m       4.4.2  py36_0                           conda-forge/linux-64      51 KB\n",
      "\u001b[32m  html5lib                     \u001b[00m         1.1  pyh9f0ad1d_0                     conda-forge/noarch        89 KB\n",
      "\u001b[32m  idna                         \u001b[00m        2.10  pyh9f0ad1d_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  importlib-metadata           \u001b[00m       3.7.0  py36h5fab9bb_0                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  importlib_metadata           \u001b[00m       3.7.0  hd8ed1ab_0                       conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  ipykernel                    \u001b[00m       5.5.0  py36he448a4c_1                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  ipython                      \u001b[00m      7.16.1  py36he448a4c_2                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  ipython_genutils             \u001b[00m       0.2.0  py36_0                           conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  ipywidgets                   \u001b[00m       7.6.3  pyhd3deb0d_0                     conda-forge/noarch       101 KB\n",
      "\u001b[32m  itsdangerous                 \u001b[00m       1.1.0  py_0                             conda-forge/noarch        16 KB\n",
      "\u001b[32m  jedi                         \u001b[00m      0.17.2  py36h9f0ad1d_1                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  jinja2                       \u001b[00m      2.11.3  pyh44b312d_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  joblib                       \u001b[00m       1.0.1  pyhd8ed1ab_0                     conda-forge/noarch       206 KB\n",
      "\u001b[32m  jsonschema                   \u001b[00m       3.2.0  py36h9f0ad1d_1                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  jupyter_client               \u001b[00m      6.1.11  pyhd8ed1ab_1                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  jupyter_core                 \u001b[00m       4.7.1  py36h5fab9bb_0                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  jupyterlab_pygments          \u001b[00m       0.1.2  pyh9f0ad1d_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  jupyterlab_widgets           \u001b[00m       1.0.0  pyhd8ed1ab_1                     conda-forge/noarch       130 KB\n",
      "\u001b[32m  libgcc                       \u001b[00m       7.2.0  h69d50b8_2                       conda-forge/linux-64     304 KB\n",
      "\u001b[32m  libgfortran-ng               \u001b[00m       7.5.0  h14aa051_18                      conda-forge/linux-64      22 KB\n",
      "\u001b[32m  libgfortran4                 \u001b[00m       7.5.0  h14aa051_18                      conda-forge/linux-64       1 MB\n",
      "\u001b[32m  libsodium                    \u001b[00m      1.0.18  h516909a_1                       conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  markupsafe                   \u001b[00m       1.1.1  py36h8f6f2f9_3                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  mistune                      \u001b[00m       0.8.4  py36h8f6f2f9_1003                conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  murmurhash                   \u001b[00m      0.26.4  py36_0                           conda-forge/linux-64      37 KB\n",
      "\u001b[32m  nbclient                     \u001b[00m       0.5.3  pyhd8ed1ab_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  nbconvert                    \u001b[00m       6.0.7  py36h5fab9bb_3                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  nbformat                     \u001b[00m       5.1.2  pyhd8ed1ab_1                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  nest-asyncio                 \u001b[00m       1.5.1  pyhd3eb1b0_0                     pkgs/main/noarch    \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  nltk                         \u001b[00m         3.5  py_0                             pkgs/main/noarch         976 KB\n",
      "\u001b[32m  notebook                     \u001b[00m       6.2.0  py36h5fab9bb_0                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  numpy                        \u001b[00m      1.12.1  py36_blas_openblash1522bff_1001  conda-forge/linux-64       4 MB\n",
      "\u001b[32m  openblas                     \u001b[00m       0.3.3  h9ac9557_1001                    conda-forge/linux-64      16 MB\n",
      "\u001b[32m  packaging                    \u001b[00m        20.9  pyh44b312d_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  pandas                       \u001b[00m      0.24.2  py36hf484d3e_0                   conda-forge/linux-64      11 MB\n",
      "\u001b[32m  pandoc                       \u001b[00m      2.11.4  h7f98852_0                       conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  pandocfilters                \u001b[00m       1.4.3  py36h06a4308_1                   pkgs/main/linux-64  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  parso                        \u001b[00m       0.7.1  pyh9f0ad1d_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  pexpect                      \u001b[00m       4.8.0  py36h9f0ad1d_1                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  pickleshare                  \u001b[00m       0.7.5  py36h9f0ad1d_1002                conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  plac                         \u001b[00m       0.9.6  py36_0                           conda-forge/linux-64      36 KB\n",
      "\u001b[32m  preshed                      \u001b[00m       1.0.1  py36hf484d3e_1000                conda-forge/linux-64      83 KB\n",
      "\u001b[32m  prometheus_client            \u001b[00m       0.9.0  pyhd3deb0d_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  prompt-toolkit               \u001b[00m      3.0.16  pyha770c72_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  ptyprocess                   \u001b[00m       0.7.0  pyhd3deb0d_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  pycparser                    \u001b[00m        2.20  pyh9f0ad1d_2                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  pygments                     \u001b[00m       2.8.0  pyhd8ed1ab_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  pyopenssl                    \u001b[00m      20.0.1  pyhd8ed1ab_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  pyparsing                    \u001b[00m       2.4.7  pyh9f0ad1d_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  pyro4                        \u001b[00m        4.80  pyh9f0ad1d_0                     conda-forge/noarch        67 KB\n",
      "\u001b[32m  pyrsistent                   \u001b[00m      0.17.3  py36h8f6f2f9_2                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  pysocks                      \u001b[00m       1.7.1  py36h5fab9bb_3                   conda-forge/linux-64      27 KB\n",
      "\u001b[32m  python-dateutil              \u001b[00m       2.8.1  py_0                             conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  python-levenshtein           \u001b[00m      0.12.2  py36h8f6f2f9_0                   conda-forge/linux-64      80 KB\n",
      "\u001b[32m  pytz                         \u001b[00m      2021.1  pyhd8ed1ab_0                     conda-forge/noarch       239 KB\n",
      "\u001b[32m  pyzmq                        \u001b[00m      22.0.3  py36h7068817_1                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  regex                        \u001b[00m  2017.11.09  py36_0                           conda-forge/linux-64     670 KB\n",
      "\u001b[32m  requests                     \u001b[00m      2.25.1  pyhd3deb0d_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  send2trash                   \u001b[00m       1.5.0  py_0                             conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  serpent                      \u001b[00m      1.30.2  py_0                             conda-forge/noarch        12 KB\n",
      "\u001b[32m  six                          \u001b[00m      1.15.0  pyh9f0ad1d_0                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  spacy                        \u001b[00m       1.9.0  np112py36_1                      conda-forge/linux-64       8 MB\n",
      "\u001b[32m  termcolor                    \u001b[00m       1.1.0  py36_1                           conda-forge/linux-64       6 KB\n",
      "\u001b[32m  terminado                    \u001b[00m       0.9.2  py36h5fab9bb_0                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  testpath                     \u001b[00m       0.4.4  py_0                             conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  thinc                        \u001b[00m       6.5.2  np112py36_0                      conda-forge/linux-64       2 MB\n",
      "\u001b[32m  toolz                        \u001b[00m      0.11.1  py_0                             conda-forge/noarch        46 KB\n",
      "\u001b[32m  tornado                      \u001b[00m         6.1  py36h8f6f2f9_1                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  tqdm                         \u001b[00m      4.58.0  pyhd8ed1ab_0                     conda-forge/noarch        76 KB\n",
      "\u001b[32m  traitlets                    \u001b[00m       4.3.3  py36h9f0ad1d_1                   conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  typing_extensions            \u001b[00m     3.7.4.3  py_0                             conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  ujson                        \u001b[00m       4.0.2  py36hc4f0c31_0                   conda-forge/linux-64      48 KB\n",
      "\u001b[32m  urllib3                      \u001b[00m      1.26.3  pyhd8ed1ab_0                     conda-forge/noarch        99 KB\n",
      "\u001b[32m  wcwidth                      \u001b[00m       0.2.5  pyh9f0ad1d_2                     conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  webencodings                 \u001b[00m       0.5.1  py_1                             conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  werkzeug                     \u001b[00m       1.0.1  pyh9f0ad1d_0                     conda-forge/noarch       239 KB\n",
      "\u001b[32m  widgetsnbextension           \u001b[00m       3.5.1  py36h9f0ad1d_4                   conda-forge/linux-64       2 MB\n",
      "\u001b[32m  wrapt                        \u001b[00m      1.12.1  py36h8f6f2f9_3                   conda-forge/linux-64      47 KB\n",
      "\u001b[32m  zeromq                       \u001b[00m       4.3.4  h9c3ff4c_0                       conda-forge/linux-64\u001b[32m     Cached\u001b[00m\n",
      "\u001b[32m  zipp                         \u001b[00m       3.4.0  py_0                             conda-forge/noarch  \u001b[32m     Cached\u001b[00m\n",
      "\n",
      "  Upgrade:\n",
      "───────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
      "\n",
      "\u001b[31m  ca-certificates              \u001b[00m   2020.12.5  ha878542_0                       installed                      \n",
      "\u001b[32m  ca-certificates              \u001b[00m   2021.1.19  h06a4308_0                       pkgs/main/linux-64  \u001b[32m     Cached\u001b[00m\n",
      "\n",
      "  Summary:\n",
      "\n",
      "  Install: 101 packages\n",
      "  Upgrade: 1 packages\n",
      "\n",
      "  Total download: 50 MB\n",
      "\n",
      "───────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
      "\n",
      "Preparing transaction: ...working... done\n",
      "Verifying transaction: ...working... done\n",
      "Executing transaction: ...working... Enabling notebook extension jupyter-js-widgets/extension...\n",
      "      - Validating: \u001b[32mOK\u001b[0m\n",
      "\n",
      "done\n",
      "\u001b[33mWARNING: The candidate selected for download or install is a yanked version: 'python-levenshtein' candidate (version 0.12.0 at https://files.pythonhosted.org/packages/42/a9/d1785c85ebf9b7dfacd08938dd028209c34a0ea3b1bcdb895208bd40a67d/python-Levenshtein-0.12.0.tar.gz#sha256=033a11de5e3d19ea25c9302d11224e1a1898fe5abd23c61c7c360c25195e3eb1 (from https://pypi.org/simple/python-levenshtein/))\n",
      "Reason for being yanked: Insecure, upgrade to 0.12.1\u001b[0m\n",
      "\u001b[33mCache entry deserialization failed, entry ignored\u001b[0m\n",
      "\n",
      "Setting up nltk and spaCy\n",
      "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
      "[nltk_data]   Unzipping tokenizers/punkt.zip.\n",
      "\n",
      "    Downloading en_core_web_sm-1.2.0/en_core_web_sm-1.2.0.tar.gz\n",
      "\n",
      "Collecting https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-1.2.0/en_core_web_sm-1.2.0.tar.gz\n",
      "  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-1.2.0/en_core_web_sm-1.2.0.tar.gz (52.2 MB)\n",
      "\u001b[K     |################################| 52.2 MB 24.2 MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: spacy<2.0.0,>=1.7.0 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from en-core-web-sm==1.2.0) (1.9.0)\n",
      "Requirement already satisfied: numpy>=1.7 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (1.16.1)\n",
      "Requirement already satisfied: murmurhash<0.27,>=0.26 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (0.26.4)\n",
      "Requirement already satisfied: cymem<1.32,>=1.30 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (1.31.2)\n",
      "Requirement already satisfied: preshed<2.0.0,>=1.0.0 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (1.0.1)\n",
      "Requirement already satisfied: thinc<6.6.0,>=6.5.0 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (6.5.2)\n",
      "Requirement already satisfied: plac<1.0.0,>=0.9.6 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (0.9.6)\n",
      "Collecting pip<10.0.0,>=9.0.0\n",
      "  Downloading pip-9.0.3-py2.py3-none-any.whl (1.4 MB)\n",
      "\u001b[K     |################################| 1.4 MB 5.3 MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: six in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (1.15.0)\n",
      "Requirement already satisfied: pathlib in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (1.0.1)\n",
      "Requirement already satisfied: ujson>=1.35 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (4.0.2)\n",
      "Requirement already satisfied: dill<0.3,>=0.2 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (0.2.9)\n",
      "Requirement already satisfied: requests<3.0.0,>=2.13.0 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (2.25.1)\n",
      "Requirement already satisfied: regex<2017.12.1,>=2017.4.1 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (2017.11.9)\n",
      "Requirement already satisfied: ftfy<5.0.0,>=4.4.2 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (4.4.2)\n",
      "Requirement already satisfied: html5lib in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from ftfy<5.0.0,>=4.4.2->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (1.1)\n",
      "Requirement already satisfied: wcwidth in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from ftfy<5.0.0,>=4.4.2->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (0.2.5)\n",
      "Requirement already satisfied: chardet<5,>=3.0.2 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from requests<3.0.0,>=2.13.0->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (4.0.0)\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from requests<3.0.0,>=2.13.0->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (1.26.3)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from requests<3.0.0,>=2.13.0->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (2020.12.5)\n",
      "Requirement already satisfied: idna<3,>=2.5 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from requests<3.0.0,>=2.13.0->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (2.10)\n",
      "Requirement already satisfied: wrapt in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from thinc<6.6.0,>=6.5.0->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (1.12.1)\n",
      "Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from thinc<6.6.0,>=6.5.0->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (4.58.0)\n",
      "Requirement already satisfied: cytoolz<0.9,>=0.8 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from thinc<6.6.0,>=6.5.0->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (0.8.2)\n",
      "Requirement already satisfied: termcolor in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from thinc<6.6.0,>=6.5.0->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (1.1.0)\n",
      "Requirement already satisfied: toolz>=0.8.0 in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from cytoolz<0.9,>=0.8->thinc<6.6.0,>=6.5.0->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (0.11.1)\n",
      "Requirement already satisfied: webencodings in /root/mambaforge/envs/heptabot/lib/python3.6/site-packages (from html5lib->ftfy<5.0.0,>=4.4.2->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0) (0.5.1)\n",
      "Building wheels for collected packages: en-core-web-sm\n",
      "  Building wheel for en-core-web-sm (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for en-core-web-sm: filename=en_core_web_sm-1.2.0-py3-none-any.whl size=53385727 sha256=0f16c2f7de7c08fbe924793826bbbe31048c522378cae8a216eb7c1715d2982d\n",
      "  Stored in directory: /tmp/pip-ephem-wheel-cache-cokv_6aw/wheels/f9/4d/f5/0dd60e5d14063e5b09aab0c3d928b974fb854779603d92d408\n",
      "Successfully built en-core-web-sm\n",
      "Installing collected packages: pip, en-core-web-sm\n",
      "  Attempting uninstall: pip\n",
      "    Found existing installation: pip 21.0.1\n",
      "    Uninstalling pip-21.0.1:\n",
      "      Successfully uninstalled pip-21.0.1\n",
      "Successfully installed en-core-web-sm-1.2.0 pip-9.0.3\n",
      "\n",
      "\u001b[93m    Linking successful\u001b[0m\n",
      "\n",
      "    /root/mambaforge/envs/heptabot/lib/python3.6/site-packages/en_core_web_sm/en_core_web_sm-1.2.0\n",
      "    -->\n",
      "    /root/mambaforge/envs/heptabot/lib/python3.6/site-packages/spacy/data/en\n",
      "\n",
      "    You can now load the model via spacy.load('en').\n",
      "\n",
      "\n",
      "Setting up heptabot for jupyter (optional)\n",
      "Installed kernelspec heptabot in /root/.local/share/jupyter/kernels/heptabot\n",
      "\n",
      "Downloading models\n",
      "err_type_classifier 100%[===================>]  56.24M  17.6MB/s    in 3.2s    \n",
      "saved_model.pb      100%[===================>]   6.79M  5.88MB/s    in 1.2s    \n",
      "variables.data-0000 100%[===================>]       8  --.-KB/s    in 0s      \n",
      "variables.data-0000 100%[===================>] 425.15M  18.7MB/s    in 22s     \n",
      "variables.index     100%[===================>]  11.47K  --.-KB/s    in 0.001s  \n",
      "\n",
      "heptabot is ready to use!\n",
      "run conda init; conda activate heptabot; ./start.sh to activate the system\n",
      "\n",
      "real\t5m41.003s\n",
      "user\t3m23.580s\n",
      "sys\t1m7.247s\n"
     ]
    }
   ],
   "source": [
    "time ./setup.sh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Congratulations, `heptabot` is now ready! Note that it also created a `heptabot` kernel for you, which will become available if you go to Jupyter starting page and hit **F5**."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}