notebooks/2017-05-11-Running Jupyter Notebook on clusters.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Running Jupyter notebook on TSCC"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a script, say \"jupter_notebook\", with the following contents on TSCC."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```bash\n",
    "#!/bin/bash\n",
    "#PBS -q condo\n",
    "#PBS -N notebook\n",
    "#PBS -l nodes=1:ppn=1:ib\n",
    "#PBS -l walltime=8:00:00\n",
    "#PBS -V\n",
    "#PBS -A YOUR-group-account\n",
    "\n",
    "cd $PBS_O_WORKDIR\n",
    "\n",
    "# choose your own unique port between 8000 and 9999\n",
    "NOTEBOOKPORT=8181\n",
    "\n",
    "# setup tunnel between computing and login node\n",
    "ssh -N -f -R $NOTEBOOKPORT:localhost:$NOTEBOOKPORT $PBS_O_HOST\n",
    "\n",
    "# launch the notebook\n",
    "jupyter notebook --port=$NOTEBOOKPORT --no-browser > notebook.log\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a script, say \"tunnel_notebook\", with the following contents at your laptop"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```bash\n",
    "ssh -f -N -L 8181:localhost:8181 username@tscc-login$1.sdsc.edu\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Open a terminal, and run \"bash tunnel_notebook login_node\" , here login_node is a number. For instance, [username@tscc-login2 ~], login_node=2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Open ur favorite web browser, e.g Chrome.  Type \"localhost:8181\" in the address. It will open the jupyter notebook that you do data analysis on TSCC."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Running Jupyter notebook on Comet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a script, say \"jupter_notebook\", with the following contents on Comet."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```bash\n",
    "#!/bin/bash\n",
    "#SBATCH --nodes 1\n",
    "\n",
    "#### just 1 core (5.3GB RAM)\n",
    "#SBATCH --ntasks-per-node 1\n",
    "#SBATCH --partition shared  \n",
    "\n",
    "#### If you need a large memory(with maximum of 1.5T) uncomment below two lines\n",
    "####SBATCH -p large-shared\n",
    "####SBATCH --mem=10G\n",
    "\n",
    "#SBATCH --time  8:00:00\n",
    "#SBATCH --job-name notebook\n",
    "#SBATCH --output notebook.log\n",
    "\n",
    "# choose your own unique port between 8000 and 9999\n",
    "NOTEBOOKPORT=8181\n",
    "\n",
    "# setup tunnel between computing and login node\n",
    "ssh -N -f -R $NOTEBOOKPORT:localhost:$NOTEBOOKPORT $SLURM_SUBMIT_HOST\n",
    "\n",
    "# launch the notebook\n",
    "ipython notebook --port=$NOTEBOOKPORT --no-browser\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a script, say \"tunnel_notebook\", with the following contents at your laptop"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```bash\n",
    "ssh -f -N -L 8181:localhost:8181 username@comet-ln$1.sdsc.edu\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Open a terminal, and run \"bash tunnel_notebook login_node\" , here login_node is a number. For instance, [username@comet-ln3 ~], login_node=3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Open ur favorite web browser, e.g Chrome.  Type \"localhost:8181\" in the address. It will open the jupyter notebook that you do data analysis on TSCC."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Running Jupyter notebook on Cori"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Go go \"https://jupyter-dev.nersc.gov\" to login and use the notebook"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}