lectures/week_01/bfx_workshop_01_overview.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BFX Workshop Week 01\n",
    "\n",
    "Welcome to the Bioinformatics (BFX) Workshop, a.k.a Applied Genomics for Bioinformatics I. In order to get the most out of this course, you're going to need some things:\n",
    "\n",
    "- a reasonably modern computer with some tools/programs installed\n",
    "- basic Unix command line skills\n",
    "\n",
    "This document describes setup instructions for MacOSX or Linux. If you're using Windows, go to the [other notebook](bfx_workshop_01_overview_windows.ipynb). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Software you'll need to have installed:\n",
    "\n",
    "There are numerous ways to install Python, R, Notebooks, etc. In fact, you will see various ways of installing software and configuring your environment throughout the workshop. The sections below describe a path that assumes you are starting from scratch.  If you have conda installed already, for example, then you'll need to adjust accordingly. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Miniconda\n",
    "Conda is a package manager that works on Windows, Linux, and macOS. We will use a lightweight version of the package manager called Miniconda to install packages and manage the system environment. This tool will help minimize dependencies and differences in system-level libraries across the participants in the BFX Workshop.\n",
    "\n",
    "If you're installing from scratch, just choose the [latest available installer for your OS from the Miniconda page](https://docs.anaconda.com/miniconda/miniconda-install/)\n",
    "\n",
    "Once you've complete the install process, open a fresh terminal and check that conda is now installed by listing the Help menu:\n",
    "\n",
    "```\n",
    "conda -h\n",
    "```\n",
    "\n",
    "An example image of what a terminal might look like when running the above command:\n",
    "![Example Conda Help](https://github.com/genome/bfx-workshop/raw/master/archive/v2020-2021/images/conda_help.png)\n",
    "\n",
    "(With a lot more lines underneath, providing info on specific parameters and subcommands)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Jupyter\n",
    "Jupyter is a web-based interactive computing platform that allows users to create and run \"notebooks\" that mix code and data. We'll be exploring some details below (and in future workshop sessions), but for now, just get it set up. Other installation methods will work, but for the sake of simplicity, we will install using conda like so:\n",
    "\n",
    "```\n",
    "conda install jupyter\n",
    "```\n",
    "\n",
    "Check to see that a recent version of Python3 is now used in your base conda environment:\n",
    "\n",
    "```\n",
    "python -V\n",
    "```\n",
    "\n",
    "As of Fall 2024, the base conda Python version is 3.12.4. See the example image below from 2020 (v3.8.3):\n",
    "\n",
    "![Example Python Version](https://github.com/genome/bfx-workshop/raw/master/archive/v2020-2021/images/python_version.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Docker\n",
    "Docker is a system that allows you to launch \"containers\" on your computer that contain alternate operating systems and all the dependencies for specific tools. We'll discuss this in detail during the course. \n",
    "\n",
    "For now, install it using the [instructions for installing Docker for Mac](https://docs.docker.com/docker-for-mac/install/). That page's sidebar also contains links to Linux instructions\n",
    "\n",
    "Once it's installed, launch the docker app from your Applications folder (if it isn't already running), and then confirm that it's installed using:\n",
    "\n",
    "```\n",
    "docker --version\n",
    "```\n",
    "\n",
    "Then make sure it can download and run images using: \n",
    "\n",
    "```\n",
    "docker run hello-world\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Java\n",
    "Java is required to use the Integrated Genomics Viewer (IGV) locally on your workstation or laptop. Please see the operating system install instructions for Java located [here](https://www.java.com/en/download/help/download_options.xml).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### R\n",
    "\n",
    "We'll be using the R Studio framework in most sessions that involve creating and running R code. Please follow the instructions for [installing R and R Studio here](https://posit.co/download/rstudio-desktop/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Git\n",
    "\n",
    "Git is a a version-control program for source code. On MacOSX, the easiest way is generally to install mac developer tools using:\n",
    "\n",
    "```\n",
    "xcode-select --install\n",
    "```\n",
    "\n",
    "Most linux installs will already include git, and if they don't, [there are detailed instructions here for installing Git on most platforms](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) \n",
    "\n",
    "After doing so, you can verify that it's installed by running:\n",
    "\n",
    "```\n",
    "git --version\n",
    "```\n",
    "\n",
    "If you want to become more familiar with Git and version control for code, you can check out the [Linkedin Learning Git and Github module](https://www.linkedin.com/learning-login/share?account=57884865&forceAccount=false&redirect=https%3A%2F%2Fwww.linkedin.com%2Flearning%2Flearning-git-and-github-14213624%3Ftrk%3Dshare_ent_url%26shareId%3DBNNkL8hRQAqPOHmqiNOyZg%253D%253D) which should be available freely on campus.\n",
    "\n",
    "In the meantime, let's start simple and clone the class git repository from GitHub. Open a terminal, which will start in your home directory. Make a directory called `workshop` with the following command:\n",
    "\n",
    "```mkdir workshop```\n",
    "\n",
    "Use `cd` to move into the `workshop` directory you just created and then clone the [course repository](https://github.com/genome/bfx_workshop).\n",
    "\n",
    "![Example GitHub Clone/Download Code](https://github.com/genome/bfx-workshop/raw/master/archive/v2020-2021/images/github_clone_repo.png)\n",
    "\n",
    "NOTE: Pull down the \"Code\" button to get the path of the class repository. Then use `git clone PATH` (replacing PATH) to grab your copy.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Jupyter Notebook\n",
    "\n",
    "Now that we have Miniconda, Jupyter, and Git installed and functional, we can begin using the Jupyter Notebook as an interactive shell and development environment.\n",
    "\n",
    "First we should navigate the filesytem, using `cd`, to move to the directory where we cloned the course repository.\n",
    "\n",
    "A list, or `ls`, of that directory should show at least a `README.md` file and an `lectures` directory.\n",
    "\n",
    "From the terminal command line, in the git repo directory, start a Jupyter Notebook. The following command should launch a browser window showing the contents of that directory. \n",
    "\n",
    "```\n",
    "jupyter notebook\n",
    "```\n",
    "\n",
    "From there, you can navigate to lectures/week_01, and click on the bfx_workshop_01_overview.ipynb file to launch *THIS* tutorial in an interactive browser session.  \n",
    "\n",
    "At this point, **switch from viewing the notebook on github to following it here in the interactive notebook you just launched.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### iPython and Shell Commands\n",
    "\n",
    "Many common shell commands can be executed directly through a Notebook using iPython\n",
    "For more on iPython, see this [link](https://jakevdp.github.io/PythonDataScienceHandbook/01.05-ipython-and-shell-commands.html).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/cmiller\n"
     ]
    }
   ],
   "source": [
    "cd ~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mDesktop\u001b[m\u001b[m/              \u001b[34mannotations\u001b[m\u001b[m/          notes.txt\r\n",
      "\u001b[34mDocuments\u001b[m\u001b[m/            \u001b[34mbbedit\u001b[m\u001b[m/               \u001b[34mopt\u001b[m\u001b[m/\r\n",
      "\u001b[34mDownloads\u001b[m\u001b[m/            \u001b[34mLibrary\u001b[m\u001b[m/              \u001b[34mbin\u001b[m\u001b[m/\r\n",
      "\u001b[34mMovies\u001b[m\u001b[m/               \u001b[34mc.a.miller_s1_backup\u001b[m\u001b[m/ \u001b[34mscreenshots\u001b[m\u001b[m/\r\n",
      "\u001b[34mMusic\u001b[m\u001b[m/                \u001b[34mgit\u001b[m\u001b[m/                  \u001b[34msrc\u001b[m\u001b[m/\r\n",
      "\u001b[34mPictures\u001b[m\u001b[m/             \u001b[34migv\u001b[m\u001b[m/                  \u001b[35mtmp\u001b[m\u001b[m@\r\n",
      "\u001b[34mPublic\u001b[m\u001b[m/               \u001b[34mimages\u001b[m\u001b[m/               \u001b[34mworkshop\u001b[m\u001b[m/\r\n"
     ]
    }
   ],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "mkdir -p ~/tempdir/test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mtest\u001b[m\u001b[m/\r\n"
     ]
    }
   ],
   "source": [
    "ls ~/tempdir/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "rmdir ~/tempdir/test; rmdir ~/tempdir"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### iPython and Shell Commands\n",
    "\n",
    "Many common shell commands can be executed directly through a Notebook using iPython\n",
    "For more on iPython, see this [link](https://jakevdp.github.io/PythonDataScienceHandbook/01.05-ipython-and-shell-commands.html).  For now, let's try a simple shell command to print text to the screen:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-10-1c643a414f93>, line 1)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-10-1c643a414f93>\"\u001b[0;36m, line \u001b[0;32m1\u001b[0m\n\u001b[0;31m    echo \"Hello World\"\u001b[0m\n\u001b[0m         ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "echo \"Hello World\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That didn't work, let's try preceding our command with an exclamation point:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello World\r\n"
     ]
    }
   ],
   "source": [
    "!echo \"Hello World\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's enough for now - we'll talk more about jupyter notebooks and their usage later in the course. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unix command line\n",
    "\n",
    "You've set up software on your system now, and run a few commands, but in order to complete this course, you'll need to get very familiar with the command line: working at a shell and running standard commands for navigating a filesystem, performing tasks on files and directories, etc. This workshop will offer a *brief* introduction to the unix shell in Week 2, but you will not be able to succeed in this course without developing some strong command line skills.\n",
    "\n",
    "If you are not comfortable at the command line right now, that's okay - there's still time! \n",
    "\n",
    "**If you're new to this, start by working through the [Terminal Basics tutorial](https://sandbox.bio/tutorials?id=terminal-basics) at sandbox.bio this week**.  Next week, we'll dive further into the command line and present more resources that you can work through to build a solid foundation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Other Resources\n",
    "\n",
    "### Compute and Storage Access\n",
    "\n",
    "Almost all of the modules in this course will be able to be run on your local computer, using small data sets. If you wanted to, you could also run them on the local high-performance cluster or on the cloud. These resources are not required right now, but worth becoming familiar with, especially as you begin to extend your knowledge to running tools on your own (probably much larger) data sets.\n",
    "\n",
    "### Google Cloud\n",
    "\n",
    "* [WUIT Google Cloud](https://it.wustl.edu/services/cloud-computing/google-cloud-platform/)\n",
    "* [Google Cloud Console](https://console.cloud.google.com/)\n",
    "* The [Division of Oncology has some guidelines and code](https://github.com/wustl-oncology) that may help with getting up and running \n",
    "\n",
    "### WashU Local Compute and Storage\n",
    "\n",
    "* There is training available through Becker Library in September covering many basics of computing on the local research cluster. Topics include Open On Demand, Command Line, High Performance Computing, and the RIS Scientific Compute Platform.  [Details and Registration for these upcoming workshops are here](https://becker.wustl.edu/services/research-computing/)\n",
    "\n",
    "* WUIT's Research Infrastructure Services (RIS) supported [Scientific Compute Platform](https://ris.wustl.edu/services/compute/) provides info on getting started with the compute cluster and storage, both linked from their homepage, and with [more details on their documentation page](https://docs.ris.wustl.edu/).\n",
    "\n",
    "* RIS has done their own trainings, and you can [view their archived videos and resources](https://docs.ris.wustl.edu/doc/compute/compute-workshops.html)\n",
    "\n",
    "* If you're looking to get started on submitting jobs to the cluster, working through this [LSF and Docker tutorial](https://gist.github.com/chrisamiller/4b17a8dd310374f078da2bf12b3e2a49) might prove useful.\n",
    "\n",
    "*  In order to connect to the VPN, you'll either need to be connected to WUSM-Secure (on campus), or logged into the VPN. [The VPN section of this page](https://it.wustl.edu/items/connect/) has details. \n",
    "\n",
    "\n",
    "### Unix Skills\n",
    "\n",
    "If you can't wait to get started with the command line (or think you might need extra time) you can jump ahead to the [Week 2 notes](https://github.com/genome/bfx-workshop/blob/master/lectures/week_02/) and get started working through the exercises, or peruse the other resources linked there. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.19"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}