sp-m0-how-to-start.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Speech Processing Labs: How to use these notebooks\n",
    "\n",
    "This repository includes lab notebooks for the University of Edinburgh course: Speech Processing (LASC11158/LASC10061). You should work through these notebooks in the weekly lab sessions of the course. You'll get the most out of these notebooks if you watch the videos associated with the notebook topics (on [speech.zone](https://speech.zone/courses/speech-processing/)). \n",
    "\n",
    "The notebooks were primarily developed for the [signal processing](./signals) modules of the course.  These have an interactive component that you can run directly in the notebook. We will also publish the lab exercises for the [phonetics/phonology](./phon) content here, but these require you to use some external tools (e.g. Praat). \n",
    "\n",
    "The TTS labs will focus on the first assignment, in which you will investigate the [Festival text-to-speech system](https://speech.zone/exercises/the-festival-text-to-speech-system/).  The ASR labs help you with the second assignment, where you will use HTK to [build a digit recognizer](https://speech.zone/exercises/digit-recogniser/). The details the TTS and ASR assignments can be found in the [Practical Exercises](https://speech.zone/exercises/) section of [speech.zone](https://speech.zone/courses/speech-processing/)). \n",
    "\n",
    "\n",
    "<div class=\"alert alert-success\" role=\"alert\">\n",
    "These notebooks have been updated for the 2023/24 academic year.\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1 Finding the notebooks\n",
    "\n",
    "You can find the latest version of the notebooks in this [github](https://guides.github.com/activities/hello-world/) repository: \n",
    "\n",
    "https://github.com/laic/uoe_speech_processing_course\n",
    "\n",
    "These notebooks are [Jupyter notebooks](https://jupyter.org). These provide an interactive way to run python code and write nicely formatted text using [Markdown](https://www.markdownguide.org/cheat-sheet/).  Once you've got your own copy of the notebooks you can use them to run the interactive bits and also make your own notes for your tutorials (or develop your own code if you like!)\n",
    "\n",
    "The notebooks are available in a public repository so you don't need a github account to access it, but you might like to get one anyway. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 Viewing the notebooks\n",
    "\n",
    "You can view the notebooks directly via the github link above by just browsing through the links.  However, this won't let you run the interactive bits or add your own notes,  and probably won't let you directly play the audio links. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3 Getting your own copy \n",
    "\n",
    "To get your own copy of the repository you will need to 'clone' it using git commands. How you do this will depend a little on how you want to run them.  Here are the two main options. \n",
    "\n",
    "### Run the notebooks online using Edina Noteable\n",
    "\n",
    "University of Edinburgh students have access to [Edina Noteable](https://www.ed.ac.uk/information-services/learning-technology/noteable): the university's online Jupyter Notebook server. This allows you to use Jupyter Notebooks, Python and all the extra packages we need through your browser (you don't need to install anything on your side).  \n",
    "\n",
    "<div class=\"alert alert-success\" role=\"alert\">\n",
    "If your internet connection is fairly fast and stable, using Edina Noteable for this is probably your best option (just remember to save any changes you make to your notebooks often!).\n",
    "</div>\n",
    "\n",
    "\n",
    "You can find lots of information on how to use Edina Noteable (including some videos) [here](https://noteable.edina.ac.uk/user_guide/#hide_ge_2).  \n",
    "\n",
    "As of 2023, you will need to access Noteable via Learn. \n",
    "\n",
    "How to get these notebooks on Noteable: \n",
    "\n",
    "* Start up Noteable either by:\n",
    "    * Go to the Learn page for this course and click on the \"Noteable LTI 1.3\" link\n",
    "    * Go to the following URL in your browser login in with your EASE username and password:\n",
    "      * https://noteable.edina.ac.uk/login\n",
    "      * **NOTE: As of 2023 You will need to be logged into Learn for this to work**. \n",
    "\n",
    "* Select Standard Notebook (if you're given an option) and click on the start button.\n",
    "\n",
    "* You'll then see the jupyter start directory, showing you links to the files in it (probably empty at this point). \n",
    "\n",
    "#### Cloning the repository ###\n",
    "\n",
    "We suggest getting the repository using the Noteable terminal interface so that you can easily get any updates to the notebooks in the repository. \n",
    "\n",
    "The terminal interface allows you to type in and run [unix shell commands](https://missing.csail.mit.edu/2020/course-shell/). To start the terminal click on the `new` drop down menu (found to the right of the +GitRepo button) and start a new terminal instance.\n",
    "\n",
    "You should then see a unix shell interface in the browser.  \n",
    "\n",
    "You can then get the repo by typing or copy/paste in the following command: \n",
    "\n",
    "```\n",
    "git clone https://github.com/laic/uoe_speech_processing_course\n",
    "```\n",
    "\n",
    "This basically tells the computer to get the files at that URL and make a copy in your current directory. \n",
    "\n",
    "To get updates, you'll need open terminal and change directory `cd` to our notebook folder \n",
    "```\n",
    "cd uoe_speech_processing_course\n",
    "```\n",
    "You can see the files in that directory by using the `ls` command:\n",
    "```\n",
    "ls\n",
    "```\n",
    "When you're in the `uoe_speech_processing_course` directory, you can get updates by running the following in the terminal:\n",
    "```\n",
    "git pull\n",
    "```\n",
    "\n",
    "Hopefully there won't be too many updates for you to get, but it's good to get used to using version control systems (particularly git), especially if you're planning on doing computational things in the future (e.g. an MSc SLP dissertation!). Most open-source code is shared through github nowadays. Once you've got the hang of it, it can also be a lot easier to use the terminal to do a lot of things, e.g. organize your files using shell commands.\n",
    "\n",
    "We'll use the terminal more for the TTS and ASR labs.  But for the Signals labs you can now close the terminal window and go back to the notebook page.  You should see the `uoe_speech_processing_course` appear in your files (refresh the page if not)\n",
    "\n",
    "\n",
    "\n",
    "<div class=\"alert alert-warning\" role=\"alert\">\n",
    "If you're using Noteable, you might as well just use it through your normal computer browser (i.e. not through the remote desktop).  It'll save your computer a little bit of work. \n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running Jupyter Notebooks on your own computer\n",
    "\n",
    "You can also run Jupyter Notebooks locally on your own computer. Once you have everything installed, you should be able to go through all the lab materials offline. To do this, you'll need to have a few things installed: \n",
    "\n",
    "* Python 3.x (e.g. Python 3.8 should be fine, we last tested with Python 3.9)\n",
    "* Jupyter notebook\n",
    "* numpy\n",
    "* matplotlib\n",
    "* librosa\n",
    "  \n",
    "If you already have Python 3 installed, you could just use `pip` to get the rest.  But, I'd recommend using some version of anaconda (e.g. miniconda, the lightweight version of anaconda) because it will be useful for many other things later.\n",
    "\n",
    "#### Note for Windows Users\n",
    "\n",
    "Most of these instructions assume that you're using a unix like environment (i.e. the bash shell).  If you're using a Windows machine, you'll need to use a shell emulator.  A comprehensive one to try is [Ubuntu WSL](https://ubuntu.com/wsl), which basically allows you to run Ubuntu linux (a unix variant) on Windows. \n",
    "\n",
    "You can try the bash emulator that comes with [Git for Windows](https://gitforwindows.org/).  This also comes with graphical user interface for git which you might find handy.\n",
    "\n",
    "If you use this, you can try these [instructions for installing bash, git, conda and python on windows](https://www.earthdatascience.org/workshops/setup-earth-analytics-python/setup-git-bash-conda/).\n",
    "\n",
    "#### Install Python 3.9 and Miniconda\n",
    "\n",
    "Download the Miniconda 3.9 installer (Other newer versions of python should work too). You'll need to choose the appropriate version for your operating system (Window, MacOs or Linux) and the number of bits your CPU uses (64 or 32).  \n",
    "\n",
    "* https://docs.conda.io/en/latest/miniconda.html\n",
    "\n",
    "\n",
    "You can find installation instructions for different operating systems (e.g. Windows) here:\n",
    "* https://conda.io/projects/conda/en/latest/user-guide/install/index.html\n",
    "\n",
    "\n",
    "#### Basic instructions for installing miniconda on the unix shell:  \n",
    "\n",
    "Once you've downloaded the installer, open up a terminal and go to the directory you installed it into (probably `Downloads`).  \n",
    "\n",
    "Run the following command to check the downloaded file isn't corrupted.  You can check that the string it returns matches SHA256 hash listed next to the download link you used. \n",
    "```\n",
    "sha256sum ./Miniconda3-latest-Linux-x86_64.sh\n",
    "```\n",
    "\n",
    "Change the permissions on the Miniconda installer so you can run it ('u' for user, 'x' for execute): \n",
    "```\n",
    "chmod u+x ./Miniconda3-latest-Linux-x86_64.sh\n",
    "```\n",
    "\n",
    "And then run the installer script: \n",
    "\n",
    "```\n",
    "./Miniconda3-latest-Linux-x86_64.sh\n",
    "```\n",
    "The installer will ask you a bunch of questions (agree to the license etc) which you basically have to say yes to if you want to use Miniconda.  You can also choose where it installs Miniconda, but usually the default location (in your home directory) is fine.\n",
    "\n",
    "####  Create a conda environment\n",
    "\n",
    "You'll then need to close and reopen your terminal.  Now run the following command to create a new Python 3.8 environment called `slp` (actually you can call it whatever you want!). \n",
    "\n",
    "```\n",
    "conda create -n slp python=3.9\n",
    "```\n",
    "Now, activate the environment. \n",
    "```\n",
    "conda activate slp\n",
    "```\n",
    "Doing this basically means that you're telling the shell to use this version of python instead of the default version of python on your computer.  Activating the environment also means you can download python packages in a systematic way and you (probably) won't run into permission problems on computers where you don't have administrator access.  \n",
    "\n",
    "In general, it's good practice you use python enviroments for your projects so that you can keep track of dependencies.  New updates to python packages can sometimes mess up how old code works, so if you want someone else to run your code, having a specific conda environment means someone  else (maybe even your future self?) can easily recreate the conditions in which your code was created on a different machine. \n",
    "\n",
    "#### Install Jupyter Notebooks and other dependencies\n",
    "\n",
    "\n",
    "Now we need can use conda to install a bunch of stuff, i.e. jupyter notebooks:\n",
    "```\n",
    "conda install -c conda-forge  notebook numpy matplotlib ipython ffmpeg\n",
    "```\n",
    "\n",
    "Now we can finally startup jupyter notebooks! Note you need to be in the slp environment we just created for this to work (i.e. run `conda activate slp` before this). \n",
    "```\n",
    "jupyter notebook\n",
    "```\n",
    "\n",
    "This command starts a local notebook server and will give you a link to it. Open that link in your browser and you're all set!  \n",
    "\n",
    "Note: You'll need to get the notebook repository from github.  First, you'll need to [install git](https://git-scm.com/downloads). After that, go to the directory you want to download the notebooks into and run the following command:\n",
    "```\n",
    "git clone https://github.com/laic/uoe_speech_processing_course\n",
    "```\n",
    "You should now see a directory call `uoe_speech_processing_course` in the directory where you called the `git clone` command. \n",
    "\n",
    "You can also potentially use a [github desktop user interface](https://desktop.github.com) to clone the repository. \n",
    "\n",
    "\n",
    "#### Jupyter Notebooks official install info:\n",
    "\n",
    "* https://jupyter.org/install\n",
    "* https://jupyter.readthedocs.io/en/latest/running.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Downloading a zip file of the repo\n",
    "\n",
    "You can also download the repository as zip file from the github page.  However, we'd recommend you either use Edina Noteable or take a minute and figure out how to use the git clone command as this will make it easier to get an updates to the repository later.   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4 Running the Notebooks in Interactive Mode\n",
    "\n",
    "\n",
    "As mentioned above, once you've got your notebooks on Edina Noteable, clicking on an .ipynb file should start running it. \n",
    "\n",
    "If want to run a notebook server locally on your own machine, you'll have to start it up on the command line:\n",
    "\n",
    "```\n",
    "jupyter notebook\n",
    "```\n",
    "   Amongst the terminal output you should see a (localhost) link which you can then go to in your browser. \n",
    "   \n",
    "You can then navigate to the `uoe_speech_processing_course` directory from that browser page and start up a notebook. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Some Very Basic Jupyter Notebook Commands\n",
    "\n",
    "You definitely don't need to be a Jupyter expert to use these notebooks for this course. You basically just need to run every cell in turn! Here's the very bare bones of what you need to know.\n",
    "\n",
    "#### Cells: \n",
    "\n",
    "Jupyter notebooks have two types of cells: \n",
    "\n",
    "* **Code cells:** i.e. code you can actually run \n",
    " \n",
    "* **Markdown cells:** html-ish writing cells, where you can write your notes (like this one you're reading now).\n",
    "    * You can include various types of formatting: [Markdown reference](https://wordpress.com/support/markdown-quick-reference/)\n",
    "    * You can also write equations in [latex math mode](https://towardsdatascience.com/write-markdown-latex-in-the-jupyter-notebook-10985edb91fd)\n",
    "\n",
    "After you've written something in a cell, you can run it by clicking the _Run_ button on the menu bar. \n",
    "\n",
    "<div class=\"alert alert-warning\" role=\"alert\">\n",
    "You don't have to run the cells in the order they appear on the page.  When you run a code cell the variables set there will retain those values for the next cell you run.  This means that variables are updated in the order that you run the cells, not necessarily the order they appear on the page!  \n",
    "</div>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Try it out! \n",
    "\n",
    "If you're already in interactive mode (i.e. viewing this very jupyter notebook on a jupyter server rather than on github.com) you can try the following code cell:\n",
    "\n",
    "* Click on the next cell\n",
    "* Edit the python code to print out your own name instead of `YOUR_NAME_HERE`\n",
    "* Press Shift-Enter to run the code \n",
    "* You should see the output directly under the cell\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Hello! My name is YOUR_NAME_HERE\")\n",
    "a = 793  ## This is a comment\n",
    "b = 13   ## You could also change this value and see a change in the second sentence printed out\n",
    "print(\"Did you know that %d + %d = %d?\" % (a, b, a+b))\n",
    "print(\"Did you know that {} + {} = {}?\".format(a, b, a+b)) #python 3 format()\n",
    "print(f\"Did you know that {a} + {b} = {a+b}?\") #python 3 f-string"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, Run the next code cell a few times to to see that the value of the variable `a` gets updated every time you run the cell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = a + b\n",
    "print(\"Did you know that %d + %d = %d?\" % (a, b, a+b))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Double click on this _markdown_ cell to edit it! \n",
    "* (**WRITE SOMETHING HERE**)\n",
    "* Then press `Shift-Enter` to run the cell and finishing editing it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Basic keyboard shortcuts\n",
    "\n",
    "As you go through a notebook, you'll probably soon find yourself wanting to use the keyboard shortcuts. Here are  the most common ones: \n",
    "\n",
    "* Click on a code cell to edit it \n",
    "* Double click on a rendered Markdown cell to edit it\n",
    "* `Ctrl-Enter`: Run the current cell (this is Cmd-Enter on a Mac)\n",
    "* `Shift-Enter`: Run the current cell and move to the next\n",
    "    * Useful if you're moving through a notebook (which is what you'll likely be doing)\n",
    "* `Alt-Enter`: Run the current cell and insert another after it\n",
    "    * Useful if you're writing a notebook! \n",
    "\n",
    "If you click on the small keyboard icon on the menu bar (next to the cell type), you'll see many more command short cuts. \n",
    "\n",
    "**Note**: You'll need to double click on a markdown cell to edit it (if it's already been run).\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Try it out! \n",
    "\n",
    "* Select one of the code cells above and type `shift-enter` on the keyboard to run that cell and move onto the next\n",
    "* Select a cell above and type `alt-enter` to run the cell and add another (code) cell below it\n",
    "* To delete an unwanted cell, first click outside of the code box or press the `Esc` key, and then press the `x` key (you can also press the `z` key to undo an action!)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#### Select this code cell and type 'Esc' then 'm' to turn it into a Markdown cell\n",
    "#### Select this markdown cell and type 'Esc' then 'y' to turn it back into a code cell"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### More on how to use Jupyter Notebooks \n",
    "\n",
    "There are many tutorials about using Jupyter Notebooks on the internet.  If you've never used them before, it's a good idea to go through at least one to get used to how they work.  \n",
    "\n",
    "* [A very quick overview from the Jupyter docs](https://notebooks.gesis.org/binder/jupyter/user/ipython-ipython-in-depth-r9ntjg0c/notebooks/examples/Notebook/Notebook%20Basics.ipynb)\n",
    "* [An interactive tutorial from the Binder project that's made up of Jupyter notebooks](https://notebooks.gesis.org/binder/jupyter/user/ipython-ipython-in-depth-koh48p7z/notebooks/binder/Index.ipynb)\n",
    "* [Jupyter docs](https://jupyter-notebook.readthedocs.io/en/stable/notebook.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5  Updating the repository\n",
    "\n",
    "\n",
    "If you've already cloned the repository somewhere, you can get the lastest version by going to the `uoe_speech_processing_course` directory in a terminal \n",
    "and typing in the following command: \n",
    "\n",
    "```\n",
    "git pull\n",
    "```\n",
    "\n",
    "To update the repo in Edina Noteable you'll have to use the terminal interface (see above).\n",
    "\n",
    "There are also some more [instructions and links here](https://speech.zone/forums/topic/getting-updated-versions-of-the-notebook-repository-from-github/), if you want to stash your own changes before getting updates.  \n",
    "\n",
    "\n",
    "<div class=\"alert alert-warning\" role=\"alert\">\n",
    "This may cause merge conflicts (i.e., your version local clashes with the new version)  if you've modified and saved the notebooks already (which you should be doing!).  We'll try to avoid this by making sure we create notebooks with different names if we update one you might already have worked on.  In practice, though, merging and resolving conflicts is a very normal thing in software engineering when you're working with other people.  You can read more about updating repositories in the <a href=\"https://docs.github.com/en/github/using-git/getting-changes-from-a-remote-repository\">github documentation</a>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6 More info on git (and github)\n",
    "\n",
    "Nowadays pretty much everyone uses git, often via github, for sharing and tracking code (i.e. version control). We won't really use it beyond hosting this repository in this course.  However, knowing some basic git is really quite essential for working with code these days so it's worth spending some time learning a bit about this.  \n",
    "\n",
    "* [The github guide to git](https://guides.github.com/introduction/git-handbook/)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7 What should I do in the labs? \n",
    "\n",
    "The goal of these labs is to get you to think through the concepts introduced in the lecture videos.  \n",
    "\n",
    "The _phon_ and _signals_ labs generally aren't about coding. For notebooks that have python code included, you don't need to understand all the workings of the code and you definitely won't need to reproduce it. However, we do want you to get some exposure to how these abstract concepts get translated into concrete programming (and do  something a bit interactive with the computer and each other!).\n",
    "\n",
    "The _TTS_ and _ASR_ labs are time for you to work on the TTS and ASR assignments.  So, they will have more of a coding component (generally bash shell scripting).   \n",
    "\n",
    "The main tasks for the labs are these:\n",
    "\n",
    "* Go through the notebooks and try out the exercises.  You may want/need to rewatch some of the videos/lecture material for that week too or check the readings. \n",
    "* If you get stuck at any point, ask for help from the tutors (or your classmates!)\n",
    "* Discuss the material with other people in the class and your tutors\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8 Help! \n",
    "\n",
    "People taking this course come from lots of different backgrounds. If you run into trouble or this is all a bit much, ask for help! You can ask questions in the labs, the lecture, as well as the forum on the [speech zone website](http://speech.zone/courses/speech-processing/).  The speech.zone forum has questions and answers from many previous years of the course, so you may find that your question has already been answered there. If you want some more 1-1 help, you can always book an appointment with the course lecturers. \n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## A spare code cell! "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}