lab4_exercises_ANSWERS.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Programming Bootcamp 2016\n",
    "# Lesson 4 Exercises - ANSWERS\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 1. Guess the output: list practice (1pt)\n",
    "\n",
    "For the following blocks of code, first try to guess what the output will be, and then run the code yourself. Points will be given for filling in the guesses; guessing wrong won't be penalized."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n"
     ]
    }
   ],
   "source": [
    "ages = [65, 34, 96, 47]\n",
    "\n",
    "print len(ages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    }
   ],
   "source": [
    "names = [\"Wilfred\", \"Manfred\", \"Wadsworth\", \"Jeeves\"]\n",
    "ages = [65, 34, 96, 47]\n",
    "\n",
    "print len(ages) == len(names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "65\n",
      "34\n",
      "96\n",
      "47\n"
     ]
    }
   ],
   "source": [
    "ages = [65, 34, 96, 47]\n",
    "\n",
    "for hippopotamus in ages:\n",
    "    print hippopotamus"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[34, 96]\n"
     ]
    }
   ],
   "source": [
    "ages = [65, 34, 96, 47]\n",
    "\n",
    "print ages[1:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Wilfred', 'Manfred', 'Wadsworth', 'Jeeves', 'Willard']\n"
     ]
    }
   ],
   "source": [
    "names = [\"Wilfred\", \"Manfred\", \"Wadsworth\", \"Jeeves\"]\n",
    "\n",
    "if \"Willard\" not in names:\n",
    "    names.append(\"Willard\")\n",
    "print names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wilfred is 65\n",
      "Manfred is 34\n",
      "Wadsworth is 96\n",
      "Jeeves is 47\n"
     ]
    }
   ],
   "source": [
    "names = [\"Wilfred\", \"Manfred\", \"Wadsworth\", \"Jeeves\"]\n",
    "ages = [65, 34, 96, 47]\n",
    "\n",
    "for i in range(len(names)):\n",
    "    print names[i],\"is\",ages[i]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[34, 47, 65, 96]\n"
     ]
    }
   ],
   "source": [
    "ages = [65, 34, 96, 47]\n",
    "\n",
    "ages.sort()\n",
    "print ages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "ages = [65, 34, 96, 47]\n",
    "\n",
    "ages = ages.sort()\n",
    "print ages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Remember that `.sort()` is an in-place function. Its return value is \"None\". (This is a special value in Python that basically means \"null\". It's used as a placeholder sometimes when we don't want to give something a value.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "96\n"
     ]
    }
   ],
   "source": [
    "ages = [65, 34, 96, 47]\n",
    "\n",
    "print max(ages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "M\n",
      "i\n",
      "t\n",
      "s\n",
      "w\n",
      "o\n",
      "r\n",
      "t\n",
      "h\n"
     ]
    }
   ],
   "source": [
    "cat = \"Mitsworth\"\n",
    "\n",
    "for i in range(len(cat)):\n",
    "    print cat[i]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mits\n"
     ]
    }
   ],
   "source": [
    "cat = \"Mitsworth\"\n",
    "\n",
    "print cat[:4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Good', 'morning,', 'Mr.', 'Mitsworth.']\n",
      "Good morning, Mr. Mitsworth.\n"
     ]
    }
   ],
   "source": [
    "str1 = \"Good morning, Mr. Mitsworth.\"\n",
    "\n",
    "parts = str1.split()\n",
    "print parts\n",
    "print str1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Good morning', ' Mr. Mitsworth.']\n"
     ]
    }
   ],
   "source": [
    "str1 = \"Good morning, Mr. Mitsworth.\"\n",
    "\n",
    "parts = str1.split(\",\")\n",
    "print parts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Jeeves\n"
     ]
    }
   ],
   "source": [
    "names = [\"Wilfred\", \"Manfred\", \"Wadsworth\", \"Jeeves\"]\n",
    "\n",
    "print names[-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2, 6, 1]\n"
     ]
    }
   ],
   "source": [
    "oldList = [2, 2, 6, 1, 2, 6]\n",
    "newList = []\n",
    "\n",
    "for item in oldList:\n",
    "    if item not in newList:\n",
    "        newList.append(item)\n",
    "print newList"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> This is an example of how to remove duplicates."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 2. On your own: Lists (3pts)\n",
    "\n",
    "Write code to accomplish each of the following tasks **using list functions**. Do not copy and paste the list to make changes. ***You must pretend you don't know what's in the list.***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# run this first!\n",
    "geneNames = [\"Ppia\", \"Gria2\", \"Mecp2\", \"Omd\", \"Zfp410\", \"Hsp1\", \"Mtap1a\", \"Cfl1\", \n",
    "             \"Slc25a40\", \"Dync1i1\", \"V1ra4\", \"Fmnl1\", \"Mtap2\", \"Atp5b\", \"Olfr259\", \n",
    "             \"Atf3\", \"Vapb\", \"Dhx8\", \"Slc22a15\", \"Orai3\", \"Ifitm7\", \"Kcna2\", \"Timm23\", \"Shank1\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(A)** (1pt) Replace the 12th element in `geneNames` with the string `\"Camk2a\"`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "geneNames[11] = \"Camk2a\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(B)** (1pt) Add the string `\"Shank3\"` to the end of the `geneNames` list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "geneNames.append(\"Shank3\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(C)** (1pt) Using a loop, print the elements in `geneNames` that start with \"S\" or \"O\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Omd\n",
      "Slc25a40\n",
      "Olfr259\n",
      "Slc22a15\n",
      "Orai3\n",
      "Shank1\n",
      "Shank3\n"
     ]
    }
   ],
   "source": [
    "for gene in geneNames:\n",
    "    firstChar = gene[0]\n",
    "    if firstChar == \"S\" or firstChar == \"O\":\n",
    "        print gene"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 3. On your own: Strings & Splitting (3pts)\n",
    "\n",
    "Write code to accomplish each of the following tasks **using string slicing and splitting**. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(A)** (1pt) Print characters 3-9 of the string stored in the variable `magicWords` using string slicing. (You should get **\"racadab\"**)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "racadab\n"
     ]
    }
   ],
   "source": [
    "magicWords = \"abracadabra\"\n",
    "print magicWords[2:9]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(B)** (1pt) Prompt the user to input a sentence using `raw_input()`. Print each word of their sentence on a separate line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input a sentence: No, you input a sentence!\n",
      "No,\n",
      "you\n",
      "input\n",
      "a\n",
      "sentence!\n"
     ]
    }
   ],
   "source": [
    "sentence = raw_input(\"Input a sentence: \")\n",
    "splitSentence = sentence.split()\n",
    "for word in splitSentence:\n",
    "    print word"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(C)** (1pt) Using `raw_input()`, prompt the user to enter some good ideas for cat names (have them enter the names on one line, separating each name with a comma). Separate the names so that each one is a separate element of a list. Then choose one randomly (using the random module) and print it out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input some cat names (separate by comma): Mittens, Tatertot, Meatball\n",
      " Tatertot\n"
     ]
    }
   ],
   "source": [
    "import random\n",
    "\n",
    "catNamesStr = raw_input(\"Input some cat names (separate by comma): \")\n",
    "catNamesList = catNamesStr.split(\",\")\n",
    "\n",
    "randNum = random.randint(0, len(catNamesList)-1)\n",
    "print catNamesList[randNum]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Note: if you want to strip off the extra whitespace, you can do this:\n",
    "```\n",
    "print catNamesList[randNum].strip(\" \")\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 4. File reading and lists (3pts)\n",
    "\n",
    "For this problem, use the file `genes.txt` provided on Piazza. It contains some gene IDs, one on each line."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(A)** (1pt) Read the file and print to the screen only the gene IDs that contain \"uc007\".\n",
    "\n",
    "**[ Check your answer ]** You should end up printing 14 gene IDs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "uc007zzs.1\n",
      "uc007hnl.1\n",
      "uc007hnl.1\n",
      "uc007xgk.1\n",
      "uc007shu.4\n",
      "uc007ztg.2\n",
      "uc007nkt.1\n",
      "uc007piu.2\n",
      "uc007ker.1\n",
      "uc007vlq.1\n",
      "uc007xog.1\n",
      "uc007kmj.1\n",
      "uc007cib.1\n",
      "uc007ket.1\n"
     ]
    }
   ],
   "source": [
    "inFile = \"genes.txt\"\n",
    "ins = open(inFile, 'r')\n",
    "for line in ins:\n",
    "    line = line.rstrip('\\n')\n",
    "    if \"uc007\" in line:\n",
    "        print line\n",
    "ins.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(B)** (1 pt) Print to the screen only unique gene IDs (remove the duplicates). Do not assume repeat IDs appear consecutively in the file. \n",
    "\n",
    "*Hint: see problem 1 for an example of duplicate checking.*\n",
    "\n",
    "**[ Check your answer ]** You should end up printing 49 gene IDs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "uc007zzs.1\n",
      "uc009akk.2\n",
      "uc009eyb.1\n",
      "uc008vlv.1\n",
      "uc008wzq.1\n",
      "uc007hnl.1\n",
      "uc008tvu.1\n",
      "uc008vlv.3\n",
      "uc007xgk.1\n",
      "uc009qsh.1\n",
      "uc008all.1\n",
      "uc008eda.1\n",
      "uc007shu.4\n",
      "uc009mor.1\n",
      "uc008fux.1\n",
      "uc007ztg.2\n",
      "uc007nkt.1\n",
      "uc008qul.3\n",
      "uc008ktr.2\n",
      "uc008iwn.1\n",
      "uc009fxp.2\n",
      "uc008vsh.1\n",
      "uc008gkj.2\n",
      "uc007piu.2\n",
      "uc008vsk.1\n",
      "uc008vsv.1\n",
      "uc008kjh.1\n",
      "uc009dri.1\n",
      "uc008vlv.2\n",
      "uc009rxy.1\n",
      "uc008fyq.1\n",
      "uc009act.1\n",
      "uc008lub.1\n",
      "uc007ker.1\n",
      "uc008qiz.2\n",
      "uc008bak.1\n",
      "uc008kcg.1\n",
      "uc009cjg.1\n",
      "uc007vlq.1\n",
      "uc007xog.1\n",
      "uc009avv.1\n",
      "uc008kcg.2\n",
      "uc007kmj.1\n",
      "uc008oaj.1\n",
      "uc007cib.1\n",
      "uc007ket.1\n",
      "uc009rpf.1\n",
      "uc008owo.1\n",
      "uc008jaq.1\n"
     ]
    }
   ],
   "source": [
    "inFile = \"genes.txt\"\n",
    "geneList = []\n",
    "ins = open(inFile, 'r')\n",
    "for line in ins:\n",
    "    line = line.rstrip('\\n')\n",
    "    if line not in geneList: #keep track of what we've already seen using a list\n",
    "        geneList.append(line)\n",
    "        print line\n",
    "ins.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(C)** (1 pt) Print to the screen only the gene IDs that are still unique after removing the \".X\" suffix (where X is a number).\n",
    "\n",
    "**[ Check your answer ]** You should end up printing 46 gene IDs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "uc007zzs\n",
      "uc009akk\n",
      "uc009eyb\n",
      "uc008vlv\n",
      "uc008wzq\n",
      "uc007hnl\n",
      "uc008tvu\n",
      "uc007xgk\n",
      "uc009qsh\n",
      "uc008all\n",
      "uc008eda\n",
      "uc007shu\n",
      "uc009mor\n",
      "uc008fux\n",
      "uc007ztg\n",
      "uc007nkt\n",
      "uc008qul\n",
      "uc008ktr\n",
      "uc008iwn\n",
      "uc009fxp\n",
      "uc008vsh\n",
      "uc008gkj\n",
      "uc007piu\n",
      "uc008vsk\n",
      "uc008vsv\n",
      "uc008kjh\n",
      "uc009dri\n",
      "uc009rxy\n",
      "uc008fyq\n",
      "uc009act\n",
      "uc008lub\n",
      "uc007ker\n",
      "uc008qiz\n",
      "uc008bak\n",
      "uc008kcg\n",
      "uc009cjg\n",
      "uc007vlq\n",
      "uc007xog\n",
      "uc009avv\n",
      "uc007kmj\n",
      "uc008oaj\n",
      "uc007cib\n",
      "uc007ket\n",
      "uc009rpf\n",
      "uc008owo\n",
      "uc008jaq\n"
     ]
    }
   ],
   "source": [
    "inFile = \"genes.txt\"\n",
    "geneList = []\n",
    "ins = open(inFile, 'r')\n",
    "for line in ins:\n",
    "    line = line.rstrip('\\n')\n",
    "    splitLine = line.split(\".\") #could also do this using string slicing: line[:-2]\n",
    "    geneID = splitLine[0] #the splitting returns a list, and the geneID should be the first part\n",
    "    if geneID not in geneList: \n",
    "        geneList.append(geneID)\n",
    "        print geneID\n",
    "ins.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 5. Practice with `.split()` (4pts)\n",
    "\n",
    "Use `init_sites.txt` to complete the following. This file contains a subset of translation initiation sites in mouse, identified by Ingolia et al. (Cell, 2011). Note that this file has a header, which you will want to skip over."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(A)** (2 pt) Write a script that reads this file and computes the average CDS length (i.e. average the values in the 7th column).\n",
    "\n",
    "**[ Check your answer ]** You should get an average of 236.36."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "236.36\n"
     ]
    }
   ],
   "source": [
    "fileName = \"init_sites.txt\"\n",
    "totalLen = 0\n",
    "numLines = 0\n",
    "ins = open(fileName, 'r')\n",
    "ins.readline()\n",
    "for line in ins:\n",
    "    line = line.rstrip('\\n')\n",
    "    lineParts = line.split()\n",
    "    totalLen = totalLen + int(lineParts[6]) #all file input is read as string; must convert to int\n",
    "    numLines = numLines + 1\n",
    "print float(totalLen)/numLines\n",
    "ins.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(B)** (2 pt) Write a script that reads this file and prints the \"Init Context\" from each line (i.e. the 6th column) **if and only if** the \"Codon\" column (column 12) is \"aug\" for that line.\n",
    "\n",
    "**[ Check your answer ]** You should print 38 init contexts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CAGATGC\n",
      "AAGATGG\n",
      "GACATGG\n",
      "ACCATGA\n",
      "GACATGG\n",
      "GCTATGG\n",
      "AAGATGG\n",
      "ACCATGG\n",
      "GCTATGT\n",
      "GCCATGA\n",
      "ataatgg\n",
      "GAAATGG\n",
      "GCCATGG\n",
      "AACATGG\n",
      "GAAATGT\n",
      "ACTATGG\n",
      "AGAATGA\n",
      "GTCATGT\n",
      "GCCATGA\n",
      "AGAATGA\n",
      "AGAATGA\n",
      "AAGATGG\n",
      "AGGATGG\n",
      "GCCATGA\n",
      "GCCATGG\n",
      "ACCATGG\n",
      "AATATGT\n",
      "AAAATGG\n",
      "AAGATGG\n",
      "GTCATGC\n",
      "ACGATGC\n",
      "AGCATGG\n",
      "GCCATGG\n",
      "GCCATGT\n",
      "GCAATGG\n",
      "AGTATGT\n",
      "GCCATGA\n",
      "GACATGC\n"
     ]
    }
   ],
   "source": [
    "fileName = \"init_sites.txt\"\n",
    "ins = open(fileName, 'r')\n",
    "for line in ins:\n",
    "    line = line.rstrip('\\n')\n",
    "    lineParts = line.split()\n",
    "    if lineParts[11] == \"aug\":\n",
    "        print lineParts[5]\n",
    "ins.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(C)** (0 pt) For fun (?), copy and paste your output from (b) into http://weblogo.berkeley.edu/logo.cgi to create a motif logo of the sequence around these initiation sites. What positions/nt seem to be most common?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ATG at positions 4,5,6 (as expected, since this is the canonical initiation codon) and a preference for A/G at position 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 6. Cross-referencing (4pts)\n",
    "\n",
    "Here you will extract and print the data from `init_sites.txt` that corresponds to genes with high expression. There isn't gene expression data in `init_sites.txt`, so we'll have to integrate information from another file.\n",
    "\n",
    "- First, use `gene_expr.txt` to create a list of genes with high expression. We'll say high expression is anything >= 50. \n",
    "- Then read through `init_sites.txt` and print the GeneName (2nd column) and PeakScore (11th column) from any line that matches an ID in your high-expression list. \n",
    "- Finally, separately compute the average PeakScore for high expression genes and non-high expression genes. Print both averages to the screen.\n",
    "\n",
    "**[ Check your answer ]** There should be 10 lines corresponding to high-expression genes that you print info about. Your average peak scores should be 4.371 and 4.39325 for high and non-high expression genes, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pa2g4 \t4.54\n",
      "Pa2g4 \t4.54\n",
      "Rexo4 \t4.39\n",
      "Rexo4 \t4.39\n",
      "Psmc4 \t4.39\n",
      "Phb2 \t4.33\n",
      "Prune \t4.30\n",
      "Tmsb10 \t4.29\n",
      "Rhebl1 \t4.28\n",
      "Tsn \t4.26\n",
      "\n",
      "Avg PeakScore high expression genes: 4.371\n",
      "Avg PeakScore low expression genes: 4.39325\n"
     ]
    }
   ],
   "source": [
    "initFile = \"init_sites.txt\"\n",
    "exprFile = \"gene_expr.txt\"\n",
    "\n",
    "# create a list of \"high expression\" genes\n",
    "highExpr = []\n",
    "ins = open(exprFile)\n",
    "ins.readline() #skip header\n",
    "for line in ins:\n",
    "    line = line.rstrip('\\n')\n",
    "    lineParts = line.split()\n",
    "    if float(lineParts[1]) >= 50:\n",
    "        highExpr.append(lineParts[0])\n",
    "ins.close()\n",
    "\n",
    "# average the peak scores for high vs low genes\n",
    "highExprTotal = 0\n",
    "highExprCount = 0\n",
    "lowExprTotal = 0\n",
    "lowExprCount = 0\n",
    "\n",
    "ins = open(initFile, 'r')\n",
    "ins.readline() #skip header\n",
    "for line in ins:\n",
    "    line = line.rstrip('\\n')\n",
    "    lineParts = line.split()\n",
    "    if lineParts[0] in highExpr:\n",
    "        print lineParts[1], \"\\t\", lineParts[10]\n",
    "        highExprTotal = highExprTotal + float(lineParts[10])\n",
    "        highExprCount = highExprCount + 1\n",
    "    else:\n",
    "        lowExprTotal = lowExprTotal + float(lineParts[10])\n",
    "        lowExprCount = lowExprCount + 1\n",
    "ins.close()\n",
    "\n",
    "print \"\"\n",
    "print \"Avg PeakScore high expression genes:\", float(highExprTotal) / highExprCount\n",
    "print \"Avg PeakScore low expression genes:\", float(lowExprTotal) / lowExprCount"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 7. All-against-all comparisons (4pts)\n",
    "\n",
    "A common situation that arises in data analysis is that we have a list of data points and we would like to compare each data point to each other data point. Here, we will write a script that computes the \"distance\" between each pair of strings in a file and outputs a distance matrix. We will define \"distance\" between two strings as the number of mismatches between two strings when they are lined up, divided by their length.\n",
    "\n",
    "First we'll use a toy dataset. We'll create a list as follows:\n",
    "```\n",
    "things = [1, 2, 5, 10, 25, 50]\n",
    "```\n",
    "We'll start off by doing a very simple type of pairwise comparison: taking the numerical difference between two numbers. To systematically do this for all possible pairs of numbers in our list, we can make a nested for loop:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "4\n",
      "9\n",
      "24\n",
      "49\n",
      "1\n",
      "0\n",
      "3\n",
      "8\n",
      "23\n",
      "48\n",
      "4\n",
      "3\n",
      "0\n",
      "5\n",
      "20\n",
      "45\n",
      "9\n",
      "8\n",
      "5\n",
      "0\n",
      "15\n",
      "40\n",
      "24\n",
      "23\n",
      "20\n",
      "15\n",
      "0\n",
      "25\n",
      "49\n",
      "48\n",
      "45\n",
      "40\n",
      "25\n",
      "0\n"
     ]
    }
   ],
   "source": [
    "things = [1, 2, 5, 10, 25, 50]\n",
    "\n",
    "for i in range(len(things)):\n",
    "    for j in range(len(things)):\n",
    "        print abs(things[i] - things[j]) #absolute value of the difference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try running this code yourself and observe the output. Everything prints out on its own line, which isn't what we want -- we'd usually prefer a matrix-type format. Try this slightly modified code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 \t1 \t4 \t9 \t24 \t49 \t\n",
      "1 \t0 \t3 \t8 \t23 \t48 \t\n",
      "4 \t3 \t0 \t5 \t20 \t45 \t\n",
      "9 \t8 \t5 \t0 \t15 \t40 \t\n",
      "24 \t23 \t20 \t15 \t0 \t25 \t\n",
      "49 \t48 \t45 \t40 \t25 \t0 \t\n"
     ]
    }
   ],
   "source": [
    "things = [1, 2, 5, 10, 25, 50]\n",
    "\n",
    "for i in range(len(things)):\n",
    "    for j in range(len(things)):\n",
    "        print abs(things[i] - things[j]), \"\\t\",\n",
    "    print \"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This gives us the matrix format we want. Make sure you understand how this code works. FYI, the \"\\t\" is a tab character, and much like \"\\n\", it is invisible once it's printed (it becomes a tab). The comma at the very end of the `print` statement suppresses the \\n that `print` usually adds on to the end.\n",
    "\n",
    "So now we know how to do an all-against-all comparison. But how do we compute the number of mismatches between strings? As long as the strings are the same length, we can do something simple like the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dist = 0.43\n"
     ]
    }
   ],
   "source": [
    "str1 = \"Wilfred\"\n",
    "str2 = \"Manfred\"\n",
    "diffs = 0\n",
    "\n",
    "for k in range(len(str1)):\n",
    "    if str1[k] != str2[k]: #compare the two strings at the same index\n",
    "        diffs = diffs + 1\n",
    "        \n",
    "print \"dist =\", round(float(diffs) / len(str1), 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So this outputs the distance between the two strings, where the distance is defined as the fraction of the sequence length that is mismatched.\n",
    "\n",
    "**Using these two pieces of code as starting points, complete the following:**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(A)** (2 pt) Create a list of a few short strings *of the same length*. For example:\n",
    "```\n",
    "things = [\"bear\", \"pear\", \"boar\", \"tops\", \"bops\"]\n",
    "```\n",
    "Write code that prints a distance matrix for this list. As in the last example, use the fraction of mismatches between a given pair of words as the measure of their \"distance\" from each other. Round the distances to 2 decimals."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0 \t0.25 \t0.25 \t1.0 \t0.75 \t\n",
      "0.25 \t0.0 \t0.5 \t1.0 \t1.0 \t\n",
      "0.25 \t0.5 \t0.0 \t0.75 \t0.5 \t\n",
      "1.0 \t1.0 \t0.75 \t0.0 \t0.25 \t\n",
      "0.75 \t1.0 \t0.5 \t0.25 \t0.0 \t\n"
     ]
    }
   ],
   "source": [
    "things = [\"bear\", \"pear\", \"boar\", \"tops\", \"bops\"]\n",
    "for i in range(len(things)):\n",
    "    for j in range(len(things)):\n",
    "        str1 = things[i]\n",
    "        str2 = things[j]\n",
    "        diffs = 0\n",
    "        for k in range(len(str1)):\n",
    "            if str1[k] != str2[k]:\n",
    "                diffs += 1\n",
    "        print round(float(diffs)/len(str1),2), \"\\t\",\n",
    "    print \"\" "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(B)** (2 pt) Now, instead of using a hard coded list like you did in (A), create a list of DNA sequences by reading in the file `sequences2.txt`. Compute the distance matrix between these sequences and print the distance matrix. Looking at this matrix, do you see a pair of sequences that are much less \"distant\" from each other than all the rest?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0 \t0.62 \t0.78 \t0.78 \t0.7 \t0.7 \t0.92 \t0.8 \t0.76 \t0.6 \t\n",
      "0.62 \t0.0 \t0.78 \t0.76 \t0.68 \t0.72 \t0.78 \t0.76 \t0.76 \t0.8 \t\n",
      "0.78 \t0.78 \t0.0 \t0.8 \t0.78 \t0.72 \t0.72 \t0.82 \t0.12 \t0.76 \t\n",
      "0.78 \t0.76 \t0.8 \t0.0 \t0.72 \t0.84 \t0.64 \t0.84 \t0.8 \t0.86 \t\n",
      "0.7 \t0.68 \t0.78 \t0.72 \t0.0 \t0.7 \t0.82 \t0.64 \t0.76 \t0.86 \t\n",
      "0.7 \t0.72 \t0.72 \t0.84 \t0.7 \t0.0 \t0.78 \t0.78 \t0.72 \t0.8 \t\n",
      "0.92 \t0.78 \t0.72 \t0.64 \t0.82 \t0.78 \t0.0 \t0.78 \t0.74 \t0.7 \t\n",
      "0.8 \t0.76 \t0.82 \t0.84 \t0.64 \t0.78 \t0.78 \t0.0 \t0.74 \t0.74 \t\n",
      "0.76 \t0.76 \t0.12 \t0.8 \t0.76 \t0.72 \t0.74 \t0.74 \t0.0 \t0.72 \t\n",
      "0.6 \t0.8 \t0.76 \t0.86 \t0.86 \t0.8 \t0.7 \t0.74 \t0.72 \t0.0 \t\n"
     ]
    }
   ],
   "source": [
    "# create list of sequences\n",
    "things = []\n",
    "ins = open(\"sequences2.txt\", 'r')\n",
    "for line in ins:\n",
    "    line = line.rstrip('\\n')\n",
    "    things.append(line)\n",
    "ins.close()\n",
    "\n",
    "# create distance matrix\n",
    "for i in range(len(things)):\n",
    "    for j in range(len(things)):\n",
    "        str1 = things[i]\n",
    "        str2 = things[j]\n",
    "        diffs = 0\n",
    "        for k in range(len(str1)):\n",
    "            if str1[k] != str2[k]:\n",
    "                diffs += 1\n",
    "        print round(float(diffs)/len(str1),2), \"\\t\",\n",
    "    print \"\" "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "---\n",
    "## Extra questions (0pts)\n",
    "\n",
    "These questions are for people who would like extra practice. They will not be counted for points."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(A)** Following from problem 7 above: As you can see, the distance matrix you made in (B) is symmetrical around the diagonal. This means dist(i,j) is the same as dist(j,i), so we're doing some redundant calculations. \n",
    "\n",
    "Change the code so that we don't do any unnecessary calculations (including comparing a sequence to itself, which always is\n",
    "0). For any calculations you skip, you can print \"-\" or some other place-holder to keep the printed matrix looking neat. \n",
    "\n",
    "*Hint: There's a really simple way to do this! Think about the range of the second loop...*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "METHOD 1: lower triangle\n",
      "\n",
      "\n",
      "0.62 \t\n",
      "0.78 \t0.78 \t\n",
      "0.78 \t0.76 \t0.8 \t\n",
      "0.7 \t0.68 \t0.78 \t0.72 \t\n",
      "0.7 \t0.72 \t0.72 \t0.84 \t0.7 \t\n",
      "0.92 \t0.78 \t0.72 \t0.64 \t0.82 \t0.78 \t\n",
      "0.8 \t0.76 \t0.82 \t0.84 \t0.64 \t0.78 \t0.78 \t\n",
      "0.76 \t0.76 \t0.12 \t0.8 \t0.76 \t0.72 \t0.74 \t0.74 \t\n",
      "0.6 \t0.8 \t0.76 \t0.86 \t0.86 \t0.8 \t0.7 \t0.74 \t0.72 \t\n",
      "\n",
      "METHOD 2: upper triangle, messed up format\n",
      "\n",
      "0.62 \t0.78 \t0.78 \t0.7 \t0.7 \t0.92 \t0.8 \t0.76 \t0.6 \t\n",
      "0.78 \t0.76 \t0.68 \t0.72 \t0.78 \t0.76 \t0.76 \t0.8 \t\n",
      "0.8 \t0.78 \t0.72 \t0.72 \t0.82 \t0.12 \t0.76 \t\n",
      "0.72 \t0.84 \t0.64 \t0.84 \t0.8 \t0.86 \t\n",
      "0.7 \t0.82 \t0.64 \t0.76 \t0.86 \t\n",
      "0.78 \t0.78 \t0.72 \t0.8 \t\n",
      "0.78 \t0.74 \t0.7 \t\n",
      "0.74 \t0.74 \t\n",
      "0.72 \t\n",
      "\n",
      "\n",
      "METHOD 3: upper triangle with formatting\n",
      "\n",
      " - \t0.62 \t0.78 \t0.78 \t0.7 \t0.7 \t0.92 \t0.8 \t0.76 \t0.6 \t\n",
      " - \t - \t0.78 \t0.76 \t0.68 \t0.72 \t0.78 \t0.76 \t0.76 \t0.8 \t\n",
      " - \t - \t - \t0.8 \t0.78 \t0.72 \t0.72 \t0.82 \t0.12 \t0.76 \t\n",
      " - \t - \t - \t - \t0.72 \t0.84 \t0.64 \t0.84 \t0.8 \t0.86 \t\n",
      " - \t - \t - \t - \t - \t0.7 \t0.82 \t0.64 \t0.76 \t0.86 \t\n",
      " - \t - \t - \t - \t - \t - \t0.78 \t0.78 \t0.72 \t0.8 \t\n",
      " - \t - \t - \t - \t - \t - \t - \t0.78 \t0.74 \t0.7 \t\n",
      " - \t - \t - \t - \t - \t - \t - \t - \t0.74 \t0.74 \t\n",
      " - \t - \t - \t - \t - \t - \t - \t - \t - \t0.72 \t\n",
      " - \t - \t - \t - \t - \t - \t - \t - \t - \t - \t\n"
     ]
    }
   ],
   "source": [
    "### METHOD #1\n",
    "\n",
    "print \"\"\n",
    "print \"METHOD 1: lower triangle\"\n",
    "print \"\"\n",
    "\n",
    "things = []\n",
    "ins = open(\"sequences2.txt\", 'r')\n",
    "for line in ins:\n",
    "    line = line.rstrip('\\n')\n",
    "    things.append(line)\n",
    "ins.close()\n",
    "\n",
    "for i in range(len(things)):\n",
    "    for j in range(0,i): #<-- this is the only change we need to make\n",
    "        str1 = things[i]\n",
    "        str2 = things[j]\n",
    "        diffs = 0\n",
    "        for k in range(len(str1)):\n",
    "            if str1[k] != str2[k]:\n",
    "                diffs += 1\n",
    "        print round(float(diffs)/len(str1),2), \"\\t\",\n",
    "    print \"\" \n",
    "\n",
    "\n",
    "\n",
    "\n",
    "### METHOD #2\n",
    "# note that the way it prints will look weird! see below for another example that fixes this\n",
    "\n",
    "print \"\"\n",
    "print \"METHOD 2: upper triangle, messed up format\"\n",
    "print \"\"\n",
    "\n",
    "things = []\n",
    "ins = open(\"sequences2.txt\", 'r')\n",
    "for line in ins:\n",
    "    line = line.rstrip('\\n')\n",
    "    things.append(line)\n",
    "ins.close()\n",
    "\n",
    "for i in range(len(things)):\n",
    "    for j in range(i+1,len(things)): #<-- \n",
    "        str1 = things[i]\n",
    "        str2 = things[j]\n",
    "        diffs = 0\n",
    "        for k in range(len(str1)):\n",
    "            if str1[k] != str2[k]:\n",
    "                diffs += 1\n",
    "        print round(float(diffs)/len(str1),2), \"\\t\",\n",
    "    print \"\" \n",
    "\n",
    "\n",
    "\n",
    "### METHOD #3\n",
    "# similar to above, but fixed output formatting to be more easily interpreted\n",
    "\n",
    "print \"\"\n",
    "print \"METHOD 3: upper triangle with formatting\"\n",
    "print \"\"\n",
    "\n",
    "things = []\n",
    "ins = open(\"sequences2.txt\", 'r')\n",
    "for line in ins:\n",
    "    line = line.rstrip('\\n')\n",
    "    things.append(line)\n",
    "ins.close()\n",
    "\n",
    "for i in range(len(things)):\n",
    "    for j in range(len(things)):\n",
    "        if j > i:\n",
    "            str1 = things[i]\n",
    "            str2 = things[j]\n",
    "            diffs = 0\n",
    "            for k in range(len(str1)):\n",
    "                if str1[k] != str2[k]:\n",
    "                    diffs += 1\n",
    "            print round(float(diffs)/len(str1),2), \"\\t\",\n",
    "        else:\n",
    "            print \" - \\t\",\n",
    "    print \"\" "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(B)** Below is a loop that creates a list. Do the same thing but with a list comprehension instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[7, 6, 9, 3, 3, 5, 6, 0, 10, 4, 8, 1, 1, 0, 3, 8, 5, 6, 3, 0, 7, 4, 3, 10, 0, 8, 2, 8, 1, 0, 9, 10, 2, 1, 0, 3, 5, 8, 6, 7, 4, 4, 4, 9, 3, 2, 5, 4, 2, 4, 9, 2, 7, 5, 6, 3, 4, 2, 10, 9, 1, 10, 8, 0, 10, 0, 8, 9, 2, 2, 0, 4, 5, 1, 10, 2, 8, 5, 9, 3, 10, 0, 3, 1, 5, 3, 2, 8, 0, 4, 9, 7, 9, 3, 4, 2, 6, 8, 7, 2]\n"
     ]
    }
   ],
   "source": [
    "# Loop version:\n",
    "import random\n",
    "randomNums = []\n",
    "\n",
    "for i in range(100):\n",
    "    randomNums.append(random.randint(0,10))\n",
    "    \n",
    "print randomNums"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[5, 10, 4, 5, 1, 2, 3, 7, 8, 4, 9, 6, 8, 0, 9, 3, 6, 0, 7, 0, 6, 4, 8, 0, 4, 0, 6, 3, 1, 10, 8, 4, 4, 1, 0, 5, 0, 2, 3, 10, 4, 5, 6, 1, 4, 1, 7, 2, 3, 1, 3, 2, 2, 0, 10, 1, 10, 7, 8, 7, 0, 3, 6, 5, 1, 4, 3, 4, 9, 6, 6, 0, 4, 10, 5, 8, 0, 5, 4, 7, 8, 2, 10, 5, 8, 4, 5, 1, 3, 7, 3, 6, 2, 0, 4, 10, 5, 7, 9, 3]\n"
     ]
    }
   ],
   "source": [
    "# Your list comprehension version here:\n",
    "randomNums = [random.randint(0,10) for i in range(100)]\n",
    "print randomNums"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(C)** Below is a loop that creates a list. Do the same thing but with a list comprehension instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['5', '6', '1', '4', '8', '6', '7', '7', '9', '2', '5', '0', '8', '7', '3', '7', '7', '5', '3', '2', '7', '3', '6', '8', '0', '5', '9', '5', '0', '7', '1', '9', '0', '9', '4', '3', '5', '5', '6', '6', '10', '0', '8', '3', '2', '3', '2', '7', '4', '1']\n"
     ]
    }
   ],
   "source": [
    "# Loop version:\n",
    "import random\n",
    "randomNums = []\n",
    "\n",
    "for i in range(100):\n",
    "    if (i % 2) == 0:\n",
    "        randNum = random.randint(0,10)\n",
    "        randNumStr = str(randNum)\n",
    "        randomNums.append(randNumStr)\n",
    "    \n",
    "print randomNums"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Note: I changed this slightly from how it was originally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['6', '1', '10', '5', '10', '7', '10', '5', '10', '9', '7', '10', '7', '9', '0', '1', '9', '9', '4', '10', '0', '6', '3', '8', '4', '3', '0', '0', '1', '2', '6', '7', '3', '6', '4', '6', '6', '9', '8', '8', '2', '3', '3', '2', '0', '5', '7', '9', '0', '2']\n"
     ]
    }
   ],
   "source": [
    "# Your list comprehension version here:\n",
    "randomNums = [str(random.randint(0,10)) for i in range(100) if i % 2 == 0]\n",
    "print randomNums"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}