Assignment_2_Task5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook snippet shows corrected versions of the Task 5 cells for Assignment 2. It implements the corrections already announced on the LMS, for your reference.\n",
    "\n",
    "You cannot actually run this notebook snippet as it does not import any modules or data. It's just for reference.\n",
    "\n",
    "Here are the changes relative to the original assignment notebook:\n",
    "\n",
    "- In the first code cell, we have replaced `pca.fit(tissue_expression.loc[module,:].values.transpose())` with `pca.fit(tissue_expression.loc[module,:].values)` (i.e. we have removed the transpose).\n",
    "- The two `eigengene()` test cases now have comments showing the correct expected outputs, i.e. `array([-0.44, -0.48, -0.76])` and `array([-0.6 , -0.72, -0.36])`.\n",
    "- The very last code cell now draws a normalised plot which is more useful for seeing the resulting eigengenes (this change is not important for assessment as you do not carry out any task using this plot)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 5 - Finding module eigengenes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The eigengene of a module is defined as the first principal component of all genes in a module. This can be considered a representative expression profile for the genes in the module."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at the explained variance of the main eigengene for a module, and the next few principal components or 'eigengenes':"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Arbitrarily, we'll examine the first module\n",
    "module = modules_hierarchical[0]\n",
    "# Let's get the first 10 'eigengenes' of this module \n",
    "# (usually we only care about the first eigengene)\n",
    "pca = PCA(10)\n",
    "pca.fit(tissue_expression.loc[module,:].values)\n",
    "plt.plot(range(1,10+1),pca.explained_variance_ratio_, 'o-')\n",
    "plt.ylim(0,1)\n",
    "plt.xlabel('Principal component')\n",
    "plt.ylabel('Explained variance')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task 5 (a)**\n",
    "\n",
    "Based on the explained variance of the first few principal components, do you think the eigengene captures most of the relevant information?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*# ~~ GRADED CELL - your answer here --*\n",
    "\n",
    "The retained variance of the first principal component explains over 70% of the variance, so the eigengene is likely to be a good representation of the module."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Task 5(b)**\n",
    "\n",
    "Complete the `eigengene()` function to perform PCA on a module and calculate the eigengene expression values. You only need to return the first, or principal, eigengene."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ~~ GRADED CELL - complete this cell ~~\n",
    "\n",
    "def eigengene(expression, module_genes):\n",
    "    '''\n",
    "    Given a dataframe representing log-expression values,\n",
    "    and a list of gene names, \n",
    "    find the eigengene.\n",
    "    '''\n",
    "    ### YOUR CODE HERE - fill in this function\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Should return array([-0.44, -0.48, -0.76])\n",
    "\n",
    "test_df = pd.DataFrame([[ 3.8,  2.7,  4.5],\n",
    "                       [ 4.3,  3.4,  6.2],\n",
    "                       [ 5.3,  4.3,  7. ],\n",
    "                       [ 4.6,  6. ,  7.7],\n",
    "                       [ 5.2,  7.3,  8.8],\n",
    "                       [ 6.2,  8.5,  9.4]], \n",
    "                         columns=['Tissue1', 'Tissue2', 'Tissue3'],\n",
    "                         index=['GeneA', 'GeneB', 'GeneC', 'GeneD', 'GeneE', 'GeneF'])\n",
    "\n",
    "eigengene(test_df, ['GeneA', 'GeneB', 'GeneC'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Should return array([-0.6 , -0.72, -0.36])\n",
    "\n",
    "test_df = pd.DataFrame([[ 3.8,  2.7,  4.5],\n",
    "                       [ 4.3,  3.4,  6.2],\n",
    "                       [ 5.3,  4.3,  7. ],\n",
    "                       [ 4.6,  6. ,  7.7],\n",
    "                       [ 5.2,  7.3,  8.8],\n",
    "                       [ 6.2,  8.5,  9.4]], \n",
    "                 columns=['Tissue1', 'Tissue2', 'Tissue3'],\n",
    "                 index=['GeneA', 'GeneB', 'GeneC', 'GeneD', 'GeneE', 'GeneF'])\n",
    "\n",
    "eigengene(test_df, ['GeneE', 'GeneF'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each module eigengene is an expression pattern across our fly tissue samples.\n",
    "We can visualise the eigengenes by showing the expression level of each for each tissue - this is an indication of the behaviour of each module across different tissue types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "eigengenes = []\n",
    "for module in modules_hierarchical:\n",
    "    eigengenes.append(eigengene(tissue_expression, module))\n",
    "eigengene_df = pd.DataFrame(np.array(eigengenes), columns=tissue_expression.columns)\n",
    "\n",
    "print('Module sizes:',[len(m) for m in modules_hierarchical])\n",
    "fig,ax = plt.subplots(figsize=(8,10))\n",
    "# Normalise each column of eigengene_df, so we are only looking at the expression of\n",
    "# an eigengene in a tissue relative to other tissues for the same eigengene\n",
    "sns.heatmap((eigengene_df.transpose() - eigengene_df.transpose().mean())/eigengene_df.transpose().std(), cmap='Greens',ax=ax)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}