small things/text

Author: mahort

Lecture-1-Linear-Regression.ipynb

@@ -22,7 +22,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Set up environment"
+    "### Set up the environment"
    ]
   },
   {
@@ -50,15 +50,15 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Download data"
+    "### Load the data"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We here load the yeast cross dataset.\n",
-    "The data used in this study have been preconverted into an hdf5 file. \n",
+    "Here, we load the yeast cross dataset.\n",
+    "The data used in this study have already been converted into an hdf5 file. \n",
     "To process your own data, please use the limix command line binary (see [here](http://nbviewer.jupyter.org/github/limix/limix-tutorials/blob/master/preprocessing_QC/loading_files.ipynb))."
    ]
   },
@@ -81,14 +81,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Set up data object"
+    "### Set up the data object"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Phenotypes and genotypes are stored inside the HDF5 file. Load them into a dataframe and select the first 3 phenotypes. "
+    "Both the phenotypes and the genotypes are stored inside an HDF5 file. Load the data into a dataframe; here, we focus on the first 3 phenotypes. "
    ]
   },
   {
@@ -462,14 +462,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Normal distributed phenotypes and phenotype transformations"
+    "## Check the model assumptions (are the data normal?)"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "To explore the phenotypic data, we create a histogram of the phenotype values."
+    "Here, we use histograms to look at the distributions of the phenotypes."
    ]
   },
   {
@@ -562,25 +562,23 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Some of the phenotypes deviate from a normal distribution.\n",
-    "One of the assumptions of the linear regression model we use for association testing is that the model residuals are normal distrbuted.\n",
-    "Violation of this assumption leads to biases in the analysis.\n",
-    "We only have access to the residuals after fitting the model.\n",
-    "Under the assumption that the model eplains only a small portion of phenotypic variation we can assess the phenotype values instead."
+    "Your data will often deviate from a normal distribution (sometimes drastically, like Cadmium Chloride shown above).\n",
+    "Unfortunately, one of the assumptions of the model that we use in GWAS is that the residuals are normally distrbuted.\n",
+    "Violations of this assumption can result in model misspecification and thus biased parameter estimates."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Transforming phenotypes"
+    "### Variance stabilizing transformations; standardizing the phenotypes"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "To make the data look more normal distrbuted, we apply two different phenotype transformations, the Box-Cox transformation and a non-parametric rank-based transformation."
+    "There are a wide variety of methods to stabilize variance and make data normally distributed. Here, we explore the usefulness of the Box-Cox transformation as well as a (non-parametric) rank-based transformation."
    ]
   },
   {
@@ -594,7 +592,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The Box-Cox transformation makes the data \"more normal\" by fitting a power transformation with one parameter to the observed phenotypic data."
+    "The Box-Cox transformation makes the data \"more normal\" by fitting a power transformation ($y^{\\lambda}$, where $\\lambda$ is found using maximum likelihood) to the observed phenotypic data."
    ]
   },
   {
@@ -926,14 +924,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Manhattan plot"
+    "### Plotting the results"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "A common way to visualize the results of a GWAS is a so-called Manhattan plot, where the $-log_{10}$ P-values are plotted against the genomic position.\n",
+    "A common way to visualize the results from GWAS is by using a so-called Manhattan plot, where the $-log_{10}$ P-values are plotted against the genomic position.\n",
     "\n",
     "The LIMIX function for producing Manhattan plots is ``limix.plot.plot_manhattan`` (see [here][1]).\n",
     "\n",
@@ -1026,14 +1024,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "##### GWAS using linear regression on the transformed phenotypes:"
+    "### Conducting GWAS with the transformed phenotypes:"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "First we analyze the Box-Cox transformed phenotypes."
+    "First we perform GWAS with the Box-Cox transformed phenotypes."
    ]
   },
   {
@@ -1059,7 +1057,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Next, we analyze the phenotypes transformed by the rank-based transformation."
+    "Next, we investigate the rank-transformed phenotypes."
    ]
   },
   {
@@ -1085,7 +1083,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "To compare the results of the various transformations, we plot the p-values against one another:"
+    "To compare the results of the transformations, we can plot the p-values against one another:"
    ]
   },
   {
@@ -1641,16 +1639,16 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 29,
+   "execution_count": 25,
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "<matplotlib.legend.Legend at 0x1a5bb55d10>"
+       "<matplotlib.legend.Legend at 0x1a24e668d0>"
       ]
      },
-     "execution_count": 29,
+     "execution_count": 25,
      "metadata": {},
      "output_type": "execute_result"
     },
@@ -1716,17 +1714,17 @@
     "covars_conditional= sp.concatenate((geno_df.loc[sample_idx].values[:,imax:imax+1], sp.ones((phenotype_vals.values.shape[0],1))),1)\n",
     "                                  \n",
     "\n",
-    "#run linear regression on each SNP\n",
+    "#run linear regression on each SNP, while conditioning on the top SNP as a covariate.\n",
     "lm_conditional = qtl_test_lm(snps=geno_df.loc[sample_idx].values,pheno=phenotype_vals.values,covs=covars_conditional)\n",
     "\n",
-    "#convert P-values to a DataFrame for nice output writing:\n",
+    "#convert P-values to a pandas DataFrame:\n",
     "pvalues_lm_conditional = pd.DataFrame(data=lm_conditional.pvalues.T,index=positions,\n",
     "                       columns=phenotype_ID)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 30,
+   "execution_count": 27,
    "metadata": {},
    "outputs": [
     {

Lecture-2-Linear-Mixed-Model.ipynb

@@ -60,9 +60,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We here load the arabidopsis dataset.\n",
-    "The data used in this study have been pre-converted into an hdf5 file. \n",
-    "To process your own data, please use the limix command line binary (see [here](http://nbviewer.jupyter.org/github/limix/limix-tutorials/blob/master/preprocessing_QC/loading_files.ipynb))."
+    "Here, we load the arabidopsis data, which have already been converted into an hdf5 file. \n",
+    "\n",
+    "To process your own data, use the limix command line binary (see [here](http://nbviewer.jupyter.org/github/limix/limix-tutorials/blob/master/preprocessing_QC/loading_files.ipynb) for an example)."
    ]
   },
   {
@@ -87,7 +87,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The data object allows to query specific genotype or phenotype data"
+    "The HDF5 file holds both the genotype and phenotype data."
    ]
   },
   {
@@ -295,7 +295,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 35,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -307,6 +307,26 @@
     "                      dtype='float64')"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 36,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(1179, 21456)"
+      ]
+     },
+     "execution_count": 36,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "geno_df.shape"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 7,
@@ -520,7 +540,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Explore flowering time phenotypes"
+    "### Let's work with flowering time data"
    ]
   },
   {
@@ -675,7 +695,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Note that when cov are not set (None), LIMIX fits an intercept (i.e., ``covs=sp.ones((N,1))``)."
+    "When you do not include covariates (None), LIMIX still fits an intercept (i.e., ``covs=sp.ones((N,1))``)."
    ]
   },
   {
@@ -817,7 +837,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Manhattan plot"
+    "### To get a quick idea of the results from GWAS, visualize the P-values with a Manhattan plot"
    ]
   },
   {
@@ -1052,7 +1072,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Here, we perform GWAS while using an increasing number of principal components (after PCA of the genotype matrix) in an attempt to correct for confounding due to population structure. Then, we use QQ-plots to check for (possible) P-value inflation."
+    "Here, we perform GWAS using principal components (after PCA of the genotype matrix) in an attempt to correct for confounding due to population structure. Like earlier, we use QQ-plots to check for (possible) P-value inflation."
    ]
   },
   {