example_pool_design.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example pool design"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For CRISPR lentiviral libraries, we recommend using the CROPseq vector, which allows direct _in situ_ sequencing of the CRISPR sgRNA. The code in `ops.pool_design` can select sgRNAs for _in situ_ sequencing, combine multiple within one oligo array, and export the final oligos to order.\n",
    "\n",
    "Inputs needed to design an oligo array with `ops.pool_design`:\n",
    "\n",
    "- **sgRNA table**: A list of gene IDs and corresponding sgRNAs. \n",
    "    - The sgRNAs for each gene can be ranked, so that higher-ranked sgRNAs are selected first. Library designs usually rank sgRNAs by taking into account on-target efficiency, potential off-target sites, and targeting position within a gene.\n",
    "    - There are many publicly available library designs and design tools for CRISPR screens. In this example, the [Brunello] and [TKOv3] CRISPR KO libraries are used.\n",
    "- **gene list**: A text file with one gene ID per row. There should be one gene list for each `design` (for example, a pool of all kinases).\n",
    "- **pool design**: A spreadsheet with one row for each gene set in a pool. \n",
    "    - Multiple `subpools`, each with a different gene `design`, can be synthesized in one pool. Subpools with different `dialout` adapters can be specifically amplified by PCR. \n",
    "    - Within a `group`, sgRNAs will have unique 5' prefixes so they can be pooled together and read out by 5'-to-3' sequencing-by-synthesis. Prefixes are selected based on `prefix_length` and `edit_distance`. A longer `prefix_length` allows more sgRNAs to be included, but requires more cycles of _in situ_ sequencing. If minimum `edit_distance` between prefixes is increased from 1 to 2 or 3, prefixes will be robust to 1 or 2 single-base errors (insertions, deletions or substitutions resulting from synthesis or sequencing).\n",
    "    - The library size is set by the number of genes (`num_genes`) and targeting sgRNAs per gene (`sgRNAs_per_gene`). Oligos can be duplicated to balance subpool size or reduce abundance bias due to synthesis (`duplicate_oligos`).\n",
    "\n",
    "[Brunello]: https://www.addgene.org/pooled-library/broadgpp-human-knockout-brunello/\n",
    "[TKOv3]: https://www.addgene.org/pooled-library/moffat-crispr-knockout-tkov3/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ops.imports_ipython import *\n",
    "import ops.pool_design\n",
    "\n",
    "# runs example from example_pool/ sub-directory of project\n",
    "home = os.path.dirname(os.path.dirname(ops.__file__))\n",
    "os.chdir(os.path.join(home, 'example_pool'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "### Prepare sgRNA table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "# Download CRISPR KO library designs\n",
    "# https://www.addgene.org/pooled-library/broadgpp-human-knockout-brunello/\n",
    "brunello_url = ('https://www.addgene.org/static/cms/filer_public/'\n",
    "                '8b/4c/8b4c89d9-eac1-44b2-bb2f-8fea95672705/'\n",
    "                'broadgpp-brunello-library-contents.txt')\n",
    "\n",
    "# https://www.addgene.org/pooled-library/moffat-crispr-knockout-tkov3/\n",
    "tkov3_url = ('https://media.addgene.org/cms/filer_public/'\n",
    "             '71/a8/71a81179-7a62-4d75-9b53-236e6f6b7d4d/'\n",
    "             'tkov3_guide_sequence.xlsx')\n",
    "\n",
    "!curl -LO {brunello_url}\n",
    "!curl -LO {tkov3_url}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "hidden": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8    8935\n",
       "7    6023\n",
       "4    1896\n",
       "6    1650\n",
       "5     392\n",
       "3     307\n",
       "2      28\n",
       "1      17\n",
       "Name: sgRNA_counts per gene ID, dtype: int64"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load brunello library\n",
    "f = 'broadgpp-brunello-library-contents.txt'\n",
    "df_brunello = (ops.pool_design.import_brunello(f)\n",
    " .assign(source='1_brunello'))\n",
    "\n",
    "# load TKOv3 library\n",
    "f = 'NCBI_ids.tsv'\n",
    "df_ncbi = ops.pool_design.import_hugo_ncbi(f)\n",
    "\n",
    "f = 'tkov3_guide_sequence.xlsx'\n",
    "df_tkov3 = (ops.pool_design.import_tkov3(f, df_ncbi)\n",
    " .assign(source='2_tkov3'))\n",
    "\n",
    "# combine libraries\n",
    "(pd.concat([df_brunello, df_tkov3], sort=True)\n",
    " .sort_values(['gene_id', 'source', 'rank'])\n",
    " .drop_duplicates('sgRNA')\n",
    " .assign(rank=lambda x: ops.utils.rank_by_order(x, 'gene_id'))\n",
    " .to_csv('sgRNAs.csv', index=None)\n",
    ")\n",
    "\n",
    "df_sgRNAs = (pd.read_csv('sgRNAs.csv')\n",
    " .pipe(ops.pool_design.filter_sgRNAs)\n",
    ")\n",
    "\n",
    "(df_sgRNAs\n",
    "# remove non-targeting\n",
    " .query('gene_id != -1')\n",
    "['gene_id'].value_counts().value_counts()\n",
    ".rename('sgRNA_counts per gene ID'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "### Load pool design and gene lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "df_design = (pd.read_excel('design.xls', skiprows=1)\n",
    " .pipe(ops.pool_design.validate_design))\n",
    "\n",
    "gene_lists = ['X.txt', 'Y.txt', 'Z.txt']\n",
    "df_genes = (pd.concat([ops.pool_design.load_gene_list(d + '.txt')\n",
    "           for d in set(df_design['design'])])\n",
    " # optionally: convert gene symbols to gene ids\n",
    " .join(df_design.set_index('design'), on='design')\n",
    " .reset_index(drop=True)\n",
    " .pipe(ops.pool_design.validate_genes, df_sgRNAs)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "### Select sgRNAs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "f = 'kosuri_dialout_primers.csv'\n",
    "dialout_primers = ops.pool_design.import_dialout_primers(f)\n",
    "\n",
    "cols = ['subpool', 'dialout', 'design', 'vector', 'group', \n",
    "        'prefix_length', 'edit_distance', 'gene_id', 'source', \n",
    "        'rank', 'duplicate_oligos', 'sgRNA']\n",
    "\n",
    "df_oligos = (df_genes\n",
    " # select sgRNAs separately for each prefix group\n",
    " .groupby('group')\n",
    " .apply(ops.pool_design.select_prefix_group, df_sgRNAs)\n",
    " .reset_index(drop=True)\n",
    " [cols]\n",
    " # build the full oligo sequence\n",
    " .assign(oligo=lambda x: \n",
    "         ops.pool_design.build_sgRNA_oligos(x, dialout_primers))\n",
    " # add duplicate oligos where requested\n",
    " .reset_index(drop=True)\n",
    " .pipe(lambda x: \n",
    "      x.loc[np.repeat(x.index.values, x['duplicate_oligos'])])\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "### Check for genes with less sgRNAs than requested"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "hidden": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subpool</th>\n",
       "      <th>design</th>\n",
       "      <th>missing_sgRNAs</th>\n",
       "      <th>gene_ids</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>pool0_0</td>\n",
       "      <td>X</td>\n",
       "      <td>0</td>\n",
       "      <td>497</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>pool0_0</td>\n",
       "      <td>X</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pool0_1</td>\n",
       "      <td>Y</td>\n",
       "      <td>0</td>\n",
       "      <td>497</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>pool0_1</td>\n",
       "      <td>Y</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>pool0_2</td>\n",
       "      <td>nontargeting</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>pool0_3</td>\n",
       "      <td>Z</td>\n",
       "      <td>0</td>\n",
       "      <td>472</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>pool0_3</td>\n",
       "      <td>Z</td>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>pool0_3</td>\n",
       "      <td>Z</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>pool0_3</td>\n",
       "      <td>nontargeting</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   subpool        design  missing_sgRNAs  gene_ids\n",
       "0  pool0_0             X               0       497\n",
       "1  pool0_0             X               1         3\n",
       "2  pool0_1             Y               0       497\n",
       "3  pool0_1             Y               1         3\n",
       "4  pool0_2  nontargeting               0         1\n",
       "5  pool0_3             Z               0       472\n",
       "6  pool0_3             Z               1        26\n",
       "7  pool0_3             Z               2         2\n",
       "8  pool0_3  nontargeting               0         1"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "designed = (df_oligos\n",
    " .drop_duplicates('oligo')\n",
    " .groupby(['subpool', 'design', 'gene_id']).size()\n",
    ")\n",
    "requested = (df_genes\n",
    " .set_index(['subpool', 'design', 'gene_id'])\n",
    " ['sgRNAs_per_gene']\n",
    ")\n",
    "\n",
    "(requested.sub(designed, fill_value=0)\n",
    " .rename('missing_sgRNAs')\n",
    " .reset_index()\n",
    " .groupby(['subpool', 'design'])\n",
    " ['missing_sgRNAs'].value_counts().rename('gene_ids')\n",
    " .reset_index()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "### Validate and export oligo pool"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "hidden": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking good!\n"
     ]
    }
   ],
   "source": [
    "df_oligos.to_csv('pool0_design.csv', index=None)\n",
    "\n",
    "(df_oligos['oligo']\n",
    " # optional: randomize oligo order for synthesis\n",
    " # .sample(frac=1)\n",
    " .to_csv('pool0_oligos.txt', index=None)\n",
    ")\n",
    "\n",
    "df_test = (pd.read_csv('pool0_design.csv')\n",
    " .pipe(ops.pool_design.build_test, dialout_primers)\n",
    " .pipe(ops.pool_design.validate_test))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}