notebooks/Convert ns-ALEX HT3 files to Photon-HDF5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Convert ns-ALEX HT3 files to Photon-HDF5\n",
    "<br>\n",
    "<div style=\"border: 1px solid #d4d4d4;\n",
    "    background-color: #EBF4FA;\n",
    "    border-radius: 8px;\n",
    "    padding: 40px;\">\n",
    "<i>\n",
    "<h2> Summary </h2>\n",
    "<p>This [Jupyter notebook](https://jupyter.org/)\n",
    "will guide you through the conversion of a ns-ALEX data file from **HT3**\n",
    "to [Photon-HDF5](http://photon-hdf5.org) format. \n",
    "For more info on how to edit\n",
    "a jupyter notebook refer to [this example](http://nbviewer.jupyter.org/github/jupyter/notebook/blob/master/docs/source/examples/Notebook/Notebook%20Basics.ipynb#The-Notebook-dashboard).\n",
    "\n",
    "<p>If you are running this notebook online please note that this is a demo service and the\n",
    "file size of input data files is limited to 35MB.\n",
    "\n",
    "<p>Please send feedback and report any problems to the \n",
    "[Photon-HDF5 google group](https://groups.google.com/forum/#!forum/photon-hdf5).\n",
    "</i>\n",
    "</div>\n",
    "\n",
    "# 1. How to run it?\n",
    "\n",
    "The notebook is composed by \"text cells\", such as this paragraph, and \"code cells\"\n",
    "containing the code to be executed (and identifyied by an `In [ ]` prompt). \n",
    "To execute a code cell, select it and press SHIFT+ENTER. \n",
    "To modify an cell, click on it to enter \"edit mode\" (indicated by a green frame), \n",
    "than type.\n",
    "\n",
    "You can run this notebook directly online (for demo purposes), or you can \n",
    "run it on your on desktop. For a local installation please refer to:\n",
    "\n",
    "- [Jupyter Notebook Quick-Start Guide](http://jupyter-notebook-beginner-guide.readthedocs.org) \n",
    "\n",
    "<br>\n",
    "<div class=\"alert alert-success\">\n",
    "Please run each each code cell using SHIFT+ENTER.\n",
    "</div>\n",
    "\n",
    "# 2. Prepare the data file\n",
    "\n",
    "## 2.1 Upload the data file\n",
    "\n",
    "*Note: Skip to section 2.2 if you are running the notebook locally.*\n",
    "\n",
    "To start, we need to upload the file we want to convert to Photon-HDF5.\n",
    "You can use one of the example data files freely available on\n",
    "[on figshare](http://dx.doi.org/10.6084/m9.figshare.1455963). \n",
    "\n",
    "To upload the file switch to the \"Home\" tab in your browser, \n",
    "click the upload button and select the data file. \n",
    "Wait until the upload completes than switch back to this notebook.\n",
    "\n",
    "## 2.2 Select the file\n",
    "\n",
    "Specify the file name of the input data file in the following cell:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "filename = 'Pre.ht3'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next cell will check if the `filename` location is correct:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os\n",
    "try: \n",
    "    with open(filename): pass\n",
    "    print('Data file found, you can proceed.')\n",
    "except IOError:\n",
    "    print('ATTENTION: Data file not found, please check the filename.\\n'\n",
    "          '           (current value \"%s\")' % filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In case of file not found, please double check the file name\n",
    "and that the file has been uploaded."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Load the data\n",
    "\n",
    "We start by loading the software:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import phconvert as phc\n",
    "print('phconvert version: ' + phc.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we load the input file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "d, meta = phc.loader.nsalex_ht3(filename,\n",
    "                                donor = 1,\n",
    "                                acceptor = 0,\n",
    "                                alex_period_donor = (1540, 3050),\n",
    "                                alex_period_acceptor = (150, 1500),\n",
    "                                excitation_wavelengths = (470e-9, 635e-9),\n",
    "                                detection_wavelengths = (525e-9, 690e-9),\n",
    "                                )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we plot the `nanotimes` histogram:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "phc.plotter.alternation_hist(d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The previous plot is the `nanotimes` histogram for the donor and acceptor channel separately.\n",
    "The shaded areas marks the donor (*green*) and acceptor (*red*) excitation periods.\n",
    "\n",
    "If the histogram looks wrong in some aspects (no photons, wrong detectors\n",
    "assignment, wrong period selection) please go back to the previous cell \n",
    "which loads the file and change the parameters until the histogram looks correct.\n",
    "\n",
    "You may also find useful to see how many different detectors are present\n",
    "and their number of photons. This information is shown in the next cell:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "detectors = d['photon_data']['detectors']\n",
    "\n",
    "print(\"Detector    Counts\")\n",
    "print(\"--------   --------\")\n",
    "for det, count in zip(*np.unique(detectors, return_counts=True)):\n",
    "    print(\"%8d   %8d\" % (det, count))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Metadata\n",
    "\n",
    "In the next few cells, we specify some metadata that will be stored \n",
    "in the Photon-HDF5 file. Please modify these fields to reflect\n",
    "the content of the data file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "author = 'John Doe'\n",
    "author_affiliation = 'Research Institution'\n",
    "description = 'A demonstrative smFRET-nsALEX measurement.'\n",
    "sample_name = 'Doubly-labeled ssDNA partially hybridized to a complementary strand.'\n",
    "dye_names = 'ATTO488, ATTO647N'\n",
    "buffer_name = 'Tris20 mM Ph 7.8'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Conversion\n",
    "<br>\n",
    "<div class=\"alert alert-success\">\n",
    "<p>Once you finished editing the the previous sections you can proceed with\n",
    "the actual conversion. To do that, click on the menu *Cells* -> *Run All Below*.\n",
    "\n",
    "<p>After the execution go to **Section 6** to download the Photon-HDF5 file.\n",
    "</div>\n",
    "\n",
    "The cells below contain the code to convert the input file to Photon-HDF5."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.1 Add metadata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "d['description'] = description\n",
    "\n",
    "d['sample'] = dict(\n",
    "    sample_name=sample_name,\n",
    "    dye_names=dye_names,\n",
    "    buffer_name=buffer_name,\n",
    "    num_dyes = len(dye_names.split(',')))\n",
    "\n",
    "d['identity'] = dict(\n",
    "    author=author,\n",
    "    author_affiliation=author_affiliation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.2 Save to Photon-HDF5\n",
    "\n",
    "This command saves the new file to disk. If the input data does not follows the Photon-HDF5 specification it returns an error (`Invalid_PhotonHDF5`) printing what violates the specs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "phc.hdf5.save_photon_hdf5(d, overwrite=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can check it's content by using an HDF5 viewer such as [HDFView](https://www.hdfgroup.org/products/java/hdfview/).\n",
    "\n",
    "# 6. Load Photon-HDF5\n",
    "\n",
    "We can load the newly created Photon-HDF5 file to check its content:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from pprint import pprint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "filename = d['_data_file'].filename"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "h5data = phc.hdf5.load_photon_hdf5(filename)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "phc.hdf5.dict_from_group(h5data.identity)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "phc.hdf5.dict_from_group(h5data.setup)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "pprint(phc.hdf5.dict_from_group(h5data.photon_data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "h5data._v_file.close()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}