notebooks/Writing Photon-HDF5 files.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Writing Photon-HDF5 files in python\n",
    "\n",
    "> This notebook shows how to create a [Photon-HDF5](http://photon-hdf5.org) file\n",
    "> from scratch using dummy data and [phconvert](http://photon-hdf5.github.io/phconvert/).\n",
    "> \n",
    "> For more info see [Writing Photon-HDF5 files](http://photon-hdf5.readthedocs.org/en/latest/writing.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import phconvert as phc\n",
    "phc.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Create some dummy data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "timestamps = np.random.randint(low=0, high=2e8, size=10000).astype('int64')\n",
    "timestamps.sort()\n",
    "timestamps_unit = 10e-9  # 10 ns, units are always S.I."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "detectors = np.random.randint(low=0, high=1, size=10000).astype('uint8')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Create some metadata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "description = 'This is a fake dataset which mimics smFRET data.'\n",
    "\n",
    "author = 'Author Name'\n",
    "author_affiliation = 'Name of Research Institution'\n",
    "\n",
    "sample_name = 'describe the sample here'\n",
    "buffer_name = 'describe the buffer here'\n",
    "dye_names = 'Cy3B, ATTO647N'   # Comma separates names of fluorophores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Create Photon-HDF5 data structure\n",
    "\n",
    "In this section we create all the mandatory and non mandatory groups.\n",
    "Not all of the are required to save a valid Photon-HDF5 file\n",
    "(see example in section 4).\n",
    "\n",
    "## 3.1 `photon_data` group\n",
    "\n",
    "Contains arrays of photon-data: timestamps, detectors, nanotimes, etc...\n",
    "\n",
    "*See [photon_data group reference](http://photon-hdf5.readthedocs.org/en/latest/phdata.html#photon-data-group)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "photon_data = dict(\n",
    "    timestamps=timestamps,\n",
    "    detectors=detectors,\n",
    "    timestamps_specs={'timestamps_unit': timestamps_unit})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 `setup` group\n",
    "\n",
    "The `/setup` group contains information about the measurement setup. \n",
    "\n",
    "*See [setup group reference](http://photon-hdf5.readthedocs.org/en/latest/phdata.html#setup-group).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "setup = dict(\n",
    "    ## Mandatory fields\n",
    "    num_pixels = 2,                   # using 2 detectors\n",
    "    num_spots = 1,                    # a single confoca excitation\n",
    "    num_spectral_ch = 2,              # donor and acceptor detection \n",
    "    num_polarization_ch = 1,          # no polarization selection \n",
    "    num_split_ch = 1,                 # no beam splitter\n",
    "    modulated_excitation = False,     # CW excitation, no modulation \n",
    "    excitation_alternated = [False],  # CW excitation, no modulation \n",
    "    lifetime = False,                 # no TCSPC in detection\n",
    "    \n",
    "    ## Optional fields\n",
    "    excitation_wavelengths = [532e-9],         # List of excitation wavelenghts\n",
    "    excitation_cw = [True],                    # List of booleans, True if wavelength is CW\n",
    "    detection_wavelengths = [580e-9, 640e-9],  # Nominal center wavelength \n",
    "                                               # each for detection ch\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3 `provenance` group\n",
    "\n",
    "Non-mandatory group containing info about the original file \n",
    "prior to Photon-HDF5 conversion. If some information is not \n",
    "available the relative field may be omitted.\n",
    "\n",
    "*See [provenance group documentation](http://photon-hdf5.readthedocs.org/en/latest/phdata.html#provenance-group).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "provenance = dict(\n",
    "    filename='original_data_file.dat', \n",
    "    software='Acquisition Software Name')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.4 `identity` group\n",
    "\n",
    "Non-mandatory group containing info about information \n",
    "this specific Photon-HDF5 file.\n",
    "\n",
    "*See [identity group documentation](http://photon-hdf5.readthedocs.org/en/latest/phdata.html#identity-group).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "identity = dict(\n",
    "    author=author,\n",
    "    author_affiliation=author_affiliation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.5 `measurement_specs` group\n",
    "\n",
    "The optional /photon_data/measurement_specs group contains \n",
    "additional information allowing unambiguous interpretation \n",
    "of the data for each specific type of measurement.\n",
    "\n",
    "*See [measurement_specs group documentation](http://photon-hdf5.readthedocs.org/en/latest/phdata.html#measurement-specs).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "measurement_specs = dict(\n",
    "    measurement_type = 'smFRET',\n",
    "    detectors_specs = {'spectral_ch1': [0],  # list of donor's detector IDs\n",
    "                       'spectral_ch2': [1]}  # list of acceptor's detector IDs\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Save Photon-HDF5 files\n",
    "\n",
    "To save a file we need to join together the root fields and group \n",
    "in a single dictionary. Here we provide a few examples.\n",
    "\n",
    "## 4.1 Minimal file\n",
    "\n",
    "Create a bare-bone Photon-HDF5 file with only mandatory fields."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "data = dict(\n",
    "    description=description,\n",
    "    photon_data = photon_data,\n",
    "    setup=setup,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "phc.hdf5.save_photon_hdf5(data, h5_fname='dummy_dataset_barebone.h5', overwrite=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **NOTE:** a user of this file can read the data but does not know\n",
    "> what kind of measurement it is (e.g. smFRET with single laser\n",
    "> excitation and 2-colors detection)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2 Expanded Photon-HDF5\n",
    "\n",
    "Create a Photon-HDF5 with non-mandatory fields (including `measurement_specs`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "photon_data['measurement_specs'] = measurement_specs\n",
    "\n",
    "data = dict(\n",
    "    description=description,\n",
    "    photon_data = photon_data,\n",
    "    setup=setup,\n",
    "    identity=identity,\n",
    "    provenance=provenance\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "phc.hdf5.save_photon_hdf5(data, h5_fname='dummy_dataset_complete.h5', overwrite=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **NOTE:** a user of this file can correctly interpret the data\n",
    "> reading that the measurement type is 'smFRET' (meaning smFRET with single laser\n",
    "> excitation and 2-colors detection) and the IDs of donor and acceptor detectors\n",
    "> (from `detectors_specs/spectral_ch1` and `spectral_ch2` respectively)."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}