bgamari/photon-tools

Python utilities for working with photon timestamp data from fluorescence spectroscopy experiments
Python C++ Other
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
 Failed to load latest commit information. corr @ bf8963a doc photon_tools picoharp squmfit @ 8f7419b tests .gitignore .gitmodules COPYING Makefile anisotropy blink_removal.py dls-mem fcs-corr fcs-fit fcs-mem imbalance install.sh lifetime-deconvolve plot-bins plot-fret readme.mkd requirements.txt setup.py splice-times split-times summarize-timestamps trim-stamps wavelet_denoise.py

photon-tools - Tools for analysis of single-photon measurement data

photon-tools is a collection of tools for the manipulation and analysis of photon timestamp data, particularly from FRET and FCS experiments.

Installation: the two-minute version

To install photon-tools on Ubuntu,

$sudo apt-get install python3 python3-numpy python3-scipy python3-matplotlib \ python3-setuptools build-essential cython3 libboost-all-dev$ git clone git://github.com/bgamari/photon-tools.git
$cd photon-tools$ ./install.sh

Installation: the unabridged version

Many of these utilities are written in Python and generally require Python 3 or greater along with numpy. In particular, some optimized modules require Cython. Utilities capable of producing plots require the matplotlib Python plotting library. On the whole, photon-tools depends on,

• Gnu make
• Python3
• Numpy
• Scipy
• Matplotlib >= 1.4
• Cython >= 0.15
• Boost (boost-format in particular)

The scripts and libraries included in photon-tools can be installed like any Python distutils package,

$sudo ./setup.py install Note that running scripts within the photon-tools/ root directory will require that the Cython code is built in-place, due to limitations of Python's module name resolution scheme. To do this, one must run,$ ./setup.py build_ext --inplace

Supported formats

Utilities requiring timestamp data as input accept data in the following formats,

• Raw 64-bit integer timestamps (read as little endian; use extension .times)
• Raw 64-bit integer timestamps and associated 8-bit channel numbers (read as little endian; use extension .timech)
• Picoquant PT2 (.pt2, .pt3)
• Goldner FPGA timetagger (.timetag)

In all of these cases, the utilities will attempt to figure out the period of the timebase (known as the jiffy) from whatever metadata is available in the format.

Tools

The tools that photon-tools provides are command-line utilities following typical UNIX argument conventions. That is, most arguments are delimited by a dash and have both a long form (--output) and a short form (-o).

Below is a set of simple examples describing basic usage of the tools. These are, however, only basic examples and do not show all of the features of these tools. Full help for each utility is always available with from --help.

plot-bins

The plot-bins utility produces a binned timeseries plot of a photon timestamp data set. This is useful to quickly visualize the trajectory of an experiment. For example, to get a high-level view of the intensity in a FRET experiment, one might want to plot the binned intensity over a long duration (say 5 rows of 20 seconds each) with a bin width of 10 ms,

$plot-bins --rows=5 --row-width=20 --bin-width=1e-2 2012-07-26-run_013.timetag This will produce a plot looking like, Note that plot-bins by default assumes a FRET experiment, taking channel 0 to be the donor and channel 1 to be the acceptor. This can be overridden with the --channel command (e.g. --channel=0=acceptor). fcs-corr In the case of an FCS experiment, the first task in the analysis process is generally to compute a correlation function. photon-tools provides the fcs-corr tool to conveniently compute and plot a correlation function from timestamp data. To compute and plot a correlation function from$\tau$of 1 microsecond to 1 second (which is the default range, but we will set it explicitly here for completeness),$ fcs-corr --min-lag=1e-6 --max-lag=1 2012-10-26-run_000.timetag

This will produce three files,

• 2012-10-25-run_001.timetag.acorr-0: the auto-correlation of channel 0
• 2012-10-25-run_001.timetag.acorr-1: the auto-correlation of channel 1
• 2012-10-25-run_001.timetag.xcorr-0-1: the cross-correlation of channels 0 and 1

Moreover, if we pass the --plot option a plot will be produced of each of these functions,

In addition, fcs-corr supports a simple means of afterpulsing correction correction. If we have a dataset with large afterpulsing signal,

We can take a dataset of uncorrelated emissions (say of a large background signal from room light) and correct for this contribution with,

$fcs-corr my-data.timetag -A room-light.timetag --plot This will produce a signal with the afterpulsing contribution removed. Note, however, that the points at small lags will likely still be quite scattered due to count statistics, fcs-fit After one has computed the correlation function to a data set, it is typical that one would next fit a physically relevant model to the resulting function. The fcs-fit tool provides a means of fitting a model across one or several sets of observations. In the simplest case, it allows one to fit a single model (e.g. a three-dimensional diffusion model, --model=3d_diff) to a correlation function produced by fcs-corr,$ fcs-fit --plot -m3d_diff 9-24-2012-001.pt2.acorr-0
Fitting 1 curves against model 3d_diff

Initial parameters:
a  (fitted)   =    3.000e+00        Aspect ratio
F  (fixed )   =    0.000e+00        Fraction of particles in triplet state
tau_F  (fixed )   =    1.000e+00 us     Triplet state relaxation time
tau_d  (fitted)   =    1.000e+02 us     Diffusion time
n  (fitted)   =    5.000e-01        Concentration
offset  (fixed )   =    0.000e+00        Offset
alpha  (fixed )   =    1.000e+00        Anomalous diffusion exponent (1=normal diffusion)

Fitted parameters:
a  (fitted)   =    1.270e+01        Aspect ratio
F  (fixed )   =    0.000e+00        Fraction of particles in triplet state
tau_F  (fixed )   =    1.000e+00 us     Triplet state relaxation time
tau_d  (fitted)   =    1.370e+01 us     Diffusion time
n  (fitted)   =    3.810e+00        Concentration
offset  (fixed )   =    0.000e+00        Offset
alpha  (fixed )   =    1.000e+00        Anomalous diffusion exponent (1=normal diffusion)

Goodness of fit:
9-24-2012-001.pt2.acorr-0
Chi^2 = 1.642541e+02
Chi^2 / DOF = 1.013914

In the first block of this output we see the parameters of the model (3d_diff), along with their initial values and scope (e.g. whether the parameter is taken to be fixed or will be fitted). After this we see a similar list reflecting the parameter values resulting from the fit. Finally, we see some commonly used measures of goodness-of-fit.

Of course, things aren't always this easy. Fits are notoriously finicky in the presence of imperfect data as demonstrated in this example (omitting --model=3d_diff since this is the default),

$fcs-fit 2012-10-25-run_001.timetag.acorr-0 -p Fitting 1 curves against model 3d_diff Initial Parameters: a (fitted) = 3.000e+00 Aspect ratio F (fixed ) = 0.000e+00 Fraction of particles in triplet state tau_F (fixed ) = 1.000e+00 us Triplet state relaxation time tau_d (fitted) = 1.000e+02 us Diffusion time n (fitted) = 5.000e-01 Concentration offset (fixed ) = 0.000e+00 Offset alpha (fixed ) = 1.000e+00 Anomalous diffusion exponent (1=normal diffusion) Failed to converge Fit failed to converge (flat axis) Looking at the plot produced by fcs-corr, we find the reason for the fit failing is quite clear, Here we see a pronounced triplet artifact starting at around$\tau = $1 microsecond, in addition to a strong indication that the correlation function has not converged to zero a$\tau = 1$second. While the wisdom of forcing such flawed data to fit a model is of course questionable, we can nevertheless do so by cutting out the triplet (--early-cutoff=1e-6) and allowing the model to fit an offset (--set=fitted:offset=0.5),$ fcs-fit --plot 2012-10-25-run_001.timetag.acorr-0 -e 1e-6 -s fit:offset -s fit:tau_F -s fit:F
Fitting 1 curves against model 3d_diff

Initial parameters:
a  (fitted)   =    3.000e+00         Aspect ratio
F  (fixed )   =    0.000e+00         Fraction of particles in triplet state
tau_F  (fixed )   =    1.000e+00 us      Triplet state relaxation time
tau_d  (fitted)   =    1.000e+02 us      Diffusion time
n  (fitted)   =    5.000e-01         Concentration
offset  (fitted)   =    0.000e-01         Offset
alpha  (fixed )   =    1.000e+00         Anomalous diffusion exponent (1=normal diffusion)

Fitted parameters:
a  (fitted)   =    1.193e+01         Aspect ratio
F  (fitted)   =    2.128e-01         Fraction of particles in triplet state
tau_F  (fixed )   =    4.612e+01 us      Triplet state relaxation time
tau_d  (fitted)   =    3.919e+02 us      Diffusion time
n  (fitted)   =    3.146e-01         Concentration
offset  (fitted)   =    1.441e-01         Offset
alpha  (fixed )   =    1.000e+00         Anomalous diffusion exponent (1=normal diffusion)

Goodness of fit:
2012-10-25-run_001.timetag.acorr-0
Chi^2 = 3.26+02
Chi^2 / DOF = 2.13

Here, we have set the offset parameter of the model with an initial value (0.5) and allowed the parameter to be varied by the fit. We find that the fit now fits (only ever so slightly) better,

Occasionally one might want to fit several curves to a model sharing some set of parameters. For example, in an FCS experiment one might take several datasets on the same instrument expecting some parameters (e.g. the aspect ratio of the observation volume) to remain constant over the course of the day. In this case, one can fit these datasets collectively, sharing the common parameters. fcs-fit enables this sort of fit by using

anisotropy (fluorescence lifetime and anisotropy analysis)

anisotropy is a tool for the instrument response correction and fitting of fluorescence lifetime histograms.

The analysis of fluorescence lifetime data roughly resembles the traditionally fitting analysis of FCS data. A histogram from a typical fluorescence lifetime experiment will look something like this,

Here the $x$ axis shows the time since the last excitation. Data such as the above is modelled as some decay model (typically one or more exponential decays) convolved with an instrument response function (IRF). The IRF characterizes the response of the experimental apparatus to an instantaneous emission and is typically measured using a scatterer along with the fluorescence measurements. An IRF histogram might look something like the following,

There are two (at least) means of accounting for the IRF in the analysis:

• Deconvolving the IRF from the observed histogram to recover the underlying decay dynamics. The decay model can then be fit directly to this curve.

• Fit the decay model convolved with the IRF to the observed histogram. This is often known as reconvolution.

The problem of deconvolution is a difficult, and often ill-posed, inverse problem. For this reason anisotropy uses the reconvolution method to account for the IRF.