Skip to content

Commit

Permalink
Update validation.rst
Browse files Browse the repository at this point in the history
Add Headline sections
  • Loading branch information
Oftatkofta committed Sep 4, 2020
1 parent c37d763 commit 32f0253
Showing 1 changed file with 27 additions and 2 deletions.
29 changes: 27 additions & 2 deletions docs/validation.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
Validation of the Cellocity Software
====================================

Validation dataset
------------------

In order to validate the underlying analyzers in Cellocity we have generated a “ground truth”, real-world microscopy dataset.
The dataset was generated by translating and imaging, with DIC contrast, a fixed monolayer of primary gut epithelium on a high precision linear microscope stage, using a wide selection of magnifications.
10 images were acquired with the stage translated 1 :math:`{\mu m}` in either the X, Y or both directions simultaneously between frames. Images were acquired on a Nikon Eclipse Ti-2 microscope, equipped with a Photometrics Prime 95B camera (1608x1608, 11 um pixel size).
Expand Down Expand Up @@ -43,7 +46,15 @@ The general structure of the dataset is outlined in the table below.

This dataset allowed us compare the "golden standard" of cell layer dynamics analysis, Particle Image Velocimetry (PIV) analysis, with the less frequently used Optical Flow analysis.
Our conclusion mirror what was found in [#vig]_, which is that Optical Flow analysis is indeed superior to PIV analysis, both with respect to accuracy and efficiency.
The following section will substantiate this finding. All analyses were run on a early 2020 Dell XPS 15 7590 laptop, running Windows 10. The dataset has been deposited into the BioStudies database with the accession number `S-BSST461 <https://www.ebi.ac.uk/biostudies/studies/S-BSST461>`_ and can be downloaded from there.
The following section will substantiate this finding. All analyses were run on a early 2020 Dell XPS 15 7590 laptop, running Windows 10.

Downloading the validation dataset
----------------------------------

The dataset has been deposited into the BioStudies database with the accession number `S-BSST461 <https://www.ebi.ac.uk/biostudies/studies/S-BSST461>`_ and can be downloaded from there.

Performing the validation on your local installation
----------------------------------------------------

All the validation figures can be re-generated on your local install by running the following code:

Expand All @@ -59,6 +70,11 @@ All the validation figures can be re-generated on your local install by running
After some time you should have generated the 3 figures below in this chapter in your chosen output folder.



Process time
------------

.. figure:: _static/process_time_compare.png
:align: left
:alt: Figure comparing processing time Optical Flow vs PIV
Expand All @@ -70,6 +86,9 @@ Since the dataset was created by translating a high precision stage on a well ca
In our case we translated the stage 1 :math:`{\mu m}` between images, and if we set the frame interval to 1 second, then the speed should be 1 :math:`{\mu m/s}` for the X and Y translation
and :math:`\sqrt{2} = 1.42` :math:`{\mu m/s}` for the X+Y translation.

Analysis of flow speeds
-----------------------

.. figure:: _static/avg_speed_compare.png
:align: left
:alt: Figure comparing average speed calculated from Optical Flow vs PIV
Expand All @@ -79,6 +98,9 @@ and :math:`\sqrt{2} = 1.42` :math:`{\mu m/s}` for the X+Y translation.
Both ``Analyzers`` produce results close to the expected, but the ``OpenPivAnalyzer`` has a tendency to underestimate the speed and has greater variance.

Cell monolayers grown on loose hydrogel support, as those used in our validation dataset here, are seldom completely planar and portions are often out of focus during imaging. This phenomenon has also been captured in the analysis. If we draw a visualization of the flow generated superimposed on the background ``Channel``, we can study this phenomenon in more detail.

Qualitative vector field comparison
-----------------------------------

.. figure:: _static/40X_vector_panels_compare.png
:align: left
Expand All @@ -90,6 +112,9 @@ Cell monolayers grown on loose hydrogel support, as those used in our validation
Studying the above figure allows us to get a deeper understanding of why optical flow and PIV differ. Note that the area in the bottom right corner is not properly focused. This causes the PIV algorithm problems in accurately determining the flow, as illustrated by the inhomogeneities in the vector field.
This error can be quantified by calculating the alignment index, a measurement on how well each component vector aligns with the average flow. In our test dataset the flow should be close to completely uniform, giving an expected alignment index of 1.0.

Quantitative vector field comparison
------------------------------------

.. figure:: _static/alignment_index_compare.png
:align: left
:alt: Figure comparing average frame alignment index from Optical Flow vs PIV
Expand All @@ -116,5 +141,5 @@ To our knowledge, there has not been a systematic evaluation of different pre-pr
References
----------

.. [#vig] Dhruv K. Vig and Alex E. Hamby and Charles W. Wolgemuth. On the Quantification of Cellular Velocity Fields. **Biophysical Journal**, 110:1469-1475, 2016. `doi:10.1016/j.bpj.2016.02.032. <https://doi.org/10.1016/j.bpj.2016.02.032>`_
.. [#vig] Dhruv K. Vig and Alex E. Hamby and Charles W. Wolgemuth. On the Quantification of Cellular Velocity Fields. *Biophysical Journal*, 110:1469-1475, 2016. `doi:10.1016/j.bpj.2016.02.032. <https://doi.org/10.1016/j.bpj.2016.02.032>`_

0 comments on commit 32f0253

Please sign in to comment.