diff --git a/docs/validation.rst b/docs/validation.rst index a0616a9..d225266 100644 --- a/docs/validation.rst +++ b/docs/validation.rst @@ -1,6 +1,9 @@ Validation of the Cellocity Software ==================================== +Validation dataset +------------------ + In order to validate the underlying analyzers in Cellocity we have generated a “ground truth”, real-world microscopy dataset. The dataset was generated by translating and imaging, with DIC contrast, a fixed monolayer of primary gut epithelium on a high precision linear microscope stage, using a wide selection of magnifications. 10 images were acquired with the stage translated 1 :math:`{\mu m}` in either the X, Y or both directions simultaneously between frames. Images were acquired on a Nikon Eclipse Ti-2 microscope, equipped with a Photometrics Prime 95B camera (1608x1608, 11 um pixel size). @@ -43,7 +46,15 @@ The general structure of the dataset is outlined in the table below. This dataset allowed us compare the "golden standard" of cell layer dynamics analysis, Particle Image Velocimetry (PIV) analysis, with the less frequently used Optical Flow analysis. Our conclusion mirror what was found in [#vig]_, which is that Optical Flow analysis is indeed superior to PIV analysis, both with respect to accuracy and efficiency. -The following section will substantiate this finding. All analyses were run on a early 2020 Dell XPS 15 7590 laptop, running Windows 10. The dataset has been deposited into the BioStudies database with the accession number `S-BSST461 `_ and can be downloaded from there. +The following section will substantiate this finding. All analyses were run on a early 2020 Dell XPS 15 7590 laptop, running Windows 10. + +Downloading the validation dataset +---------------------------------- + +The dataset has been deposited into the BioStudies database with the accession number `S-BSST461 `_ and can be downloaded from there. + +Performing the validation on your local installation +---------------------------------------------------- All the validation figures can be re-generated on your local install by running the following code: @@ -59,6 +70,11 @@ All the validation figures can be re-generated on your local install by running After some time you should have generated the 3 figures below in this chapter in your chosen output folder. + + +Process time +------------ + .. figure:: _static/process_time_compare.png :align: left :alt: Figure comparing processing time Optical Flow vs PIV @@ -70,6 +86,9 @@ Since the dataset was created by translating a high precision stage on a well ca In our case we translated the stage 1 :math:`{\mu m}` between images, and if we set the frame interval to 1 second, then the speed should be 1 :math:`{\mu m/s}` for the X and Y translation and :math:`\sqrt{2} = 1.42` :math:`{\mu m/s}` for the X+Y translation. +Analysis of flow speeds +----------------------- + .. figure:: _static/avg_speed_compare.png :align: left :alt: Figure comparing average speed calculated from Optical Flow vs PIV @@ -79,6 +98,9 @@ and :math:`\sqrt{2} = 1.42` :math:`{\mu m/s}` for the X+Y translation. Both ``Analyzers`` produce results close to the expected, but the ``OpenPivAnalyzer`` has a tendency to underestimate the speed and has greater variance. Cell monolayers grown on loose hydrogel support, as those used in our validation dataset here, are seldom completely planar and portions are often out of focus during imaging. This phenomenon has also been captured in the analysis. If we draw a visualization of the flow generated superimposed on the background ``Channel``, we can study this phenomenon in more detail. + +Qualitative vector field comparison +----------------------------------- .. figure:: _static/40X_vector_panels_compare.png :align: left @@ -90,6 +112,9 @@ Cell monolayers grown on loose hydrogel support, as those used in our validation Studying the above figure allows us to get a deeper understanding of why optical flow and PIV differ. Note that the area in the bottom right corner is not properly focused. This causes the PIV algorithm problems in accurately determining the flow, as illustrated by the inhomogeneities in the vector field. This error can be quantified by calculating the alignment index, a measurement on how well each component vector aligns with the average flow. In our test dataset the flow should be close to completely uniform, giving an expected alignment index of 1.0. +Quantitative vector field comparison +------------------------------------ + .. figure:: _static/alignment_index_compare.png :align: left :alt: Figure comparing average frame alignment index from Optical Flow vs PIV @@ -116,5 +141,5 @@ To our knowledge, there has not been a systematic evaluation of different pre-pr References ---------- -.. [#vig] Dhruv K. Vig and Alex E. Hamby and Charles W. Wolgemuth. On the Quantification of Cellular Velocity Fields. **Biophysical Journal**, 110:1469-1475, 2016. `doi:10.1016/j.bpj.2016.02.032. `_ +.. [#vig] Dhruv K. Vig and Alex E. Hamby and Charles W. Wolgemuth. On the Quantification of Cellular Velocity Fields. *Biophysical Journal*, 110:1469-1475, 2016. `doi:10.1016/j.bpj.2016.02.032. `_