Skip to content

Evaluate Segmentation (EvaluateSeg)

Stephen Plaza edited this page Jul 26, 2015 · 2 revisions

The goal of the evaluate segmentation workflow is to analyze the similarities/differences between two segmentations over the same dataset. From a Big Data perspective the workflow is very simple and only involves one Spark job with one stage.

  1. Divide volumes into several small subvolumes. The size of the volume can be adjusted to whatever gives the best cluster performance.

  2. Compute the intersections between labels for each subvolume.

  3. Collect all the results and analyze at the driver (generally pretty quick)

The algorithm computes a couple standard similarity measures like Variation of Information (VI) and the Rand Index (rand), but also provides several enhancements to provide more insights into the quality of the segmentation.

  • Decomposes the metrics to be a function of a given body label and a function of the X,Y,Z location in the volume. In this manner, there is not just one number produced for the volume and more insight can be gained about where the worst / best areas are.

  • Defines a new per body VI metric that describes how bad a body was segmented.

  • Implements a thresholded edit distance function to figure out the distance between two segmentations.

  • Introduces a new connectivity metric based on thresholded contacts.

  • Introduces the notion of a comparison type. The services allows comparisons to be made over voxels or other particular points such as skeletal points or synapses. There is special handling of synapses points for use in connectomic applications.

  • A body goodness measure that scores by the maximum overlap of a segmentation label (given some constraints)

  • Some options to filter noise in the ground truth segmentation (such as edge dilation and small body filters) to provide a more robust comparison.

Clone this wiki locally