Synspy is an interactive volume segmentation tool. Synspy is being developed to support research involving 3D fluorescence microscopy of live Zebrafish synapses.
Synspy is experimental software that is subject to frequent changes in direction depending on the needs of the authors and their scientific collaborators.
Synspy has three usage scenarios:
- A framework for developing batch image analysis tools that can segment and measure very large images.
- A framework for image segmentation tools, where a basic interactive volume rendering capability can complement custom data-processing tools.
- A standalone application for interactively segmenting images with our current segmentation algorithm.
Synspy is developed primarily on Linux with Python 2.7 but also tested on Mac OSX. It has several requirements:
- Volspy volume rendering framework.
- Vispy visualization library to access OpenGL GPUs. A recent development version is needed, and it is tested with the fork karlcz/vispy which is periodically synchronized with the upstream project.
- Numpy numerical library to process N-dimensional data.
- Numexpr numerical expression evaluator to accelerate some Numpy operations.
- PyOpenCL to access OpenCL parallel computing platforms.
- Tifffile for access to OME-TIFF and LSM microscopy file formats.
- MDAnalysis for basic transformation matrix utilities.
- Python-PCL bindings for
point-cloud registration algorithms.
- Point Cloud Library (PCL) is the
actual library. We test with
pcl-devel
on Fedora Linux. - PCL is optional and only used by
synspy.analyze.pair
routines.
- Point Cloud Library (PCL) is the
actual library. We test with
- Install all third-party prerequisites.
- Check out the development code from GitHub for Synspy and Volspy.
- Install with
python setup.py install
for each of Synspy and Volspy.
- Obtain a sample 2 channel 3D TIFF image such as: http://www.isi.edu/~karlcz/sample-data/zebra-d19-03b-D.ome.tiff.gz Warning: this is a large 886 MB file!
- Launch the viewer
synspy-viewer zebra-d19-03b-D.ome.tiff
- Interact with the viewer:
- Press
ESC
when you have had enough. - Press
?
to print UI help text to console output. - The mouse can be used to control the view or interact with segments.
- Mouse-over on segments will display some pop-up diagnostics.
- Dragging button 1 will rotate the image
- Dragging button 2 or button 3 will enable temporary slicing and cropping modes where the vertical axis controls the depth of a cutting plane.
- Clicking a segment will toggle its manual classification state.
- The
f
andF
keys control the minimum feature intensity threshold for classifying blobs as synapses. - The
n
andN
keys control the maximum neighborhood threshold to reject blobs with bright background. - The
m
andM
keys control the maximum mask threshold to reject blobs that are in autofluorescing regions according to the red-channel mask. - The
z
andZ
keys control the zoom level. - The
g
andG
keys control the gain level. - The number keys
0
to9
with and without shift modifier set gain. - The
t
andT
keys control the transparency floor, the voxel intensity at which opacity is 0.0. - The
u
andU
keys control the transparency upper bound, the voxel intensity at which opacity is 1.0. - The
o
andO
keys control the opacity factor which is applied after the linear ramp from floor to upper bound. - The
b
key cycles through feature blending mode for on-screen rendering, where unclassified voxels are rendered in blue:- Linear intensity mapping for voxels within classified segments (green for synapses, red for auto-fluorescence).
- Full intensity fill for voxels within classified segments (green for synapses, red for auto-fluorescence).
- The
d
key dumps current parameters to the console output. - The
D
key dumps a voxel classification TIFF image and a segment list CSV file, as well as current threshold parameters. - The
l
key loads a previously dumped segment list CSV file to continue making manual classification decisions in a new session. This also loads the threshold parameters saved to a special row of the CSV file, if found. (For backwards compatibility, older CSV dumps without saved threshold parameters are also supported, leaving the thresholds unchanged.) - The
h
key writes out a 2D histogram of all blobs using the feature intensity and background noise intensity measures as the two plotting axes. - The
e
key endorses currently in-threshold segments as if they have been manually classified as true synapses. - The
E
key expunges manually classified segments, making those segments unclickable so that other adjacent segments are easier to click.
Do not be alarmed by the copious diagnostic outputs streaming out on the console. Did we mention this is experimental code?
Follow the same procedure as above, except set the environment
variable SYNSPY_DETECT_NUCLEI=true
when launching the
synspy-viewer
application. This changes the size of the footprints
used for scale-sensitive blob detection, so that large nuclei scale
blobs are detected and classified instead of small synapse-scale
blobs.
We have only rudimentary support for registering multiple images when tracked in the catalog:
- Acquire two images, such as a before and after image of the same tissue and submit to catalog.
- Define image regions for nucleic and synaptic regions of interest to compare.
- Use the
synspy-launcher
GUI to perform segment classification on all four regions. - Create an image pair study for the two nucleic regions.
- Create a synaptic pair study for the two synaptic regions.
- Wait for the catalog to produce registered pointclouds.
- Launch
synspy-register syn_study_id
to interactively view the aligned pointclouds with a pairwise matching algorithm:- This requires credentials obtained in advance via the
deriva-auth
authentication agent
- This requires credentials obtained in advance via the
In the interactive viewer, the 1
... 0
keys can be pressed to
toggle the visibility of the sub-plots:
1
: unmatched nuclei from image 12
: matched nuclei from image 13
: segments connecting matched nuclei4
: matched nuclei from image 25
: unmatched nuclei from image 26
: unmatched synapses from image 17
: matched synapses from image 18
: segments connecting matched synapses9
: matched synapses from image 20
: unmatched synapses from image 2
The point matching and the pointcloud viz can be adjusted by environment parameters:
SYNSPY_HOST
: the hostname of the server (defaultsynapse.isrd.isi.edu
)SYNSPY_CATALOG
: the catalog identifier (default1
)DERIVA_CREDENTIALS
: the filename where credentials are stored (default unset to use those fromderiva-auth
)- Nucleic pairing parameters
NUC_PAIRING_RADIUS
: distance allowed between paired nuclei in microns (default 15.0)NUC_CORE_DX_RATIO
: intensity units/micron for 4D nearest-neighbor (default unset for 3D nearest-neighbor)NUC_MAX_RATIO
: maximum intensity ratio to accept for pairs
- Synaptic pairing parameters
SYN_PAIRING_RADIUS
: distance allowed between paired synapses in microns (default 5.0)SYN_CORE_DX_RATIO
: intensity units/micron for 4D nearest-neighbor (default unset for 3D nearest-neighbor)SYN_MAX_RATIO
: maximum intensity ratio to accept for pair
BACKGROUND_RGB
: 3-channel RGB in normalized 0.0-1.0 space (default0.15,0.15,0.15
for a dark gray)SHOW_AXES
:true
enables andfalse
disables axes arrows (defaulttrue
)
Several environment variables can be set to modify the behavior of the synspy-viewer
tool on a run-by-run basis, most of which are in common with the volspy-viewer
:
PEAKS_DIAM_FACTOR
(default0.75
) controls the diameter of a gaussian low-pass filter that is used to smooth the image before doing local maxima detection. This factor adjusts the diameter relative to the built-in synapse-diameter that is used to model the core intensity distribution of synapse candidates. A smaller diameter will allow the detection of more closely spaced local maxima but may introduce errors as pixel-level noise begins to dominate.DUMP_PREFIX
controls how dumped files are named. When running ondir1/ImgZfAbc20161231A1.ome.tiff
the default behaves as if you specifiedDUMP_PREFIX=./ImgZfAbc20161231A1.
and will name dump files such as./ImgZfAbc20161231A1.synapses-only.csv
.VOXEL_SAMPLE
selects volume rendering texture sampling modes fromnearest
orlinear
(default for unspecified or unrecognized values).VIEW_MODE
changes the scalar field that is volume-rendered:raw
renders the raw data (default)dog
renders a difference-of-gaussians transform to emphasize synapse-scale changes
ZYX_SLICE
selects a grid-aligned region of interest to view from the original image grid, e.g.0:10,100:200,50:800
selects a region of interest where Z<10, 100<=Y<200, and 50<=X<800. (Default slice contains the whole image.)ZYX_VIEW_GRID
changes the desired rendering grid spacing. Set a preferred ZYX micron spacing, e.g.0.5,0.5,0.5
which the program will try to approximate using integer bin-averaging of source voxels but it will only reduce grid resolution and never increase it. NOTE: Y and X values should be equal to avoid artifacts with current renderer. (Default grid is 0.25, 0.25, 0.25 micron.)ZYX_IMAGE_GRID
overrides any voxel grid spacing information from the image file metadata, useful in case it is absent or incorrect.ZYX_BLOCK_SIZE
changes the desired sub-block work unit size for decomposing large images to control Numpy or OpenCL working set size. Set a preferred ZYX voxel count, e.g.256,384,512
which the program will try to approximate to find an evenly divisible block layout.MAX_3D_TEXTURE_WIDTH
hints the per-dimension size of the volume cube loaded into an OpenGL texture. This affects the sampling precision along the ray-casting integration in the renderer. (Default is768
.)ZNOISE_PERCENTILE
enables a sensor noise estimation by calculating the Nth percentile value along the Z axis, e.g.ZNOISE_PERCENTILE_5
estimates a 2D noise image as the 5th percentile value across the Z stack, and subtracts that noise image from every slice in the stack as a pre-filtering step. WARNING: use of this feature causes the entire image to be loaded into RAM, causing a significantly higher minimum RAM size for runs with large input images. (Default is no noise estimate.)ZNOISE_ZERO_LEVEL
controls a lower value clamp for the pre-filtered data when percentile filtering is enabled. (Default is0
.)
SYNSPY_AUTO_DUMP_LOAD=true
: Automatically load an existing segments CSV file on startup and dump one on exit via theESC
key. See relatedDUMP_PREFIX
for how segments CSV file must be named. WARNING*: exiting the application by other means such as using the[X]
close button on the window decoration MAY bypass the application and exit without dumping the latest voxel classifications.SYNSPY_SPLAT_SIZE
allows some influence over the size of the splat used to represent centroids in the renderer. The valid range is0.0
to2.0
(default1.0
). The splat is always derived from a threshold on a 3D gaussian distribution in a small box, with at least the central voxel selected. The value0
is the smallest possible splat. The value1
attempts to preserve the legacy spheroid splat. The value2
expands the splat to completely fill the enclosing box. Other intermediate values will select incrementally larger or smaller voxel sets between those extremes.
The ZYX_SLICE
, ZYX_VIEW_GRID
, and MAX_3D_TEXTURE_WIDTH
parameters have different but inter-related effects on the scope of the volumetric visualization.
- The
ZYX_VIEW_GRID
can control down-sampling of voxels in arbitrary integer ratios, e.g. to set a preferred grid resolution that can differentiate features of a given size without wasting additional storage space on irrelevant small-scale details. This can save overall RAM required to store the processed volume data by reducing the global image size. The down-sampling occurs incrementally as each sub-block is processed by the block-decomposed processing pipeline. - The
ZYX_SLICE
can arbitrarily discard voxels and thus reduce the final volume size, though discarded voxels may be temporarily present in RAM and require additional memory allocation at that time. - The
MAX_3D_TEXTURE_WIDTH
can avoid allocating oversized OpenGL 3D textures which would either cause a runtime error or unacceptable performance on a given hardware implementation. This can save overall texture RAM required to store the volume data on the GPU, but actually increases the host RAM requirements since it generates a multi-resolution pyramid on the host from which different 3D texture blocks are retrieved dynamically.
Please direct questions and comments to the project issue tracker at GitHub.
Synspy is made available as open source under the (new) BSD License. Please see the LICENSE file for more information.
Synspy and Volspy are developed in the Informatics group at the USC Information Sciences Institute. The computer science researchers involved are:
- Karl Czajkowski
- Carl Kesselman