CIVET Quality Control Guidelines
Please note that the quality control (QC) page of the McConnell Brain Imaging Centre website was referenced in the making of this wiki: https://www.bic.mni.mcgill.ca/ServicesSoftware/CIVET-2-1-0-Quality-Control
The overall goal of QCing CIVET outputs is to ensure that the gray matter (GM)/ white matter (WM) classification and extraction of cortical surfaces were successful. To do so, two types of QC files outputted by CIVET are mainly examined: ‘Verify’ and ‘Clasp’ images. These and other QC image files can be found in the verify/ subdirectory of each CIVET output folder (i.e. per subject).
Found in /verify/*verify.png. Each row of the image displays a snapshot of the scan at various stages of the pipeline. The first row shows the linearly registered brain and the second row shows the non-linearly transformed brain to the stereotaxic model. Ideally, the brain in the second row should fit inside the red outline, which indicates the stereotaxic model mask. The third row shows the GM/WM classification. The fourth row indicates the quality of the cortical surfaces extraction, and can be used to evaluate the accuracy of the tissue boundaries (red outline = pial surface, blue outline = white/gray surface) extracted by CIVET. The surfaces extraction is most important for computing accurate structural measures inside CIVET (e.g., cortical thickness). Thus, we will mostly be referencing the fourth row for our QC.
Found in verify/*clasp.png. They provide more detailed information about surface extraction. The first two rows show the WM surface, which should be examined for a jagged appearance or pieces of skull attached. The third and fourth rows show the GM surface, which should be examined for noticeable overall puffiness. Generally, the white and gray surfaces should look distinct and not very close in shape. If this is not the case, there were likely errors with surface extraction.
The last two rows of the clasp image show cortical thickness tlaplace maps. In general, the brains should be a uniform green colour, although certain cortical areas are expected to be thinner (cooler colour, often in somatomotor regions) or thicker (warmer colour, often in anterior temporal regions). However, tlaplace maps with widespread colour deviations should be flagged. These indicate abnormalities in cortical thickness patterns possibly stemming from motion, poor bias field correction, processing errors, or individual variation. Scans with seemingly unusual cortical thickness patterns should be further examined with other QC files such as the verify images, _laplace.png and _converg.png images showing convergence of gray surface expansion, or _surfsurf.png images showing intersections of GM and WM surfaces.
PyQC is a useful program for efficiently viewing a set of QC images, scoring them, and storing the ratings in a .csv file. Instructions for use can be found here: https://github.com/CoBrALab/PyQC
If using X2Go or at the CIC:
> module load PyQC
> PyQC.py */verify/*verify.png
# Or */verify/*clasp.pngFor CIVET QC, the first column in PyQC can be kept blank (can fill with 0). Use keyboard numbers to indicate QC ratings (1=1; 2=0.5; 3=0). When finished, File → Save as; save as .csv.
If using on your own computer (the easiest way to do it remotely):
> wget https://github.com/CoBrALab/PyQC.git # ensure dependencies are also installed according to PyQC GitHub page
> python PyQC.py *png # must be run from directory containing PyQCNote that scrolling through the processed brain volume in Display (i.e., slice by slice) may be useful. An alternative way to visualize CIVET outputs is to open the final scans in Display with the extracted surfaces overlaid:
> module load minc-toolkit minc-toolkit-extras
> Display final/*tal.mnc surfaces/*{gray,white}_surface_{left,right}*obj (Note: if using X2Go, you will first need to transfer your CIVET outputs to the CIC).
3 point scale: 0 - 0.5 - 1
- Significant errors involving one or many of the following: parts of cortex missing, under-segmentation or over-segmentation of GM/WM, misclassification of GM/WM or brain/non-brain tissue (e.g. meninges, skull, neck muscles, optic nerve, orbital bone classified as brain). A failed rating may also be given to scans with several less obvious yet decently-sized errors or numerous small errors spread across many slices.
- Obvious pathologies
- Many GM/WM boundary intersections
- Small parts of GM/WM missed, over- or under-segmented, and/or misclassified
- Ideally minimal and localized to one or few parts of the cortex
- These might be quite common depending on your dataset
- Accurate extraction of GM and WM surfaces (red and blue lines nicely follow anatomical boundaries), no missing parts of cortex
- Brain fits nicely inside the mask (good registration)
- No major errors in the clasp surfaces or cortical thickness maps
Typically, scans with a rating of 1 or 0.5 (1 or 2 in PyQC) are considered "passes".
Hospital for Sick Children
Philadelphia Neurodevelopmental Cohort
Philadelphia Neurodevelopmental Cohort
Hospital for Sick Children
Philadelphia Neurodevelopmental Cohort
Hospital for Sick Children
General note: The recommended CIVET setting '-mask-hippocampus' excludes the hippocampus as part of the cortex. Therefore, this region will not be segmented into GM/WM and should not be considered when assigning QC ratings.
a) Appearance of WM underestimation in somatomotor areas
In CIVET 2.1.1, cortical myelin in the somatomotor (or adjacent) areas is often classified as WM due to the limitations of contrast-based classifiers. This can lead to the appearance of WM underestimation (seen in the fourth row of the verify.png), where WM seems to extend beyond the white surface boundary. In such cases, it is useful to inspect the scans in the first row to identify the location of the true WM boundary within these regions. If the location of the WM outline in the fourth row (blue) appears to follow the tissue boundary visually identifiable in the first row, the QC score should not be penalized.
Here, WM appears to be underestimated in the middle slice
But when we compare the WM boundary outline to that of the same slice from the first row, it seems to match well so we shouldn't be concerned
b) Ventricles misclassified as GM/WM
If the misclassification is very small (only slightly encroaches on the ventricle space), the scan may be usable. To determine the extent of this artifact, the final surfaces should be inspected in Display because errors that appear minor on the Verify image may in fact span a sizeable portion of the brain. This misclassification can negatively affect the accuracy of cortical measures. The following scan was failed due to erroneous extraction of surfaces inside the ventricles.
c) Blood vessels
Appear as white spots in the brain, which are generally brighter than the WM in the scan. The presence of blood vessels can be confirmed by opening the raw T1 image (.mnc) in Display and examining the coronal/sagittal/axial views slice by slice. Blood vessels will appear as bright white tubular structures across the brain. Scans which have obvious blood vessels should be selectively rerun with ‘-mask-blood-vessels’ option in CIVET.
Examples from the same subject:
Scans with blood vessel errors may also be accompanied by poor outlining of the Sylvian fissure seen here in the middle slice
d) White surface bridges
Can be observed in the fourth row of the Verify image. They look like segments of the white surface outline (blue) that connect or “bridge” two areas of WM via a line or thin loop, where such a bridge doesn’t actually exist in the original T1 image (first row).
For QC, consider the number of white surface bridges visible, whether they are localized to one slice or found all over the brain, and the overall quality of the surface extraction aside from this artifact for the subject. E.g., the presence of 1-2 white surface bridges in a subject which are either very minor/small or localized to only one area of the brain, accompanied by an otherwise good surface extraction, likely doesn’t warrant a Fail rating.
What they mean: An issue with surface extraction, which could be due to several reasons. If inputs to CIVET are not limited to subjects who passed motion QC, motion in the raw T1 could be a contributing factor. Alternatively, if the inputs are clean, but you consistently notice this artifact in your dataset or find yourself failing multiple subjects due to white surface bridges, you may need to adjust the bias field correction procedure for your data. It is also worth examining the scan for blood vessels, as white matter bridges can often appear near blood vessel spots.
e) GM underestimation in temporal pole
f) GM underestimation in frontal pole
g) Sulci poorly outlined
This subject was given a CIVET QC rating of 1 (near perfect) based on the Verify image. The Clasp image shows that WM surfaces (rows 1-2) appear smooth and not jagged, GM surfaces (rows 3-4) have distinct sulci & gyri and are not noticeably puffy, and tlaplace maps (rows 5-6) show uniform green colouration throughout the cortex apart from primary sensory areas, which are expected to be relatively thinner.
This subject failed CIVET QC. The Verify image contained WM underestimation (observed in the surface outline of row 4) in the right hemisphere, resulting in the appearance of cortical thinning in the Clasp image (widespread blue colour in last row).
This subject failed CVIET QC. The Verify image contained numerous WM bridges across multiple slices. In the Clasp image, WM surfaces appear jagged and ‘messy’, and GM surfaces are puffy with several less distinct sulci.
This subject failed CIVET QC. The Verify image contained regions of significant GM overestimations on the meninges. The Clasp image shows pieces of skull attached to the surfaces, likely due to a masking error. Atypical patterns of cortical thinning also appear in the tlaplace maps.
