Lesion inpainting (LIT) integration

FastSurferCNN/run_prediction.py

@@ -129,7 +129,7 @@ class RunModelOnData:
 
     Attributes
     ----------
-    vox_size : float, 'min'
+    vox_size : float, 'min', None
     current_plane : str
     models : Dict[str, Inference]
     view_ops : Dict[str, Dict[str, Any]]
@@ -163,7 +163,7 @@ class RunModelOnData:
         Getter.
     """
 
-    vox_size: float | Literal["min"]
+    vox_size: float | Literal["min"] | None
     current_plane: Plane
     models: dict[Plane, Inference]
     view_ops: dict[Plane, dict[str, Any]]
@@ -261,12 +261,9 @@ def __init__(
                 self.models[plane] = Inference(view["cfg"], ckpt=view["ckpt"], device=self.device, lut=self.lut)
 
         try:
-            __vox_size = _vox_size(vox_size)
-            if __vox_size is None:
-                raise ValueError("vox_size cannot be None")
-            self.vox_size = __vox_size
+            self.vox_size = _vox_size(vox_size)
         except (argparse.ArgumentTypeError, ValueError):
-            condition = "convertible to a float between 0 and 1 or 'min'"
+            condition = "convertible to a float between 0 and 1, 'min', or 'any'"
             raise ValueError(f"Invalid value for vox_size, must be {condition}, was '{vox_size}'.") from None
         self.conform_to_1mm_threshold = conform_to_1mm_threshold
 

README.md

@@ -17,7 +17,7 @@ The FastSurfer pipeline consists of two main parts for segmentation and surface
 - the segmentation sub-pipeline (`seg`) employs advanced deep learning networks for fast, accurate segmentation and volumetric calculation of the whole brain and selected substructures.
 - the surface sub-pipeline (`recon-surf`) reconstructs cortical surfaces, maps cortical labels and performs a traditional point-wise and ROI thickness analysis. 
 
-### Segmentation Modules 
+### Segmentation Modules
 - approximately 5 minutes (GPU), `--seg_only` only runs this part. 
 
 Modules (all run by default):
@@ -44,6 +44,9 @@ Modules (all run by default):
 - requires a FreeSurfer license file as it uses some FreeSurfer binaries internally.
 - requires outputs of the `asegdkt` and the `cc` modules as a prerequisite (can be included in the same run).
 
+### Extensions
+- [FastSurfer-LIT](doc/overview/modules/LIT.md) wraps the FastSurfer segmentation and surface pipelines with lesion inpainting when a lesion mask is provided via `--lesion_mask <path>`. This feature is experimental; review LIT-modified outputs before using them for downstream analyses.
+
 <!-- start of image requirements -->
 ### Requirements to input images
 All pipeline parts and modules require good quality MRI images, preferably from a 3T MR scanner.
@@ -133,13 +136,14 @@ All installation methods use the `run_fastsurfer.sh` call interface (replace the
 <!-- start of examples -->
 Examples
 --------
-The documentation includes [6 detailed Examples](doc/overview/EXAMPLES.md) on how to use FastSurfer. 
+The documentation includes [detailed Examples](doc/overview/EXAMPLES.md) on how to use FastSurfer.
 - [Example 1: FastSurfer Singularity](doc/overview/EXAMPLES.md#example-1-fastsurfer-singularity-or-apptainer)
 - [Example 2: FastSurfer Docker](doc/overview/EXAMPLES.md#example-2-fastsurfer-docker)
 - [Example 3: Native FastSurfer on subjectX with parallel processing of hemis](doc/overview/EXAMPLES.md#example-3-native-fastsurfer-on-subjectx-with-parallel-processing-of-hemis)
 - [Example 4: FastSurfer on multiple subjects](doc/overview/EXAMPLES.md#example-4-fastsurfer-on-multiple-subjects)
 - [Example 5: Quick Segmentation](doc/overview/EXAMPLES.md#example-5-quick-segmentation)
 - [Example 6: Running FastSurfer on a SLURM cluster via Singularity](doc/overview/EXAMPLES.md#example-6-running-fastsurfer-on-a-slurm-cluster-via-singularity)
+- [Example 7: Running FastSurfer with lesion inpainting using neurolit](doc/overview/EXAMPLES.md#example-7-running-fastsurfer-with-lesion-inpainting-using-neurolit)
 
 Output files
 ------------
@@ -207,6 +211,9 @@ _Estrada S, Kuegler D, Bahrami E, Xu P, Mousa D, Breteler MMB, Aziz NA, Reuter M
 
 _Pollak C, Diers K, Estrada S, Kuegler D, Reuter M, FastSurfer-CC: A robust, accurate, and comprehensive framework for corpus callosum morphometry, pre-print on arXiv: https://doi.org/10.48550/arXiv.2511.16471_
 
+Extensions:
+
+_Pollak C, Kuegler D, Bauer T, Rueber T, Reuter M. FastSurfer-LIT: Lesion Inpainting Tool for Whole Brain MRI Segmentation with Tumors, Cavities and Abnormalities. Imaging Neuroscience 2025. https://doi.org/10.1162/imag_a_00446_
 
 Stay tuned for updates and follow us on [X/Twitter](https://twitter.com/deepmilab).
 

doc/overview/EXAMPLES.md

@@ -239,3 +239,40 @@ This will create three dependent SLURM jobs, one to segment, one for surface rec
 There are many intricacies and options, so it is advised to use `--help`, `--debug` and `--dry` to inspect, what will be scheduled as well as run a test on a small subset. More control over subjects is available with `--subject_list`.
 
 The `$HOME/my_mri_data` and the `$HOME/my_fastsurfer_analysis` directories need to be accessible from cluster nodes. Most IO is performed on a work directory (automatically generated from `$HPCWORK` environment variable: `$HPCWORK/fastsurfer-processing/$(date +%Y%m%d-%H%M%S)`). Alternatively, an empty directory can be manually defined via `--work`. On successful cleanup, this directory will be removed to `$HOME/my_fastsurfer_analysis` (defined via `--sd`).
+
+## Example 7: Running FastSurfer with lesion inpainting using neurolit
+
+When T1w images contain large lesions such as tumors, surgical cavities, or other abnormalities,
+FastSurfer segmentation and surfaces can be affected by the altered anatomy. FastSurfer can be
+wrapped with the Lesion Inpainting Tool (LIT) by providing `--lesion_mask <path to file>`.
+
+> **Note:** The FastSurfer LIT extension is currently experimental. Review the LIT-modified
+> outputs before using them for downstream analyses.
+
+### Docker
+```bash
+docker run --gpus all -v /home/user/my_mri_data:/data \
+                      -v /home/user/my_fastsurfer_analysis:/output \
+                      -v /home/user/my_fs_license_dir:/fs_license \
+                      --rm --user $(id -u):$(id -g) deepmi/fastsurfer:latest \
+                      --fs_license /fs_license/license.txt \
+                      --t1 /data/subjectX/t1-weighted.nii.gz \
+                      --lesion_mask /data/subjectX/lesion_mask.nii.gz \
+                      --sid subjectX --sd /output \
+                      --threads 4
+```
+
+### Native
+```bash
+./run_fastsurfer.sh --t1 /home/user/my_mri_data/subjectX/t1-weighted.nii.gz \
+                    --lesion_mask /home/user/my_mri_data/subjectX/lesion_mask.nii.gz \
+                    --sid subjectX --sd /home/user/my_fastsurfer_analysis \
+                    --fs_license /path/to/license.txt \
+                    --threads 4
+```
+
+When using `--lesion_mask <path to file>`, FastSurfer will:
+1. Inpaint the lesion area using LIT.
+2. Run the selected parts of the segmentation and surface pipeline on the inpainted image (note that it is not compatible with `--surf_only`).
+3. Automatically map the lesion mask back into the final output files and regenerate the affected statistics.
+4. Preserve the pre-lesion outputs as `.lit` or mapped backup files and write lesion reports plus `lesion_impact_summary.yaml` in the `stats` directory.

doc/overview/OUTPUT_FILES.md

@@ -96,6 +96,74 @@ The primary output files are pial, white, and inflated surface files, the thickn
 | stats     | wmparc.DKTatlas.mapped.stats                                   | surface | table of white matter segmentation statistics                                                |
 | scripts   | recon-all.log                                                  | surface | logfile                                                                                      |
 
+Lesion Inpainting Tool (LIT, optional)
+--------------------------------------
+When `--lesion_mask <path to file>` is provided, FastSurfer wraps the segmentation and surface
+pipelines with lesion inpainting using LIT. The extension is currently experimental. It inpaints
+the lesion region, runs the requested FastSurfer modules on the inpainted image, and then maps the
+lesion back into the resulting outputs. The current LIT postprocessing workflow updates the primary
+FastSurfer files in place and keeps the original pre-lesion outputs either as `.lit` backups or,
+for some surface-derived files, in the original `.mapped.*` files.
+
+For lesion mask requirements, see the [FastSurfer-LIT module documentation](modules/LIT.md#lesion-mask-requirements).
+
+### Inpainting Outputs
+These are the key files created during the initial inpainting stage. FastSurfer with LIT writes
+these outputs directly into the standard subject directory layout.
+
+| directory | filename | module | description |
+|:----------|----------|--------|-------------|
+| mri | inpainted.lit.nii.gz | lit | inpainted T1 image used for downstream processing |
+| mri | mask.lit.nii.gz | lit | processed lesion mask in FastSurfer image space, after optional preprocessing |
+| mri/orig | mask.lit.nii.gz | lit | original lesion mask stored in the subject directory |
+| mri/orig | inpainting_original_image.lit.nii.gz | lit | conformed original image used internally by LIT |
+| mri/orig | inpainting_masked_image.lit.nii.gz | lit | conformed masked image used internally by LIT |
+| scripts | inpainting_*.lit.png | lit | preview images from the inpainting step |
+
+### Postprocessing MRI Outputs
+These files contain the lesion-integrated segmentations. LIT overwrites the primary FastSurfer outputs and stores the pre-lesion versions as `.lit` backups.
+
+| directory | filename | module | description |
+|:----------|----------|--------|-------------|
+| mri | aparc.DKTatlas+aseg.deep.mgz | lit | lesion-integrated whole-brain segmentation |
+| mri | aparc.DKTatlas+aseg.deep.lit.mgz | lit | backup of the pre-lesion whole-brain segmentation |
+| mri | aseg.auto_noCCseg.mgz | lit | lesion-integrated subcortical segmentation used for VINN statistics |
+| mri | aseg.auto_noCCseg.lit.mgz | lit | backup of the pre-lesion subcortical segmentation |
+| mri | cerebellum.CerebNet.nii.gz | lit | lesion-integrated cerebellum segmentation when CerebNet is available |
+| mri | cerebellum.CerebNet.lit.nii.gz | lit | backup of the pre-lesion cerebellum segmentation |
+| mri | hypothalamus.HypVINN.nii.gz | lit | lesion-integrated hypothalamus segmentation when HypVINN is available |
+| mri | hypothalamus.HypVINN.lit.nii.gz | lit | backup of the pre-lesion hypothalamus segmentation |
+
+### Postprocessing Statistics and Reports
+LIT regenerates the relevant stats files after lesion mapping, keeps the pre-lesion versions as `.lit` backups where applicable, and writes lesion-specific reports.
+
+| directory | filename | module | description |
+|:----------|----------|--------|-------------|
+| stats | lesion_impact_summary.yaml | lit | machine-readable summary of affected brain regions |
+| stats | aparc.DKTatlas+aseg.lesion_report.txt | lit | report of volumetric structures affected by the lesion |
+| stats | aseg.lesion_report.txt | lit | report of affected structures in the FreeSurfer aseg segmentation |
+| stats | aseg+DKT.VINN.stats | lit | lesion-integrated whole-brain/VINN summary statistics |
+| stats | aseg+DKT.VINN.lit.stats | lit | backup of the pre-lesion whole-brain/VINN statistics |
+| stats | aseg.VINN.stats | lit | lesion-integrated subcortical VINN statistics |
+| stats | aseg.VINN.lit.stats | lit | backup of the pre-lesion subcortical VINN statistics |
+| stats | cerebellum.CerebNet.stats | lit | lesion-integrated cerebellum statistics when CerebNet is available |
+| stats | cerebellum.CerebNet.lit.stats | lit | backup of the pre-lesion cerebellum statistics |
+| stats | hypothalamus.HypVINN.stats | lit | lesion-integrated hypothalamus statistics when HypVINN is available |
+| stats | hypothalamus.HypVINN.lit.stats | lit | backup of the pre-lesion hypothalamus statistics |
+
+### Surface-based Outputs
+If the surface pipeline is run, LIT also updates the relevant surface annotations and stats. The
+public annotation paths are kept at the standard FreeSurfer names, while the preserved pre-lesion
+surface stats remain in the corresponding `.mapped.stats` files.
+
+| directory | filename | module | description |
+|:----------|----------|--------|-------------|
+| label | {lh,rh}.aparc.DKTatlas.annot | lit | cortical parcellation with lesion projected onto the surface; symlink to `{lh,rh}.aparc.DKTatlas.mapped.annot` |
+| label | {lh,rh}.aparc.DKTatlas.lit.annot | lit | pre-lesion cortical parcellation; symlink to `{lh,rh}.aparc.DKTatlas.mapped.lit.annot` |
+| stats | {lh,rh}.aparc.DKTatlas.stats | lit | lesion-integrated cortical surface statistics |
+| stats | {lh,rh}.aparc.DKTatlas.mapped.stats | lit | backup of the pre-lesion cortical surface statistics |
+| stats | {lh,rh}.aparc.DKTatlas.anatomy_report.txt | lit | report of cortical structures affected by the lesion |
+
 Longitudinal Processing
 -----------------------
 When running the [longitudinal pipeline](LONG.md) the output will be as above for the individual time point directories. Note that the templateID directory for the within-subject template will not contain all files and usually is not looked at or analyzed, as it represents an intermediate step in the longitudinal pipeline. 

doc/overview/intro.rst

@@ -14,7 +14,7 @@ recommended Docker or Singularity images. But first let us tell you why we think
 * It is compatible with FreeSurfer, enabling FreeSurfer downstream tools to work directly.
 * It is much faster and provides increased reliability and sensitivity of the derived measures.
 * It natively supports high-resolution images (down to around 0.7mm) at high accuracy.
-* It has modules for full-brain (aseg+DKT), cerebellum and hypothalamic sub-segmentations.
+* It has modules for full-brain (aseg+DKT), cerebellum, corpus callosum and hypothalamic sub-segmentations.
 * The segmentation modules run within minutes and provide partial-volume corrected stats.
 * It has an optimized surface stream for cortical thickness analysis and improved correspondence.
 

doc/overview/modules/LIT.md

@@ -0,0 +1,56 @@
+# Lesion Inpainting Tool (LIT)
+
+With the **Lesion Inpainting Tool (LIT)** extension, FastSurfer is able to process T1-weighted
+images with lesions, such as tumors, cavities, or abnormalities. Since the deep learning-based
+tool LIT is designed to paint healthy-looking tissue into those lesions, downstream analysis with
+FastSurfer can produce more reliable whole-brain segmentation and cortical surface reconstruction
+in cases with significant structural alterations.
+
+> **Note:** The FastSurfer LIT extension is currently experimental. Review the LIT-modified
+> outputs before using them for downstream analyses.
+
+## FastSurfer Usage
+
+FastSurfer runs LIT when a lesion mask is passed with `--lesion_mask <path to file>`:
+
+```bash
+./run_fastsurfer.sh --t1 /path/to/T1.nii.gz \
+                    --lesion_mask /path/to/lesion_mask.nii.gz \
+                    --sid subject_id --sd /path/to/output_dir \
+                    --fs_license /path/to/license.txt
+```
+
+Lesion inpainting is not compatible with separate processing of the segmentation and surfaces with `--seg_only` and `--surf_only`. If you want to run the surface pipeline, avoid `--seg_only`!
+
+With `--lesion_mask <path to file>`, FastSurfer:
+
+1. inpaints the lesion area in the input T1w image,
+2. runs the requested FastSurfer segmentation and surface pipeline on the inpainted image, and
+3. maps the lesion into the final FastSurfer outputs and regenerates affected statistics.
+
+## Lesion Mask Requirements
+
+The lesion annotation must match the MRI volume passed as `--t1`: it should use the same voxel
+grid and vox2ras/affine, and it should be an integer or binary-compatible numeric image. Non-zero
+voxels are treated as lesion. The mask defines which voxels are inpainted and later marked in the
+FastSurfer outputs.
+
+Underestimation of lesion areas affects the segmentation more than oversegmentation of lesion masks,
+hence we recommend generous annotation of all damaged tissue and potentially dilating the mask.
+
+## Output Behavior
+
+FastSurfer with LIT updates the standard subject directory instead of writing a separate LIT output
+tree. The primary FastSurfer files are lesion-integrated, while pre-lesion versions are preserved
+as `.lit` backups or, for selected surface-derived files, as mapped backup files.
+
+`lesion_impact_summary.yaml` is currently emitted as YAML by neurolit. The accompanying text
+reports provide a human-readable summary of the affected anatomical structures.
+The complete list of LIT-related output files is documented in the
+[FastSurfer output files overview](../OUTPUT_FILES.md#lesion-inpainting-tool-lit-optional).
+
+## References
+
+If you use LIT in your research, please cite:
+
+*Pollak C, Kuegler D, Bauer T, Rueber T, Reuter M, FastSurfer-LIT: Lesion Inpainting Tool for Whole Brain MRI Segmentation with Tumors, Cavities and Abnormalities, Imaging Neuroscience 2025. https://doi.org/10.1162/imag_a_00446*

doc/overview/modules/index.rst

@@ -7,3 +7,4 @@ FastSurfer includes several specialized deep learning modules that can be run in
     :maxdepth: 2
 
     CC
+    LIT

doc/scripts/RUN_FASTSURFER.md

@@ -39,6 +39,7 @@ Optional arguments
 * `--no_cereb`: Switch off the cerebellum sub-segmentation.
 * `--no_hypothal`: Skip the hypothalamus segmentation.
 * `--no_cc`: Skip the segmentation and analysis of the corpus callosum.
+* `--lesion_mask <path to file>`: Path to a binary lesion mask in the same space as the T1 input. If provided, FastSurfer will wrap the segmentation and surface pipelines with lesion inpainting using LIT. This experimental feature is useful for images with tumors or other large lesions; review LIT-modified outputs before downstream use.
 * `--cereb_segfile`: Name of the cerebellum segmentation file. If not provided, this intermediate DL-based segmentation will not be stored, but only the merged segmentation will be stored (see --main_segfile <filename>). Requires an ABSOLUTE Path! Default location: \$SUBJECTS_DIR/\$sid/mri/cerebellum.CerebNet.nii.gz
 * `--no_biasfield`: Deactivate the biasfield correction and calculation of partial volume-corrected statistics in the segmentation modules.
 * `--native_image` or `--keepgeom`: **Only supported for `--seg_only`**, segment in native image space (keep orientation, image size and voxel size of the input image), this also includes experimental support for anisotropic images (no extreme anisotropy).

pyproject.toml

@@ -56,12 +56,23 @@ dependencies = [
     'torchvision>=0.22.1,<0.23',
     'tqdm>=4.65',
     'yacs>=0.1.8',
+    'monai>=1.4.0',
+    'meshpy>=2025.1.1',
+    'pyrr>=0.10.3',
+    'pip>=25.0',
 ]
 
 [project.optional-dependencies]
 qc  = [
     'whippersnappy>=2.1',
 ]
+lit = [
+    'neurolit>=0.6.1',
+]
+container = [
+    'fastsurfer[qc]',
+    'fastsurfer[lit]',
+]
 doc = [
     'fastsurfer[qc]',
     'furo!=2023.8.17',
@@ -97,6 +108,7 @@ all = [
     'fastsurfer[doc]',
     'fastsurfer[style]',
     'fastsurfer[qc]',
+    'fastsurfer[lit]',
     'fastsurfer[quicktest]',
 ]
 full = [