Inconsistent eTIV estimates on HCP data (FastSurfer vs FreeSurfer) despite similar segmentation

[aritban1github]

Hi,

I am running FastSurfer (latest version v2.4.2) on Human Connectome Project (HCP, both HCP Aging and HCP Young adults) T1-weighted data and observing a consistent discrepancy in eTIV estimates compared to FreeSurfer (latest stable release).

The segmentation outputs themselves (cortical and subcortical structures) look reasonable on visual inspection, and global measures such as MaskVol are also within expected ranges. In addition, WM/GM volumes from FastSurfer show strong agreement with both FreeSurfer and SynthSeg, suggesting that the segmentation step is largely behaving as expected.

However, the eTIV values reported by FastSurfer appear substantially inflated for some subjects (often >2,000,000), whereas FreeSurfer eTIV for the same subjects is within a biologically plausible range.

This discrepancy seems specific to eTIV:

• MaskVol agrees reasonably well between FastSurfer and FreeSurfer
• WM/GM volumes are consistent across methods
• but eTIV differs substantially, and is sometimes inconsistent relative to MaskVol

Since eTIV is derived from Talairach (affine) registration rather than directly from segmentation, I am wondering whether this could be related to instability or differences in the Talairach registration step, particularly for HCP data (e.g., high resolution, intensity characteristics, or acquisition differences).

Importantly, I have not observed this issue in ADNI subjects or other datasets I have tested, the discrepancy appears specific to HCP data.

Questions:

• Has this behavior been observed before with FastSurfer, especially on HCP datasets?
• Are there known differences in how Talairach registration is handled compared to FreeSurfer?
• Given this behavior, is there a recommended way to validate or cross-check FastSurfer eTIV, especially for HCP-style data?

Happy to share specific subjects, logs, or commands if that would help.

Here is a figure to explain better:

Scatterplot showing the relationship between FastSurfer and FreeSurfer eTIV estimates in 10 HCP subjects (5 HCP Aging and 5 HCP YA). Although moderately correlated (r = 0.704), substantial discrepancies are present, with FastSurfer eTIV generally higher than FreeSurfer (mean bias = −818,796 mm³). This may indicate differences in intracranial volume estimation, potentially related to the Talairach registration step.

Please let me know if you have any questions. Thanks!

[dkuegler]

Hello,

we have done testing and this difference is surprising. Can you share which commands you ran? We found that differences were typically down to using different atlases, i.e. recon-surf ... -3T but not run_fastsurfer.sh ... --3T (eTIV this the main metric effected by the --3T flag).

Generally, the differences that you are reporting are way larger than expected, so I am not sure what is goin on withour more details...

You can privately drop files here: https://nextcloud.dzne.de/index.php/s/Z2qtHW8c7p3NSJ5

[aritban1github]

Hi @dkuegler , thanks for your response.

Regarding your suggestion, I did run FastSurfer with the --3T flag enabled, so both FastSurfer and FreeSurfer were configured for 3T data. Because of this, I don’t think the discrepancy is due to an atlas mismatch.

Here are the exact FastSurfer commands I used (via Singularity on our HPC cluster):

singularity exec --no-mount home,cwd -e
-B <input_dir>:<input_dir>
-B <output_dir>:<output_dir>
-B <license_dir>:<license_dir>
fastsurfer-gpu-latest.sif
/fastsurfer/run_fastsurfer.sh
--t1 <input_T1.nii.gz>
--sid <subject_id>
--sd <output_dir>
--seg_only
--tal_reg
--3T
--fs_license <license.txt>
--threads 4

This was executed using an array job (one subject per task).

One quick question regarding data sharing: the subjects I am using are from the HCP Aging and HCP YA, and I am not entirely sure what the policy is for uploading raw Nifti files externally and if there are any restrictions, so I wanted to check before sharing anything directly on the link you provided. That said, I am happy to provide the subject IDs if that is okay, along with processing logs and derived outputs (e.g., aseg.stats, talairach.xfm, etc.), if that would be sufficient for reproducing the issue.

Please let me know what would be most helpful, and I can share accordingly. Thanks again for your help!

[anagrammarian]

What is strange is ADNI FTD and OASIS which have subjects with severe dementia and enlarged ventricles seem to work fine and it is failing in HCP YA and Aging and inconsistently. We also noticed something interesting- the mask seems to be robust. Is this mask from the segmented brain regions (some sort of a global collation)?

[m-reuter]

Thanks for bringing this up. It is highly relevant for our next training round where we will add lots of young adult cases.
Can you send us a list of the cases where you see the problem from those datasets. We should be able to replicate that here.

About the mask, it is derived from the segmentation by combining labels plus some closure operation with locally different strengths (to avoid inclusion of optical nerve etc). The mask is not used for eTIV estimation (as it is a brain mask not an ICV mask). We are currently working on a segmentation based ICV estimate which looks very promising.

[anagrammarian]

Arit can post the IDs but they are already on the graphic he posted (114318 etc for YA and HCA bla bla). We had a large number of failures in HCP and we took 10 out of those and ran Freesurfer just to check.

What is surprising is it is not young alone or old alone (as one might expect) but centered on HCP. ADNI/FTD/OASIS all worked perfectly.

POV, we are finding amazing correlation (>0.97 in some cases) between mask and Synthseg ICV.

[anagrammarian]

Your segmentation based ICV seems exciting. I think registration based ICV is not a good idea esp for dementia patients etc. Some would argue you need both T1 and T2 to get accurate ICVs on these cases.

[m-reuter]

POV, we are finding amazing correlation (>0.97 in some cases) between mask and Synthseg ICV.

Certainly one can expect a correlation between brain mask and eTIV (ICV). They are not the same (of course) and a too high correlation is problematic. It can indicate that the eTIV shrinks with atrophy and that should not be the case. Many of the segmentation based approaches are affected by that and we try to avoid it, which is why yet another tool is needed.

[aritban1github]

114318 | HCP YA
134021 | HCP YA
205725 | HCP YA
381543 | HCP YA
645551 | HCP YA
HCA6752784_V1_MR | HCP Aging
HCA7452575_V1_MR | HCP Aging
HCA8751792_V1_MR | HCP Aging
HCA9086990_V1_MR | HCP Aging
HCA9556193_V1_MR | HCP Aging

These are example subject IDs from different HCP cohorts (HCP Young Adult and HCP Aging) where we observed the discrepancy. We selected 10 representative cases across sites to illustrate the issue, however, this behavior is not limited to these subjects. We have identified more than 10 such cases showing similar discrepancies in HCP (but the 10 cases are also labeled in the figure provided in our first post above).

Here is another figure:

You will see here that across ADNI, FTD, and OASIS, Fastsurfer eTIV shows strong agreement with synthseg ICV, with points closely following the expected linear trend. However, in HCP, quite a few of the subjects deviates markedly, with eTIV substantially inflated relative to synthseg. The 10 subjects that we mention at the top are representative examples selected from a larger set of such cases observed in the full cohort.

[dkuegler]

Can you confirm, that you are comparing FastSurfer with FreeSurfer 8 here. Or rather, which version of FreeSurfer are you using... this may have been a confusion before.

[aritban1github]

To clarify, I used FreeSurfer v7.4.1 for these comparisons, not FreeSurfer 8. Therefore, the discrepancy I reported is between FastSurfer v2.4.2 and FreeSurfer v7.4.1 on the HCP cases.

[anagrammarian]

We saw the original problem when comparing against Synthseg and one thing we noticed was the mask correlated superbly across the board while eTIV had these failures in HCP. We then ran Freesurfer on 10 "failure" cases just to check and they all seem to work. The fact that mask worked and eTIV failed indicates failure of talaraich segmentation based ICV estimation in those cases from Fastsurfer. Strangely it works in Freesurfer even though the docu says it is the same (except for some NU bias correction IIRC?)

[m-reuter]

Just a small update:

1. we were able to replicate this (slightly different results, but the general issue)
2. we found out that also FreeSurfer's avi Talairach registration tool fails regularly, but then FreeSurfer executes a fallback path with a different tool. We want to avoid that, as different tools will have different cost functions and consistency different registrations (-> consistently different eTIVs which is an issue if the failures happen more often on severely atrophied cases with big ventricles, which they seem to do). This can bias age or disease comparisons, as eTIV is used to correct for head size.
3. we implemented a fix by pre-registering images to Talairach space using the already existing segmentation in FastSurfer. This can be done in seconds and provides a really good starting point for Talairach Avi registration tools.

Since we ultimately want to completely avoid these old FreeSurfer Talairach registration tools and switch to our dedicated ICV segmentation module, we will not replace the Talairach registration method at this point (which is overdue). Replacing it would mean to re-adjust the scaling coefficient to the new tool and is too much work for not much gain.

So for now, we test our fix on a larger data set and will put it into the next release. It should improve not only the failure cases, but actually all cases. It should help a lot in situations (like some of your cases) where the head position is far away from the Talariach atlas (e.g. head tilted).

[anagrammarian]

This is great. Yes mixing and matching is not a good idea absolutely and anyway the FS "fix" sounds like a bandaid.

In general, segmentation based ICV methods seem to be more reliable than registration based. When and how can we access your dedicated ICV module?

Also what was surprising was we got perfect results on AD and FTD databases but failues in both HCP YA and HCP Aging. Do you know why?

Thanks

[anagrammarian]

Ah I see you mention head position is far away- we'll check our failure cases to see if this pattern is observable !