Evaluating Deep Learning Hippocampal Segmentation Pipelines for Alzheimer’s Disease

163 

Abstract Submission 

Jiongqi Qu^1,2, Sophie Martin^1,2, James Cole^1,2

¹Centre for Medical Image Computing, Department of Computer Science, University College London, London, United Kingdom, ²Dementia Research Centre, Queen Square Institute of Neurology, University College London, London, United Kingdom

Jiongqi Qu, MRes  
Centre for Medical Image Computing, Department of Computer Science, University College London|Dementia Research Centre, Queen Square Institute of Neurology, University College London
London, United Kingdom|London, United Kingdom

Sophie Martin, MRes  
Centre for Medical Image Computing, Department of Computer Science, University College London|Dementia Research Centre, Queen Square Institute of Neurology, University College London
London, United Kingdom|London, United Kingdom

James Cole, PhD  
Centre for Medical Image Computing, Department of Computer Science, University College London|Dementia Research Centre, Queen Square Institute of Neurology, University College London
London, United Kingdom|London, United Kingdom

Deep learning has recently shown considerable promise at hippocampal segmentation from structural MRI. However, most studies focus primarily on segmentation accuracy, and overlook generalisability which is key for the clinical deployment of automated methods. Here, we aimed to evaluate 1) sensitivity to Alzheimer's disease (AD): by assessing group differences in hippocampal volume between patients with AD and cognitively normal (CN) people, and 2) reliability: by assessing intra-patient agreement of hippocampal volumes. We evaluated three "off-the-shelf" deep learning approaches (FastSurfer [Henschel et al. 2020], SynthSeg [Billot et al. 2023], nnUNet [Isensee et al. 2021)], as well as an nnUNet trained from scratch and benchmarked these models against an atlas-based method (FreeSurfer [Fischl 2012]).

To assess sensitivity, 816 scans from the Alzheimer's Disease Neuroimaging Initiative (ADNI) [Jack Jr et al. 2008] and 1276 scans from the National Alzheimer's Coordinating Center (NACC) [Beekly et al. 2007] were used, with age- and sex-matched groups. To assess reliability, we used BNU1, HNU1 and IPCAS1 from the Consortium for Reliability and Reproducibility (CoRR) [Zuo et al. 2014] database, which contains 2 repeat scans (n=48), 10 repeat scans (n=28) and 2 repeat scans (n=29) correspondingly. Among the pipelines, the original nnUNet required sub-sectioned brain data, which had unclear pre-processing steps and involved human fine-tuning. It was also trained using a mixture of healthy and non-affective psychotic disorder patients' data. To avoid under-evaluating the pipeline's performance due to these issues, an nnUNet was retrained using 366 scans from the Open Access Series of Imaging Studies (OASIS) [LaMontagne et al. 2019] data.

For sensitivity to Alzheimer's, all pipelines achieved large effect sizes (Cohen's d [Cohen 2013] > 0.5; Figure 1a). The effect sizes of the original nnUNet were substantially lower than the other pipelines. After retraining, nnUNet reached comparable performance with FreeSurfer. Generally, group-different effects were lower in NACC participants compared to ADNI, though the Cohen's d values shared a similar ordering across the two datasets. FastSurfer returned absolute volumes that were most similar to FreeSurfer (see Figure 1b for the hippocampal volume distributions in the AD and CN groups). The original nnUNet often returned lower hippocampal volumes, while SynthSeg and the retrained nnUNet tended to overestimate the volumes.

For test-retest reliability (Figure 2), the BNU1 results indicated that the four deep learning pipelines had better segmentation stability than FreeSurfer with higher median values and narrower and less overlapped confidence intervals. However, there were no significant differences between them. For the HNU1 data, both nnUNets produced unstable results for left and right hippocampal segmentation. However, their intra-class correlation coefficient (ICC(3,1)) [Weir 2005] values were still much higher than FreeSurfer (>0.94). In the IPCAS1 data, the original nnUNet was less stable and its confidence intervals overlapped with FreeSurfer by >60%.

We found that deep learning hippocampal segmentation methods can achieve comparable or better results than FreeSurfer, in terms of sensitivity to AD and test-retest reliability. These findings could help guide study design considerations or sample size calculations for research into the hippocampus in AD.

Future research could evaluate reducing similarity constraints and allow more freedom in variation during training (especially as a ground truth hippocampal segmentation does not exist), data augmentation, increasing the proportion of AD data and minimising the amount of preprocessing require.

Neurodegenerative/ Late Life (eg. Parkinson’s, Alzheimer’s) ¹

Segmentation and Parcellation ²

Aging

Computing

Data analysis

Degenerative Disease

Machine Learning

MRI

Segmentation