Training Vision Transformers to Predict Amyloid Positivity from Brain Structural MRI

205 

Abstract Submission 

Tamoghna Chattopadhyay¹, Saket Ozarkar¹, Ketaki Buwa¹, Sophia Thomopoulos², Paul Thompson³

¹University of Southern California, Los Angeles, CA, ²USC, Marina del Rey, CA, ³USC, Marina Del Rey, CA

Tamoghna Chattopadhyay  
University of Southern California
Los Angeles, CA

Saket Ozarkar  
University of Southern California
Los Angeles, CA

Ketaki Buwa  
University of Southern California
Los Angeles, CA

Sophia Thomopoulos  
USC
Marina del Rey, CA

Paul Thompson  
USC
Marina Del Rey, CA

According to the World Health Organization (WHO), around 55 million individuals worldwide suffer from dementia - of whom 60-70% are diagnosed with Alzheimer's disease (AD). The defining features of AD include abnormal buildup of beta-amyloid (Aβ) plaques and tau protein tangles in the brain [2]. Amyloid positivity (Aβ+) is commonly assessed through positron emission tomography (PET) or sampling of cerebrospinal fluid (CSF) via lumbar puncture, but these procedures are costly and invasive. Although Aβ accumulation slightly precedes atrophy [3] on MRI, there is interest in how well standard anatomical MRI may detect Aβ-related brain changes, which include atrophy and structural alterations. Deep learning methods, like Vision Transformers (ViTs), can capture long-range spatial dependencies in images using self-attention mechanisms, and show great promise in computer vision. We examined the ViT architecture's performance in predicting Aβ+ status from T1-weighted scans and compared it to widely-used 3D DenseNet convolutional neural network (CNN) architecture [6].

We analyzed 3D T1-weighted (T1w) brain MRI data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset (phases 2/GO and 3) from 1,841 participants (age: 74.04 ± 7.40 years; 860 F/981 M) with distribution of 889 CN (cognitively normal controls)/658 with mild cognitive impairment (MCI), and 294 AD; 946 Aβ+/ 895 Aβ-. All 3D T1w brain MRI volumes were pre-processed using the following steps: nonparametric intensity normalization (N4 bias field correction), 'skull stripping' for brain extraction, 6 degrees of freedom registration to a template and isometric voxel resampling to 2 mm. These images of size 91x109x91 were scaled to take values between 0 and 1 using min-max scaling, and registered to a template created using T1w MRI from the UK Biobank dataset in MNI space [7,8]. They were resized to dimensions of 64x64x64 and 128x128x128 to guarantee direct correspondence with patch sizes used in ViT models and divided into train, validation and test sets in the ratio 80:10:10. The DenseNet architecture includes four dense blocks and three transition layers, and was used as a baseline comparison architecture. In ViT architectures [9], the input image is divided into fixed-sized patch embeddings, which are amalgamated with learnable position embeddings and class tokens. The resulting sequence of vectors is fed into a transformer encoder, which comprises alternating layers of multi-head attention and a multi-layer perceptron (MLP). We used two different ViTs - neuroimage transformer (NiT) and multiple instance NiT (MINiT) [9] - where a learned block embedding was introduced to maintain positional information of the block within the scan containing each patch. Hyperparameters were tuned and model performance was assessed using test accuracy.

Results are shown in Table 1. Best performance was achieved by MINiT architecture for image size of 64x64x64; giving test accuracy of 0.791 and test ROC-AUC of 0.857. Thus, MINiT architecture improved upon both 3D DenseNet and NiT architectures. Hyperparameter tuning of attention heads, learning rate, the encoder layer and weight decay helped to improve model performance. In our experiments, performance for 64x64x64 downscaled images was better than that for 128x128x128 upsampled images.

We evaluated prediction capabilities of the Vision Transformer architecture for inferring Aβ+ from T1w brain MRI, benchmarked against the DenseNet121 architecture. In initial experiments, MINiT architecture performed better than the other two architectures considered. Our results are promising, in that less invasive scans may be beneficial for screening individuals, prior to more intrusive Aβ+ detection procedures. This study also has some limitations, including the limited testing, for now, on the ADNI dataset. Performance may improve by increasing the size and diversity of the training data, by including multimodal brain MRI, and additional cohorts.

Neurodegenerative/ Late Life (eg. Parkinson’s, Alzheimer’s) ¹

Classification and Predictive Modeling ²

Cognition

Computational Neuroscience

Machine Learning

MRI

STRUCTURAL MRI