Geometric deep learning for language prediction via pointwise tract microstructure analysis

2171 

Abstract Submission 

Yuqian Chen¹, Leo Zekelman¹, Chaoyi Zhang², Tengfei Xue², Yang Song³, Nikos Makris¹, Yogesh Rathi¹, Alexandra Golby¹, Weidong Cai², Fan Zhang⁴, Lauren O'Donnell¹

¹Harvard Medical School, Boston, MA, ²The University of Sydney, Sydney, NSW, ³University of New South Wales, Sydney, New South Wales, ⁴University of Electronic Science and Technology of China, Chengdu, Sichuan

Yuqian Chen  
Harvard Medical School
Boston, MA

Leo Zekelman  
Harvard Medical School
Boston, MA

Chaoyi Zhang  
The University of Sydney
Sydney, NSW

Tengfei Xue  
The University of Sydney
Sydney, NSW

Yang Song  
University of New South Wales
Sydney, New South Wales

Nikos Makris  
Harvard Medical School
Boston, MA

Yogesh Rathi  
Harvard Medical School
Boston, MA

Alexandra Golby  
Harvard Medical School
Boston, MA

Weidong Cai  
The University of Sydney
Sydney, NSW

Fan Zhang  
University of Electronic Science and Technology of China
Chengdu, Sichuan

Lauren O'Donnell  
Harvard Medical School
Boston, MA

The brain's white matter connections (fiber tracts) and their tissue microstructure can be quantitatively mapped using diffusion magnetic resonance imaging (dMRI) tractography [1], enabling the study of the brain's structural connectivity. To better understand how brain structure relates to function, recent research explores the prediction of individual cognitive performance based on structural neuroimaging data (such as dMRI) [2,3]. A critical challenge is how to represent white matter tracts to utilize their detailed microstructure and positional information instead of averaging or binning data along the streamline [3,4]. Another challenge is the improvement of performance in regression-based prediction and a potential direction is to utilize the intrinsic continuity in regression scores. In addition, the identification of predictive brain regions is a notable challenge drawing substantial attention [5]. Therefore, we propose a novel geometric deep learning framework, which includes a point cloud representation of tracts, a novel regression loss, and a critical region localization algorithm to predict language performance and identify predictive brain regions within tracts.

The overview of our method is shown in Figure 1. Our method was evaluated on dMRI data and two language assessment scores, the NIH Toolbox Picture Vocabulary Test (TPVT) and the Toolbox Oral Reading Recognition Test (TORRT) [6], from 809 subjects of the Human Connectome Project [7]. Whole brain tractography was generated from dMRI using a two-tensor unscented Kalman filter method [8], followed by the identification of white matter tracts. The left arcuate fasciculus (AF) and inferior longitudinal fasciculus (ILF) were selected for prediction due to their relationship to language.
Each white matter tract was represented as a point cloud (Figure 1). Each point was characterized by three spatial coordinates and two additional measurements (fractional anisotropy and number of streamlines). During each training iteration, the input of the network was formed by randomly sampling points from the cloud.
We designed a Siamese network [9] that contains two PointNet-based [10] subnetworks with shared weights. To utilize the information of continuous language scores, we propose a novel regression loss that constrains the difference between the predicted scores of the input pair to be the same as the difference between the ground truth scores.
We propose a Critical Region Localization algorithm to identify critical regions within fiber tracts for language score prediction. First, the subject-wise contributing points are identified as point sets that contribute to the max-pooled features. Then group-wise analysis is performed to localize critical regions that are consistently important for prediction across testing subjects.

The popular Pearson correlation coefficient (r) was adopted as the evaluation metric of prediction performance [2,3]. We compared our proposed method with several baseline methods that use different tract representations (mean value and AFQ [4]). For TPVT, the r values of our method, mean value and AFQ are 0.33, 0.15 and 0.16 for left AF, and 0.33, 0.25, 0.09 for left ILF. For TORRT, the r values are 0.36, 0.14 and 0.17 for left AF, and 0.33, 0.25 and 0.15 for left ILF. Therefore, our approach consistently outperformed baseline methods across both fiber tracts for both prediction tasks, as demonstrated by higher r values. Critical predictive regions, as shown in Figure 2, were distributed across the left hemisphere and all cerebral lobes for both assessments.

In this work, we propose a novel geometric deep learning framework for the prediction of language scores using white matter tracts represented as point clouds. Evaluated on a large-scale public dataset, our method showed superior prediction performance and successfully identified brain regions highly predictive of language scores.

Classification and Predictive Modeling ²

Diffusion MRI Modeling and Analysis

White Matter Anatomy, Fiber Pathways and Connectivity ¹

Machine Learning

MRI

Tractography

White Matter