Evaluating Accuracy and Reliability of Brain-Behavior Models Using Diffusion MRI

1618 

Abstract Submission 

McKenzie Hagen¹, John Kruper¹, Keshav Motwani¹, Eardi Lila¹, Jason Yeatman², Ariel Rokem¹

¹University of Washington, Seattle, WA, ²Stanford University, Stanford, CA

McKenzie Hagen  
University of Washington
Seattle, WA

John Kruper  
University of Washington
Seattle, WA

Keshav Motwani  
University of Washington
Seattle, WA

Eardi Lila  
University of Washington
Seattle, WA

Jason Yeatman  
Stanford University
Stanford, CA

Ariel Rokem  
University of Washington
Seattle, WA

Diffusion MRI measures tissue properties of white matter, which contains long-range connections between different brain regions. A variety of methods for processing raw dMRI data create distinct sets of features for subsequent analyses. One such method, tractometry, results in "tract profiles", which describe tissue properties along the length of major white matter tracts (Yeatman et al. 2012; Kruper et al. 2021). Another is the "local connectome", or connectivity within neighboring regions of cerebral white matter (Yeh et al. 2016). While these features originate from the same raw data, they have distinct characteristics that can be exploited by different types of predictive models. With these models, we can quantify the relation between white matter microstructure and various phenotypes and identify which aspects of the microstructure are most influential. However, in high-dimensional settings, where there are many more features than subjects, such as in neuroimaging data, high model variance across resamples can complicate model interpretation.

To address high-dimensional problems, regularized regression methods have been used, which perform model selection by removing non-predictive features from the model. In the case of the Least Absolute Shrinkage and Selection Operator (LASSO) regularization, a linear model fit with ordinary least squares (OLS) is penalized by the absolute value of the coefficient assigned to each feature in the model. This results in non-informative features being removed from the model by assigning them a coefficient of zero. An alternative model is sparse group LASSO (SGL), which can be used with feature sets that have inherent groups, such as tract profiles (Richie-Halford et al. 2021). SGL regularizes by and within each group of features, resulting in uninformative groups of features being removed, as well as individual features within groups.

Using both local connectome and bundle profiles from the Human Connectome Project as input features, we compared the accuracy and reliability of predictive models, with age and a subset of behavioral phenotypes as prediction targets. A LASSO model was fit separately for both local connectome and bundle profiles, and an SGL model was fit to bundle profiles with each bundle forming a separate group. Nested 5-fold cross validation with each family assigned to the same fold was used to fit and evaluate models, and bootstrapping by family was used to create 95% confidence intervals (CIs) to evaluate the stability of models.

Results: We found that both feature sets are slightly predictive of behavioral phenotypes when modeled with LASSO, with no significant difference between local connectome and bundle profiles (Fig 1). Interestingly, endurance, which was operationalized by how far an individual can walk in 2 minutes, is the most predictable after age. Across prediction targets, the average R_... value ranges between .001 and .17 for local connectome, and .001 and .14 for bundle profiles, showing that both diffusion feature sets predict some phenotypes better than others, replicating previous work (Rasero et al. 2021).

A comparison of SGL and LASSO bundle profile models predicting cognitive ability shows that while the predictive accuracy of either model isn't significantly different (Fig 1), the stability of model coefficients is. Across all nodes, the standard deviation of the model coefficients for 35 bootstraps for LASSO is .06 and SGL is 0.007 (Fig 2).

These results show that characteristics of diffusion feature sets can be leveraged with different predictive models, to improve inferences by increasing model stability. While local connectome and bundle profiles perform equally well predicting behavioral phenotypes, the instability of the LASSO regularized local connectome model challenges inference from the coefficients. In contrast, using SGL with the inherently grouped bundle profiles leads to more interpretable and stable models.

Executive Function, Cognitive Control and Decision Making

Classification and Predictive Modeling

Diffusion MRI Modeling and Analysis ¹

Multivariate Approaches ²

Cognition

White Matter

WHITE MATTER IMAGING - DTI, HARDI, DSI, ETC