Language Activation from Naturalistic fMRI Time Series using Machine Learning
Poster No:
1438
Submission Type:
Abstract Submission
Authors:
Elaine Kuan1,2, Viktor Vegh1,2, John Phamnguyen1,3, Kieran O'Brien4, Amanda Hammond4, David Reutens1,2,3
Institutions:
1Centre for Advanced Imaging, The University of Queensland, QLD, Australia, 2ARC Training Centre for Innovation in Biomedical Imaging Technology, The University of Queensland, QLD, Australia, 3Royal Brisbane and Women’s Hospital, QLD, Australia, 4Siemens Healthineers, QLD, Australia
First Author:
Elaine Kuan
Centre for Advanced Imaging, The University of Queensland|ARC Training Centre for Innovation in Biomedical Imaging Technology, The University of Queensland
QLD, Australia|QLD, Australia
Centre for Advanced Imaging, The University of Queensland|ARC Training Centre for Innovation in Biomedical Imaging Technology, The University of Queensland
QLD, Australia|QLD, Australia
Co-Author(s):
Viktor Vegh, Associate Professor
Centre for Advanced Imaging, The University of Queensland|ARC Training Centre for Innovation in Biomedical Imaging Technology, The University of Queensland
QLD, Australia|QLD, Australia
Centre for Advanced Imaging, The University of Queensland|ARC Training Centre for Innovation in Biomedical Imaging Technology, The University of Queensland
QLD, Australia|QLD, Australia
John Phamnguyen, PhD Candidate
Centre for Advanced Imaging, The University of Queensland|Royal Brisbane and Women’s Hospital
QLD, Australia|QLD, Australia
Centre for Advanced Imaging, The University of Queensland|Royal Brisbane and Women’s Hospital
QLD, Australia|QLD, Australia
David Reutens, Emeritus Professor
Centre for Advanced Imaging, The University of Queensland|ARC Training Centre for Innovation in Biomedical Imaging Technology, The University of Queensland|Royal Brisbane and Women’s Hospital
QLD, Australia|QLD, Australia|QLD, Australia
Centre for Advanced Imaging, The University of Queensland|ARC Training Centre for Innovation in Biomedical Imaging Technology, The University of Queensland|Royal Brisbane and Women’s Hospital
QLD, Australia|QLD, Australia|QLD, Australia
Introduction:
Naturalistic fMRI can overcome some limitations of traditional task-based fMRI. Due to the dynamic nature of the naturalistic stimulus, task regressors are difficult to define, which pose a challenge for conventional General Linear Model (GLM) analysis. Data driven methods, namely machine learning (ML), may provide an alternative solution as they are successful in decoding fMRI data. The aim of this study is to investigate the use of Rotation Forest (RotF) (Rodriguez et al, 2006) in classifying areas of language activation from naturalistic fMRI time series data, and to identify the segments of the naturalistic fMRI time series contributing mostly to the classification achieved by RotF.
Methods:
Twenty healthy controls completed language tasks including sentence completion (SC) and watched a 15-minute in-house created video made up of segments of a quiz show with advertisements. Gradient Echo Planar Imaging (EPI) fMRI data for each paradigm were acquired using the following parameters: matrix size = 64 x 64 x 42, TR = 2s, TE = 30ms and voxel size 3 x 3 x 3 mm³. The raw EPI data were pre-processed using the SPM12 software according to the following steps: slice timing correction, realignment of volumes, co-registration and smoothing.
Voxel-wise fMRI time series of the SC task were modelled with GLM (Friston et al., 1994) to obtain binarized activation maps at p < 0.001 uncorrected. Voxel-wise naturalistic fMRI time-series (frames 0 to 440) from 14 participants were extracted and labelled based on SC activation maps. Whilst we have evaluated a range of classification methods, here we provide results for Rotation Forest (RotF). The number of samples to be used for training was experimentally evaluated, resulting in 4872 single voxel time series samples. Similarly, voxel-wise time series data from six participants formed the test set, resulting in around 60,000 samples from fMRI images of each participant. The RotF based activation maps were first reconstructed and then compared with SC activation maps of each test participant. The Area Under the Curve (AUC) and Euclidean Distance (ED) between language activation peaks were evaluated.
Sequential Forward Feature Elimination (SFFE) (Ferri et al.,1994) was performed 10 times on the trained RotF model using a 2-fold cross validation. The top five frames were labelled with ones while the remaining frames were labelled with zeros to produce a frame importance metric over the naturalistic fMRI time series. Addition of the importance metrics over repeated runs produced an overall frame importance for the naturalistic time series data.
Voxel-wise fMRI time series of the SC task were modelled with GLM (Friston et al., 1994) to obtain binarized activation maps at p < 0.001 uncorrected. Voxel-wise naturalistic fMRI time-series (frames 0 to 440) from 14 participants were extracted and labelled based on SC activation maps. Whilst we have evaluated a range of classification methods, here we provide results for Rotation Forest (RotF). The number of samples to be used for training was experimentally evaluated, resulting in 4872 single voxel time series samples. Similarly, voxel-wise time series data from six participants formed the test set, resulting in around 60,000 samples from fMRI images of each participant. The RotF based activation maps were first reconstructed and then compared with SC activation maps of each test participant. The Area Under the Curve (AUC) and Euclidean Distance (ED) between language activation peaks were evaluated.
Sequential Forward Feature Elimination (SFFE) (Ferri et al.,1994) was performed 10 times on the trained RotF model using a 2-fold cross validation. The top five frames were labelled with ones while the remaining frames were labelled with zeros to produce a frame importance metric over the naturalistic fMRI time series. Addition of the importance metrics over repeated runs produced an overall frame importance for the naturalistic time series data.
Results:
From Fig. 1, we deduce that the RotF method is successful at capturing fronto-temporal and temporal activation associated with language, but is not as successful in capturing Supplementary Motor Area (SMA) activation. Occipital activation, expected for a visual stimulus but not for language activation, can also be observed in all test participants using RotF applied to the naturalistic fMRI time series. Notably, HC20 had limited activation in general but occipital activation was still present, which is expected due to the visual nature of the naturalistic stimulus. The mean whole-brain AUC and mean ED across all test participants were 0.83 and 5.4mm, respectively.
From Fig. 2, we observe that over 10 runs of SFFE, there are spikes in the waveform that concentrate between the first two advertisement blocks (frames 202 to 367), indicating that the naturalistic fMRI time series within those time points are important for the classification performed by RotF.
From Fig. 2, we observe that over 10 runs of SFFE, there are spikes in the waveform that concentrate between the first two advertisement blocks (frames 202 to 367), indicating that the naturalistic fMRI time series within those time points are important for the classification performed by RotF.
Conclusions:
With its success in identifying language activation corresponding to SC activation, RotF can potentially be used to identify language activation from naturalistic fMRI data. The application of SFFE further enhances the use of RotF, in allowing informative time points within the naturalistic fMRI time series to be identified, which can be translated to the stimulus to facilitate stimulus designs.
Language:
Language Acquisition 2
Modeling and Analysis Methods:
Activation (eg. BOLD task-fMRI)
Classification and Predictive Modeling 1
Methods Development
Keywords:
FUNCTIONAL MRI
Language
Machine Learning
Modeling
Univariate
Other - Naturalistic fMRI
1|2Indicates the priority used for review
Provide references using author date format
Rodriguez, J. J., Kuncheva, L. I., & Alonso, C. J. (2006). Rotation forest: A new classifier ensemble method. IEEE transactions on pattern analysis and machine intelligence, vol. 28, no. 10, pp.1619-1630.
Ferri, F. J., Pudil, P., Hatef, M., & Kittler, J. (1994). Comparative study of techniques for large-scale feature selection. In Machine intelligence and pattern recognition, vol. 16, pp. 403-413.