Magnetic resonance-based eye tracking can detect subject arousal state

1431 

Abstract Submission 

Rongjie Hu¹, Jie Liang¹, Yiwen Ding², Shuang Jian², Xiuwen Wu¹, Yanming Wang¹, Zhen Liang², Bensheng Qiu¹, Xiaoxiao Wang¹

¹University of Science and Technology of China, Hefei, China, ²Anhui Medical University, Hefei, China

Rongjie Hu  
University of Science and Technology of China
Hefei, China

Jie Liang  
University of Science and Technology of China
Hefei, China

Yiwen Ding  
Anhui Medical University
Hefei, China

Shuang Jian  
Anhui Medical University
Hefei, China

Xiuwen Wu  
University of Science and Technology of China
Hefei, China

Yanming Wang  
University of Science and Technology of China
Hefei, China

Zhen Liang  
Anhui Medical University
Hefei, China

Bensheng Qiu  
University of Science and Technology of China
Hefei, China

Xiaoxiao Wang  
University of Science and Technology of China
Hefei, China

Eye movements directly reflect human thoughts and play key roles in cognitive researches (Klein & Ettinger, 2019). fMRI combined with eye tracking provide a window into brain cognition and disease diagnosis (LaConte, Glielmi, Heberlein, & Hu, 2007). However, conventional eye tracking methods reliant on MRI-compatible equipment are expensive and intricate. Frey et al. designed an exciting CNN model 'DeepMReye' which performed camera-free eye tracking (Frey, Nau, & Doeller, 2021). But DeepMReye and our previous work (Wu et al., 2023) operate at interval of time of repetition (TR, in seconds), unable to fast eye movements such as blink and saccade. Therefore, we propose an eye-movement deep learning prediction pipeline, MRGazerII, which reads in fMRI slices and reports the eye movements at the interval of tens of milliseconds.

Our study employed data from the Human Connectome Project (HCP) 7T Movie release (Van Essen et al., 2013), in which subjects watched four movies with eye tracking (416 fMRI runs from 158 subjects). The eye balls were segmented by a binary morphology method, and the intermediate 6-layer slices of the eye balls were fed into the deep model. The model comprised a ResNet-CBAM backbone, a Transformer encoder and a fully connected layer for outputting slice-level eye movements (Fig 1a). Cross-individual 5-fold cross-validation was executed.
Preprocessing procedures, including artifact removal, registration, and regression of 12 motion parameters and signals in ventricles (Jo, Saad, Simmons, Milbury, & Cox, 2010), were applied to HCP 7T Rest1 dataset. Poudel, Innes, Bones, Watts, and Jones (2014) has found the arousal state was correlated with long blink, thus, the subjects with eye tracking data (N=131) were separated into the high arousal group (long blink proportion < 5%, N=73) and low arousal group (long blink proportion > 5%, N=58, Fig 2a). Schaefer et al. (2017) 300 parcellations were then used to generate functional connectivity (FC).
FCs-based behavioral prediction was also applied for predicting subjects list sorting score. In the feature selection step, Pearson correlation was performed between each edge and list sorting score across subjects in the training set, with top 3000 FCs being selected. Principal component analysis reduced the selected 3000 FCs in training set to 30 dimensions. Linear support vector machine (SVM) regression was used for prediction, with 10-fold cross validation. Beyond training and validating in all the subjects, we trained and validated the behavior prediction in high arousal group, then tested it in the low arousal group, and vice versal.

The proposed model achieves an f1-score of 51.2% in classifying eye movements with f1-scores of 63%, 45%, 28% and 43% for the fixation, blink, saccade and long blink respectively, surpassing the chance level (Fig 1c). Cross-dataset testing (training on HCP 7T Movie and Wu et al. (2023)'s dataset, testing on Zhang and Naya (2022)'s) reports an averaged accuracy of 38.0% (chance level at 26.0%). FC analysis reveals significantly reduced FCs in the high arousal group compared to the low arousal group (FDR corrected p < 0.01, Fig 2d). The FC-based behavior prediction achieves a Pearson's r=0.26 using data from all subjects. While training and validating on the high arousal group, the r increases to 0.38, but fails in the low arousal group (r<0.1). Training and validating on the low arousal group resulting in r=0.42, but fails in the high arousal group (r<0.1).

The proposed method, MRGazerII, can predict fixation, blink, long blink, saccade and fixation point in an acceptable accuracy at a fine interval of tens of milliseconds from fMRI data, both across individual and datasets. Eye movement based arousal grouping shows that the brain–phenotype model trained in one arousal state fails on another arousal state. This study underscores the indispensable role of eye tracking in understanding fMRI data and introduces MRGazerII as a solution.

Classification and Predictive Modeling ¹

Methods Development

Task-Independent and Resting-State Analysis ²

Computing

Data analysis

FUNCTIONAL MRI

Machine Learning

NORMAL HUMAN

Vision