EEG-based emotion recognition in VR using functional brain networks and graph theoretical measures

1642 

Abstract Submission 

Hyejeong Jo¹, Won Hee Lee¹

¹Kyung Hee University, Yongin

Hyejeong Jo  
Kyung Hee University
Yongin

Won Hee Lee  
Kyung Hee University
Yongin

Affective computing research using electroencephalography (EEG) data has traditionally used 2D displays, such as images or videos, to evoke emotional responses. However, this approach has limitations, especially when transitioning into 3D settings, as it often overlooks spatial differences in emotional responses and neural mechanisms (Suhaimi, Yuan et al. 2018; Tian, Hua et al. 2021). To address these limitations, virtual reality (VR) has emerged as a powerful tool for eliciting emotions in controlled laboratory environments, primarily due to its capability to create an immersive sense of presence in users (Alcañiz, Baños et al. 2003). Recent studies have explored the use of VR to establish highly immersive environments for emotion recognition (Suhaimi, Mountstephens et al. 2022; Yu, Xiao et al. 2022). In this work, we aim to improve the accuracy of emotion recognition using EEG data collected within VR environments. Specifically, we employ a machine learning-based approach that leverages functional brain networks and graph theoretical measures to differentiate between positive and negative emotional states.

We used the VREED dataset consisting of high-density (59-channel) EEG data collected from 19 healthy participants immersed in a VR environment (Yu, Xiao et al. 2022). We constructed functional connectivity (FC) matrices between pairs of EEG electrodes using coherence (Bowyer 2016) for each emotional stimulus (positive/negative) across six different frequency bands (delta, theta, alpha, beta, gamma, and high gamma) (Figure 1A). We applied the minimum spanning tree algorithm to transform each FC matrix into an undirected binary graph (Tewarie, van Dellen et al. 2015). We then computed graph theoretical properties of the binarized FC, including characteristic path length, global efficiency, clustering coefficient, local efficiency, betweenness centrality, node degree, and eigenvector centrality (Vecchio, Miraglia et al. 2017). We focused on the binary emotion classification of negative and positive emotional states by using FC and graph theoretical measures as features. We identified significant FC features between positive and negative states using a network-based statistic approach (Zalesky, Fornito et al. 2010). Significant graph theoretical measures were determined using ANOVA, followed by post-hoc pairwise comparisons using the Games-Howell test (Figure 1B). To assess the classification performance, we employed six commonly used machine learning algorithms (SVM, random forest, XGBoost, MLP, extra trees, kNN). We employed nested 10-fold cross-validation to evaluate the performance of our models (Figure 1C).

Figure 2 shows the confusion matrices, illustrating the binary emotion classification results for the six different machine learning algorithms. Among these algorithms, random forest achieved the lowest classification accuracy of 71.6% (recall = 67.7%, f1-score = 70.5%, precision = 73.8%, AUC = 71.6%), while support vector machine demonstrated the highest classification accuracy of 84.0% (recall = 85.1%, f1-score = 84.3%, precision = 83.6%, AUC = 84.0%).

We conducted a comprehensive evaluation of six machine learning algorithms for emotion recognition using EEG recorded during emotional experiences in a VR environment. Our results revealed that SVM achieved the highest classification accuracy of 84.0% in binary emotion recognition. Our approach outperformed the existing literature, establishing state-of-the-art performance on the VREED dataset (Uyanık, Ozcelik et al. 2022; Yu, Xiao et al. 2022). These results suggest the potential of using functional brain networks and graph theoretical measures as effective features for EEG-based emotion recognition, as they capture local brain activities responsive to emotional states and reveal interactions among different brain areas.

Connectivity (eg. functional, effective, structural) ²

EEG/MEG Modeling and Analysis ¹

Electroencephaolography (EEG)

Machine Learning

Other - graph theory;functional connectivity network;virtual reality