EEG-based VR Cybersickness Detection Using Domain Adversarial Deep Learning Model
Poster No:
1420
Submission Type:
Abstract Submission
Authors:
Wooseok Hyung1, Jung-Hwan Kim1, Hyunsub Kim1, Chang-Hwan Im1
Institutions:
1Hanyang University, Seoul, Republic of Korea
First Author:
Co-Author(s):
Introduction:
Virtual reality (VR)-induced cybersickness can cause negative effects like dizziness, nausea, and vomiting. Since these effects can be a barrier to some users wearing a VR head-mounted display (HMD) headset, detecting VR-induced cybersickness is becoming an important issue in the popularization of VR services. In this study, we propose a deep learning (DL) architecture for improved EEG-based cybersickness detection in a VR environment.
Methods:
Thirty-five participants (19 males, 16 females, 20 to 35 years old) with normal or corrected-to-normal vision were recruited for this study. Each participant watched 360-degree, 15-minute VR rollercoaster video once, while wearing VR-HMD headset. The video consisted of three stages, each lasting five minutes. In the first stage, VR rollercoaster moves slowly and statically, causing little or no cybersickness. In the second and third stages, VR rollercoaster moves faster and more dynamically to cause cybersickness intentionally. EEG signal was measured from 32 electrodes with a 2,048Hz sampling frequency. For EEG signal preprocessing, 512Hz-downsampling, 4-55Hz third-order Butterworth bandpass filtering, and common average referencing were applied. The preprocessed signal was segmented by a 2-second window with 1-second sliding. First stage was categorized as 'non-cybersickness', second and third stages as 'cybersickness' for binary classification.
Simulator sickness questionnaire (SSQ)[1] was used before and after the experiment to monitor the participant's cybersickness triggered by the video. Three participants were excluded from the analyses due to low SSQ difference, and four participants were excluded due to bad EEG signal quality.
For the subject-independent cybersickness classification, we propose a model to classify cybersickness robust to subjects, and compared the performance with conventional deep learning models. In our proposed architecture, feature extractor from TSception [2] extracts temporal and spatial features, and the classifier determines cybersickness based on the features. Subject discriminator from Domain Adversarial Neural Network (DANN) [3] was trained adversarially to the subject domain by gradient reversal, making feature extractor extract subject-irrelevant features.
Simulator sickness questionnaire (SSQ)[1] was used before and after the experiment to monitor the participant's cybersickness triggered by the video. Three participants were excluded from the analyses due to low SSQ difference, and four participants were excluded due to bad EEG signal quality.
For the subject-independent cybersickness classification, we propose a model to classify cybersickness robust to subjects, and compared the performance with conventional deep learning models. In our proposed architecture, feature extractor from TSception [2] extracts temporal and spatial features, and the classifier determines cybersickness based on the features. Subject discriminator from Domain Adversarial Neural Network (DANN) [3] was trained adversarially to the subject domain by gradient reversal, making feature extractor extract subject-irrelevant features.
·Figure 1. Framework of proposed model.
Results:
To verify the performance of the subject-independent model, leave-one-subject-out cross-validation of the proposed model and an existing EEG-based classification model was conducted. Proposed model outperforms other existing EEG-based classification models in terms of accuracy and F1-score, and that the proposed model shows fewer instances of poor performance comparing to conventional models.
·Figure 2. Average accuracy & F1-score for cybersickness classification (Left), Violin plot for accuracy & F1-score (Right)
Conclusions:
In this study, we proposed a subject-independent DL model for EEG-based cybersickness classification in VR environment. Discriminator was used and trained subject adversarially in this model to train feature extractor robust to subject. The proposed model showed highest accuracy and F1-score compared to the established models.
Modeling and Analysis Methods:
Classification and Predictive Modeling 1
EEG/MEG Modeling and Analysis 2
Motor Behavior:
Brain Machine Interface
Novel Imaging Acquisition Methods:
EEG
Keywords:
Computational Neuroscience
Electroencephaolography (EEG)
Machine Learning
Modeling
1|2Indicates the priority used for review
Provide references using author date format
[2] Ding, Y., et al. (2023), "TSception: Capturing Temporal Dynamics and Spatial Asymmetry From EEG for Emotion Recognition," in IEEE Transactions on Affective Computing, vol. 14, no. 3, pp. 2238-2250
[3] Ganin, Y., et al. (2016), "Domain-adversarial training of neural networks." The journal of machine learning research 17.1, pp. 2096-2030.