Analyze the Characteristics of Event Related Potential of the Hand gesture for Multimodal Bio signal

2052 

Abstract Submission 

Ji-Hun Jo¹, Dere Deji², Boreom Lee²

¹Gwangju Institution of the Science and Technology(GIST), Gwangju-si, Jeollanam-do, ²Gwangju Institute of Science and Technology(GIST), Gwangju-si, Jellanam-do

Ji-Hun Jo  
Gwangju Institution of the Science and Technology(GIST)
Gwangju-si, Jeollanam-do

Dere Deji  
Gwangju Institute of Science and Technology(GIST)
Gwangju-si, Jellanam-do

Boreom Lee  
Gwangju Institute of Science and Technology(GIST)
Gwangju-si, Jellanam-do

Many studies have reported on Brain-Machine Interface (BMI) using bio signals such as Electrocardiogram (ECG), Electroencephalogram (EEG), and Electromyogram (EMG). A lot of BMI works were developed for the purpose of rehabilitation, but also the same time, it has brought improvements in the monitoring and analysis technologies [1].
In the previous study, we developed the Hand gesture classification system using multimodal Bio signal using a Field programmable gate array (FPGA) which implements a deep learning model [2]. Our system shows high classification accuracy in the case of using an EMG and combining EMG and EEG. Only in the case of using EEG shows lower accuracy than other reports.
This study aims to investigate the possibility of improving the classification accuracy when other various characteristics of EEG were applied to the hand gesture classification system in the feature extraction layers. Also, comparison results of Event-related Potential (ERP) characteristics for each hand gesture of different types of grasp can be expected to determine a modified experimental protocol or model in future studies.

This study used the data from previous works to prepare a Hand gesture classification system consisting of bio signal sensors and a processing board that contains Convolution Neural Networks (CNN) based deep Learning Model Implemented FPGA chip. The experiment consisted of a cue time and performance time (5 sec) and a rest (3 sec). The subjects were ten healthy subjects (three females and seven males, 25 ± 5.5 years) (20211005-HR-63-01-02).
The gestures (large/medium-diameter grasp, three-finger, sphere grasp, prismatic pinch grasp, power flat grasp, cut, and rest) were chosen to improve the performance on the Toronto Rehabilitation Institute Hand Function Test (TRI-HFT) and the Jebsen-Taylor Hand Function Test.
The system was connected to each EMG and EEG sensor and acquired signals in an on-device format.
Ultracortex Mark IV Headgear and cyton board (OpenBCI Inc.,) were used to acquire the EEG data with 8-channels (F3, F4, C3, CZ, C4, P3, PZ, and P4) with 250Hz sampling frequency. The signal is preprocessed by Ultra96-V2 Board which contains an Arm cortex Multiprocessor system on a chip (MPSoC) and Xilinx Zynq series FPGA. Implemented deep learning model are based on a convolutional neural network (CNN), which extracts features from data with time series data. The proposed CNN implementation includes a downstream multilayer perceptron (MLP) layer to learn nonlinearities for gesture classification.
Input sources of EEG were filtered in 5-50Hz (band-pass) with 60Hz Notch, but analyzed in ERP to see a peak and Latency components in the mu rhythm (8-12Hz) range [3]. ERP signals were measured from the C3 area for the seven different hand gestures.

ERP signals were measured from the C3 area for the seven different grasp gestures. Each signal shows an average of 10 subjects for all gestures except rest. at approximately 500 ms, Maximum peak components were observed. Statistical analyses were performed with SPSS 25 (IBM Inc.,) using repeated-measures analysis of variance (ANOVA), and one-way ANOVA using the type of grasp gesture as the variable, no significant differences in maximum peak (p = 0.139) were found. However, the latency components show significant differences (p = 0.042), and between time gap of large diameter grasp is significantly faster than medium diameter grasp (p<0.05) in Bonferroni analysis.

This study investigated the possibility of improved classification accuracy when other features of EEG were applied to the system. Most of the ERP patterns showed similar to general gesture results [4], and the statistical significance shown in latency also indicates the time points of peaks for each EEG data also need to consider not only simple features such as Root Mean Square (RMS) in the time series signal in the model. In further, these results would help to develop more effective rehabilitation orthosis.

EEG/MEG Modeling and Analysis

Brain Machine Interface ¹

EEG ²

Data analysis

Electroencephaolography (EEG)

Motor