Print  Close

Synthesizing Brain Signals to Control Motor Brain-Computer Interface Using Generative Neural Network

Poster No:

2050 

Submission Type:

Abstract Submission 

Authors:

HongJune Kim1, June Sic Kim1

Institutions:

1Konkuk University Medical Center, Seoul, .

First Author:

HongJune Kim  
Konkuk University Medical Center
Seoul, .

Co-Author:

June Sic Kim  
Konkuk University Medical Center
Seoul, .

Introduction:

The cortical representation of motor components, such as kinematics or kinetics, appears differently depending on their behavioral characteristics and is distributed widely over the cerebral cortex (Kim et al., 2023). The brain-computer interface (BCI) system should utilize the cortical representation to increase accuracy and reliability. However, constraints on neural data acquisition can limit such utilization. For example, acquisition methods such as intracranial EEG (iEEG) have a high spatial resolution but limited spatial coverage (Sejnowski et al., 2014). Thus, obtaining appropriate brain signals representing the behavior could be challenging. We speculated that artificially synthesizing the primary brain signals representing the behavior using the signals of the secondary brain area could address such limitations. Here, we show a novel method using the generative deep neural network model for synthesizing the cortical signals for controlling BCI.

Methods:

We employ the generative adversarial neural network (GAN) model based on the HiFi GAN (Kong et al., 2020), which was introduced to translate signal data. We trained the model to learn the spectrotemporal features of the primary motor cortex's signal waveforms during the hand-reaching movement for the target (Fig 1).
Supporting Image: Fig1.png
   ·Signal synthesis framework
 

Results:

When source signals of a motor-related area, such as the intraparietal sulcus (IPS), entered the model, it synthesized the signal waveforms of the motor cortex (M1) by translating the spectrotemporal characteristics of IPS into those of M1. We found that the signal features of synthesized M1 signals contain the unique characteristics of motor cortex (Fig 2). Furthermore, kinematics trajectories of hand-reaching movement could be decoded from the synthesized M1 signals.
Supporting Image: FIg2.png
   ·Signal synthesize result. (Upper) Averaged signal wavefroms and normalized spectrograms. (Middle) Cross-correlation between M1 and other signals. (Low) Decoded hand-movement trajectories
 

Conclusions:

We conjecture that such findings may help to address the limitation regarding to spatial coverage or augmenting neural datasets for BCI.

Motor Behavior:

Brain Machine Interface 1
Motor Planning and Execution

Novel Imaging Acquisition Methods:

MEG 2

Keywords:

Cortex
Machine Learning
MEG
Motor
NORMAL HUMAN

1|2Indicates the priority used for review

Provide references using author date format

Kim, H., Kim, J. S., & Chung, C. K. (2023). Identification of cerebral cortices processing acceleration, velocity, and position during directional reaching movement with deep neural network and explainable AI. Neuroimage, 266, 119783.
Kong, J., Kim, J., & Bae, J. (2020). Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis. Advances in neural information processing systems, 33, 17022-17033.
Sejnowski, T. J., Churchland, P. S., & Movshon, J. A. (2014). Putting big data to good use in neuroscience. Nature Neuroscience, 17(11), 1440-1441. https://doi.org/10.1038/nn.3839