Decoding Newly Learned Sound Categories from Neuroimaging Signals: Towards an Optimal Pipeline

1668 

Abstract Submission 

Naishi Feng¹, Gangyi Feng¹

¹The Chinese University of Hong Kong, Hong Kong SAR, China

Naishi Feng  
The Chinese University of Hong Kong
Hong Kong SAR, China

Gangyi Feng  
The Chinese University of Hong Kong
Hong Kong SAR, China

Machine learning (ML) techniques have been successfully applied to neuroimaging data for decoding a diverse set of mental information. Most of the stimuli to be decoded are well-learned and familiar, such as common object categories[1], native language[2], and speech[3]. However, it is often challenging to decode unfamiliar or newly learned knowledge from noisy neuroimaging signals. This decoding task becomes even more challenging when decoding novel auditory categories with only brief training and exposure to the exemplars. The temporal fleet and multidimensional properties of acoustic signals also pose a significant challenge for machine learning algorithms to learn complex patterns. Therefore, a new signal processing and machine learning pipeline is required to overcome these challenges and increase neural decodability, which could also provide insights into how learning shapes neural responses. Here, we used Magnetoencephalography (MEG) to record neural signals of novel artificial sounds when participants learned to categorize these sounds into categories. We applied various ML algorithms to decode the newly learned sound categories to determine an optimal algorithm and analysis pipeline.

Thirty-five healthy right-handed participants were recruited for the auditory category learning study. Participants listened to the sounds and learned to categorize them into two categories based on feedback in a 40-minute session. The MEG signals were recorded by an Elekta Neuromag machine. We selected three types of machine learning algorithms: linear discriminant analysis (LDA), support vector machine (SVM), and artificial neural networks (ANN) to test their neural decoding performances. For SVM, we used both linear and nonlinear SVM. ANN includes a three-layer fully connected neural network (FNN) and a shallow ConvNet (SC)[4]. These algorithms are chosen because of their ability to extract useful information from neural signals and effectively learn complex patterns with relatively small training samples. ANN algorithms may require more training data to reveal their decoding power. FNN has two layers that can learn and store mapping relations between neural data and targets, whereas SC has two convoluted layers capable of capturing information abstracting from the raw data. In addition to the overall decoding performances, we evaluated the decoding abilities of these algorithms based on their performance for successful (top 15) and less successful (bottom 15) learners, different training blocks, and across different brain locations and time windows of sound presentation.

Among all the algorithms tested, the linear-SVM combined with PCA dimension reduction and SC was found to be more effective in decoding newly learned categories than others when using all the available data. Neural decoding accuracies gradually increased as the training blocks progressed for all the methods, which is similar to the patterns of behavioral learning. The decoding performances of the linear classifiers (LDA and linear-SVM) can achieve better performances than others when applied to subgroups of the learners (i.e., successful and less successful learners). We further demonstrated that the best performance in decoding sound categories across the time and location of the neuroimaging signals is mainly at the time windows of 200-300 and 400-500 ms after sound presentation and located at the bilateral temporal and parietal channels.

Although decoding newly learned knowledge is a challenging task, we demonstrate that the linear-SVM with dimension reduction procedure and SC algorithms could be a promising technical solution to uncover the hidden neural signals related to the new knowledge with brief learning experience and exposure. This pipeline could potentially capture the useful and core temporal and spatial neural information for decoding and provide novel insights into the neural mechanisms of auditory learning.

Classification and Predictive Modeling ²

EEG/MEG Modeling and Analysis ¹

MEG

MEG

Other - Neural Decoding; Category Learning; Machine Learning; Short-term Training