CuBNM: GPU-Accelerated Biophysical Network Modeling
Poster No:
1884
Submission Type:
Abstract Submission
Authors:
Amin Saberi1,2,3, Kevin Wischnewski1, Kyesam Jung1, Leonard Sasse1, Felix Hoffstaedter1, Oleksandr Popovych1, Boris Bernhardt4, Simon Eickhoff1, Sofie Valk2
Institutions:
1INM-7, Research Centre Jülich, Jülich, Germany, 2Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany, 3Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf, Düsseldorf, Germany, 4Montreal Neurological Institute and Hospital, Montreal, Quebec
First Author:
Amin Saberi
INM-7, Research Centre Jülich|Max Planck Institute for Human Cognitive and Brain Sciences|Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf
Jülich, Germany|Leipzig, Germany|Düsseldorf, Germany
INM-7, Research Centre Jülich|Max Planck Institute for Human Cognitive and Brain Sciences|Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf
Jülich, Germany|Leipzig, Germany|Düsseldorf, Germany
Co-Author(s):
Introduction:
Biophysical network modeling (BNM) of the brain is a promising technique to bridge macro- and microscale levels of investigation and enables inferences about latent features of brain activity, such as excitation-inhibition balance. Through this approach, personalized models of the brain can be fitted to the imaging data of individual subjects by parameter optimization [1, 2]. However, this process typically involves running several thousands of simulations for each subject, and therefore is computationally costly. This limits its scalability to a higher number of subjects and more complex models. Here, we present cuBNM (https://github.com/amnsbr/cuBNM), a toolbox for efficient simulation and optimization of BNMs using GPUs.
Methods:
To establish the functionality of our toolbox, we simulated the activity of 100 cortical regions of the Schaefer atlas [3] as network nodes governed by the reduced Wong-Wang model with analytical-numerical feedback inhibition control [4, 5] and the Balloon-Windkessel model for calculation of simulated BOLD signals [6]. We used two types of model parameterization, including a homogeneous model with constant local parameters across nodes (3 free parameters) and a heterogeneous model with regionally variable local parameters based on a linear combination of six biological maps (15 free parameters) [5, 7, 8]. The model fit to empirical data was assessed by comparing the simulated and empirical functional connectivity (FC) and functional connectivity dynamics (FCD) matrices. Model parameters were optimized by either a grid search or covariance matrix adaptation evolution strategy (CMAES). Parallelization within the grid or the sequential iterations of CMAES was done at the level of simulations (across the GPU 'blocks') and nodes (across each block's 'threads'). The calculation of FC and FCD were similarly parallelized across the simulations and matrix entries (Fig. 1). We performed benchmark tests by running N = {1, …, 215} parallel homogeneous simulations (duration 60s, BOLD TR 1s) on three types of GPU of Nvidia A100, GTX 1080 Ti and T4 GPUs as well as on a supercomputer multi-CPU node (128 cores) and a single core of CPU. Following, as an example use case of the toolbox, we fitted the BNM to the group-averaged data of the Human Connectome Project (HCP; n = 738 training and 317 test subjects) by using grid search and CMAES optimization of the homogeneous and heterogeneous models on A100 GPUs. To match the empirical data, the simulations were run for 900s with a BOLD TR of 0.72s.
Results:
Single simulation of the model network corresponding to one minute of real time was faster on GPUs (A100: 4.85s, GTX 1080 Ti: 9.67s, T4: 13.32s) than on CPUs (19.33s). On A100 GPU, running 32768 parallel simulations took 14m41s. The same simulations on a single-core CPU were estimated to take >7 days. We achieved maximum speedups of 180.3 with A100 GPU (32768 simulations), 151.7 with 128-core CPU node (512 simulations), 42.1 with T4 GPU (8192 simulations) and 27.7 with GTX 1080 Ti GPU (1024 simulations; Fig 2A). In our use case, we observed that the goodness-of-fit of the optimal simulations to the group-averaged HCP data was better for the heterogeneous models with 15 free parameters optimized by CMAES with a population size of 256 or 64 particles (training: 0.283, test: 0.276) compared to 3-dimensional homogeneous models fitted using CMAES with 64 particles (training: 0.146, test: 0.138) or grid search with 4096 (training: 0.140, test: 0.132) or 64,000 (training: 0.148, test: 0.134) simulations (Fig 2B).
Conclusions:
The BNM simulations and parameter optimization can be done considerably more efficiently on GPUs compared to CPUs. Our GPU implementation of BNMs enables scaling of this approach to a higher number of subjects as well as more complex and biologically realistic models which can ultimately increase model performance and validity.
Modeling and Analysis Methods:
fMRI Connectivity and Network Modeling 2
Methods Development 1
Keywords:
Computational Neuroscience
Modeling
Open-Source Software
Other - Graphical processing unit
1|2Indicates the priority used for review
Provide references using author date format
2. Ritter P, Schirner M, McIntosh AR, Jirsa VK. The Virtual Brain Integrates Computational Modeling and Multimodal Neuroimaging. Brain Connectivity. 2013;3:121–145.
3. Schaefer A, Kong R, Gordon EM, Laumann TO, Zuo X-N, Holmes AJ, et al. Local-Global Parcellation of the Human Cerebral Cortex from Intrinsic Functional Connectivity MRI. Cereb Cortex. 2018;28:3095–3114.
4. Deco G, Ponce-Alvarez A, Hagmann P, Romani GL, Mantini D, Corbetta M. How local excitation-inhibition ratio impacts the whole brain dynamics. J Neurosci. 2014;34:7886–7898.
5. Demirtaş M, Burt JB, Helmer M, Ji JL, Adkinson BD, Glasser MF, et al. Hierarchical Heterogeneity across Human Cortex Shapes Large-Scale Neural Dynamics. Neuron. 2019;101:1181-1194.e13.
6. Friston KJ, Harrison L, Penny W. Dynamic causal modelling. Neuroimage. 2003;19:1273–1302.
7. Kong X, Kong R, Orban C, Wang P, Zhang S, Anderson K, et al. Sensory-motor cortices shape functional connectivity dynamics in the human brain. Nat Commun. 2021;12:6373.
8. Deco G, Kringelbach ML, Arnatkeviciute A, Oldham S, Sabaroedin K, Rogasch NC, et al. Dynamical consequences of regional heterogeneity in the brain’s transcriptional landscape. Science Advances. 2021;7:eabf4752.