Efficient Deep Pyramidal Multi-Residual Network for Brain MRI Registration
Poster No:
1841
Submission Type:
Abstract Submission
Authors:
yoonguu song1, Seunghyeon Han2, min Choi3, Boreom Lee4
Institutions:
1Gwangju Institute of Science and Technology, Gwangju, Korea, Republic of, 2Gwangju Institute of Science and Technology, Gwangju, Jeollanam-do, 3Gwangju Institute of Science and Technology, Gwangju, Gwangju, 4Gwangju Institute of Science and Technology(GIST), Gwangju-si, Jellanam-do
First Author:
Co-Author(s):
Introduction:
Deep Learning has recently achieved a significant advance in registering medical images. Traditional methods tackle optimization problems to establish spatial correspondence between images, yielding decent accuracy but demanding significant time to resolve new optimization challenges. In this study, we propose a novel approach for image registration using a robust neural network specifically designed for brain magnetic resonance imaging. This study introduces a deep network called the Multi Residual Laplacian Pyramid Diffeomorphic Registration Network(MR-LapDRN). It efficiently solves the image registration optimization problem by employing a coarse-to-fine approach, improving spatial feature extraction across various scales. We've progressively increased the number of filters across the following three layers while concatenating the outputs of these layers using a residual connection. We evaluated and compared MR-LapDRN with the classical U-Net and other networks using a brain MRI dataset. While we observed only marginal enhancements in brain MRI images, our model demonstrated that, despite a relatively low total parameter count, it can produce outcomes comparable to cutting-edge algorithms.
Methods:
Using the deep learning structure, our aim is to derive the flow field function that establishes a mapping from the domain of the moving image to that of the fixed image. While U-Net[1] utilized convolutional layers exclusively in its encoder- decoder architecture for segmentation, Voxelmorph[2] employed the Spatial Transformer Network[3] to produce the displacement vector field (DVF). We apply a 3-level Laplacian pyramid framework in developing our MR-LapDRN, mirroring the traditional multi-resolution approach through multi-level kernel convolution
In the initial pyramidal structure, the input data is reduced by a scale factor of 4. In the second pyramid, the input data is downscaled by a factor of 2, and the last pyramid retains the original dataset to maintain consistent image sizes. Each pyramidal structure features a single encoder-decoder utilizing a multi-residual block. Originally, the multi-residual block consisted of three sequential convolutional blocks employing different kernel sizes to capture diverse spatial features across various scales. To maintain a similar range of spatial characteristics while reducing parameter number, we opted for a 3*3*3 kernel size with dilation size 2,3, instead of 5*5*5 and 7*7*7.
Following the passage through various kernel sizes in each layer, the resulting three layers are combined through concatenation to facilitate the influence of diverse spatial features on adjacent channel layers. This concatenated layer is then added to the original input of the multi-residual block, which has passed through a 1*1 convolutional layer. Within the encoder-decoder structure, there are five residual blocks strategically positioned to amplify feature extraction. These blocks operate by enveloping the input value x with identity mapping, ensuring that information preservation, as emphasized in [4], is maintained by passing it through to the output value.
In the initial pyramidal structure, the input data is reduced by a scale factor of 4. In the second pyramid, the input data is downscaled by a factor of 2, and the last pyramid retains the original dataset to maintain consistent image sizes. Each pyramidal structure features a single encoder-decoder utilizing a multi-residual block. Originally, the multi-residual block consisted of three sequential convolutional blocks employing different kernel sizes to capture diverse spatial features across various scales. To maintain a similar range of spatial characteristics while reducing parameter number, we opted for a 3*3*3 kernel size with dilation size 2,3, instead of 5*5*5 and 7*7*7.
Following the passage through various kernel sizes in each layer, the resulting three layers are combined through concatenation to facilitate the influence of diverse spatial features on adjacent channel layers. This concatenated layer is then added to the original input of the multi-residual block, which has passed through a 1*1 convolutional layer. Within the encoder-decoder structure, there are five residual blocks strategically positioned to amplify feature extraction. These blocks operate by enveloping the input value x with identity mapping, ensuring that information preservation, as emphasized in [4], is maintained by passing it through to the output value.
·A simplified 2D depiction instead of 3D to clarify the image registration networks, MR-LapDRN. This model features a 3-level deep Laplacian pyramid and Multi Residual architecture
Results:
We trained our model through local normalized cross-correlation (NCC) coupled with a regularization term. Our approach was trained using 414 T1-weighted brain MR scans sourced from the OASIS dataset[5], employing a random split of the dataset into training, validation, and test sets with an 8:1:1 ratio. The evaluation of our method was based on the dice score, using a labeled dataset.
In the comparison presented in the table, while U-Net achieves a dice score of 0.569 and LapIRN[6] scores 0.779, our model demonstrates a marginal enhancement despite uti- lizing 90% fewer parameters.
In the comparison presented in the table, while U-Net achieves a dice score of 0.569 and LapIRN[6] scores 0.779, our model demonstrates a marginal enhancement despite uti- lizing 90% fewer parameters.
·Table 1. Registration performances on brain MRI dataset
Conclusions:
This research introduces MR-LapDRN, an inventive network that combines Laplacian pyramid networks with a multi- residual module for image registration. Our method show- cases that, despite having fewer total parameters, it achieves comparable results to cutting-edge algorithms.
Modeling and Analysis Methods:
Image Registration and Computational Anatomy 1
Segmentation and Parcellation 2
Keywords:
Computing
Machine Learning
MRI
Other - Image registration
1|2Indicates the priority used for review
Provide references using author date format
Balakrishnan, G. (2019), 'VoxelMorph: a learning framework for deformable medical image registration'. IEEE transactions on medical imaging, 38(8), 1788-1800.
Jaderberg, M. (2015). 'Spatial transformer networks'. Advances in neural information processing systems, 28.
He, K. (2016). 'Identity mappings in deep residual networks'. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14 (pp. 630-645)
Marcus, D. S.(2010). 'Open access series of imaging studies: longitudinal MRI data in nondemented and demented older adults'. Journal of cognitive neuroscience, 22(12), 2677–2684.
Daniel S. Marcus, Tracy H. Wang, Jamie Parker, John G. Csernansky, John C. Morris, and Randy L. Buckner, “Open access series of imaging studies (oasis): Cross- sectional mri data in young, middle aged, nondemented, and demented older adults,” J. Cognitive Neuroscience, vol. 19, no. 9, pp. 1498–1507, sep 2007.
Mok, T. C. (2020). 'Large deformation diffeomorphic image registration with laplacian pyramid networks'. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part III 23 (pp. 211-221).