Adaptive Denoising of 3D Structural MR Images using Deep Learning
Poster No:
1949
Submission Type:
Abstract Submission
Authors:
Lintao Zhang1, Mingxia Liu1
Institutions:
1THE UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL, Chapel Hill, NC
First Author:
Co-Author:
Introduction:
Image denoising is a primary MRI processing step. Many studies exist on MRI denoising, but most rely on 2D methods, ignoring 3D anatomical structures in MRIs. Additionally, multi-site MRIs tend to be affected by inter-site heterogeneity caused by differences in scanners or scanning protocols, resulting in different noise levels. However, previous studies cannot adequately reduce noise across MRIs with varying degrees of noise. To this end, we design a novel adaptive 3D image denoising (A3ID) framework to adaptively remove noise from MRIs. In A3ID, we train a deep learning model to execute adaptive denoising using estimated MRI noise variance. To our knowledge, this is among the first attempts to estimate MRI noise variance and utilize this prior for 3D MRI denoising.
Methods:
A novel adaptive 3D image denoising (A3ID) framework (Fig. 1) is designed to adaptively remove noise from MRIs, including 1) noise estimation and 2) noise removal.
1) Noise Estimation. A novel gradient-based 3D MRI noise estimation strategy is proposed, where the gradient map variance of MRI in sagittal, axial, and coronal views is used to estimate the noise level. For MRIs without noise, their gradient maps only contain a small amount of texture information, such as soft tissue boundaries, and its voxel-level data distribution presents zero mean and minimal variance. The noise distribution is relatively independent, and its variance is not reduced by gradient calculation, so the MRI gradient variance can be used to estimate the noise level.
2) Noise Removal. Based on the estimated noise variance, we design a conditional UNet (ConUNet) for image denoising, incorporated by conditional instance normalization [1]. The ConUNet consists of an encoder with 5 convolutional layers, a decoder with 5 deconvolutional layers, and a conditional instance normalization layer for noise estimation. It inputs a perturbed MRI (i.e., 3D MRI+Gaussian noise) and outputs the estimated noise that can be subtracted from the input for denoising. With conditional instance normalization, the ConUNet can adapt to input MRIs with different noise levels. We use the ADAMA optimizer [2] for model training.
1) Noise Estimation. A novel gradient-based 3D MRI noise estimation strategy is proposed, where the gradient map variance of MRI in sagittal, axial, and coronal views is used to estimate the noise level. For MRIs without noise, their gradient maps only contain a small amount of texture information, such as soft tissue boundaries, and its voxel-level data distribution presents zero mean and minimal variance. The noise distribution is relatively independent, and its variance is not reduced by gradient calculation, so the MRI gradient variance can be used to estimate the noise level.
2) Noise Removal. Based on the estimated noise variance, we design a conditional UNet (ConUNet) for image denoising, incorporated by conditional instance normalization [1]. The ConUNet consists of an encoder with 5 convolutional layers, a decoder with 5 deconvolutional layers, and a conditional instance normalization layer for noise estimation. It inputs a perturbed MRI (i.e., 3D MRI+Gaussian noise) and outputs the estimated noise that can be subtracted from the input for denoising. With conditional instance normalization, the ConUNet can adapt to input MRIs with different noise levels. We use the ADAMA optimizer [2] for model training.
Results:
The MR-ART dataset [3] with 148 T1-weighted MRIs is used for evaluation. 70% of the data are used for training and the remaining for test. During training, Gaussian noise with variance randomly distributed in the range of 0~0.2 is added to MRI (image intensity range of 0~1). We add fixed variances of 0.10 and 0.15 noise during test for comparison.
1) At a noise level of 0.10 variance, PSNR, RMSE, and SSIM of our method after denoising on the test set reach 26.50 dB, 0.0481, and 0.8952, which are improved by 3.59dB, 0.0238, and 0.0932, respectively, compared with those before denoising (PSNR=22.91, RMSE=0.0719, SSIM=0.8022). We also compare our method with several state-of-the-arts: FONDUE [4] (PSNR=24.29dB, RMSE=0.0615, SSIM=0.8529), BM4D [5] (PSNR=26.28dB, RMSE=0.0495, SSIM=0.8961), UNet3D [6] (PSNR=26.01dB, RMSE=0.0510, SSIM=0.8896), nnUNet [7] (PSNR=25.96dB, RMSE=0.0511, SSIM=0.8830).
2) At a noise level of 0.15 variance, our method is superior to other methods: A3ID (PSNR=25.50dB, RMSE=0.0537, SSIM=0.8798), FONDUE (PSNR=22.93dB, RMSE=0.0717, SSIM=0.8064), BM4D (PSNR=25.15dB, RMSE=0.0559, SSIM=0.8702), UNet3D (PSNR=23.66dB, RMSE=0.0661, SSIM=0.0091), nnUNet (PSNR=23.15dB, RMSE=0.0700, SSIM=0.8097). Results suggest the effectiveness of our method in 3D MRI denoising with different noise levels.
1) At a noise level of 0.10 variance, PSNR, RMSE, and SSIM of our method after denoising on the test set reach 26.50 dB, 0.0481, and 0.8952, which are improved by 3.59dB, 0.0238, and 0.0932, respectively, compared with those before denoising (PSNR=22.91, RMSE=0.0719, SSIM=0.8022). We also compare our method with several state-of-the-arts: FONDUE [4] (PSNR=24.29dB, RMSE=0.0615, SSIM=0.8529), BM4D [5] (PSNR=26.28dB, RMSE=0.0495, SSIM=0.8961), UNet3D [6] (PSNR=26.01dB, RMSE=0.0510, SSIM=0.8896), nnUNet [7] (PSNR=25.96dB, RMSE=0.0511, SSIM=0.8830).
2) At a noise level of 0.15 variance, our method is superior to other methods: A3ID (PSNR=25.50dB, RMSE=0.0537, SSIM=0.8798), FONDUE (PSNR=22.93dB, RMSE=0.0717, SSIM=0.8064), BM4D (PSNR=25.15dB, RMSE=0.0559, SSIM=0.8702), UNet3D (PSNR=23.66dB, RMSE=0.0661, SSIM=0.0091), nnUNet (PSNR=23.15dB, RMSE=0.0700, SSIM=0.8097). Results suggest the effectiveness of our method in 3D MRI denoising with different noise levels.
Conclusions:
This work is one of the first attempts to explore adaptive denoising of 3D MRIs. The introduction of conditional instance normalization is crucial to improve the noise estimation ability of our method. Experiment results show that our model outperforms state-of-the-art methods, and this advantage is more apparent when the noise is severe.
Modeling and Analysis Methods:
Methods Development 2
Motion Correction and Preprocessing 1
Keywords:
STRUCTURAL MRI
Other - MRI Denoising; Conditional convolutional network
1|2Indicates the priority used for review
Provide references using author date format
[2] Kingma, D. P. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
[3] Nárai, Á. (2022). Movement-related artifacts (MR-ART) dataset of matched motion-corrupted and clean structural MRI brain scans. Scientific Data, 9(1), 630.
[4] Adame-Gonzalez. (2023). FONDUE: Robust resolution-invariant denoising of MR Images using Nested UNets. bioRxiv, 2023-06.
[5] Mäkinen, Y. (2020). Collaborative filtering of correlated noise: Exact transform-domain variance for improved shrinkage and patch matching. IEEE Transactions on Image Processing, 29, 8339-8354.
[6] Falk, T. (2019). U-Net: deep learning for cell counting, detection, and morphometry. Nature Methods, 16(1), 67-70.
[7] Isensee, F. (2019). No new-net. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Part II 4, 234–244.