Fast and robust structural MRI super-resolution using voxel-size independent CNNs
Poster No:
1917
Submission Type:
Abstract Submission
Authors:
Walter Adame-Gonzalez1, Roqaie Moqadam2, Yashar Zeighami3, Mahsa Dadar4
Institutions:
1McGill University, Montreal, Quebec, 2Cerebral Imaging Center, Douglas Mental Health University Institute, Montreal, Quebec, 3Douglas Research Centre, Montreal, Quebec, 4McGill University, Montreal, QC
First Author:
Co-Author(s):
Introduction:
In brain Magnetic Resonance Imaging (MRI) studies, increasing image resolution is an active research field since it can potentially facilitate improved brain morphometry in clinical environments as well as in automatic downstream segmentation pipelines used in research. However, acquiring higher-resolution MRIs comes with increased acquisition times, costs, and patient discomfort, making it prohibitive in large cohort studies. Artificially increasing the MRI resolution is thus beneficial, since it can potentially enhance assessment of the brain through morphometry. We propose a voxel-size independent Generative Adversarial Network (GAN) capable of performing MRI super-resolution.
Methods:
We used 1045 randomly selected T1-weighted scans from ABIDE, ADNI, HCP, IXI, LA5c, MIRIAD, and HBA datasets. For all images, we performed preprocessing including Denoising [1], brain segmentation [2], and brain mask generation. To train an MRI super-resolution network, we generated input-target image pairs. The input was generated by a process of downscaling (average pooling 70% and trilinear 30%) - upscaling (cubic BSpline 70%, trilinear 30%) a stack of 2D slices. The input to the network consisted of 9-2D axial slices that were in the high-resolution domain, and the target was a single 2D slice that corresponded to the middle slice of the input, i.e. the ground-truth image. The generator network was a nested UNet++ with upscaling of feature maps through PixelShuffle with a voxel-size normalization step in the latent space [3], [2], [4], [5]. The discriminator was a VGG-based network with the relativistic adversarial loss [6]. The generator loss LG was defined as a combination of L1 norm, learned perceptual image patch similarity (LPIPS) [7], total variation (TV), and relativistic adversarial loss [6]. The network was trained until convergence was reached using OneCycleLR and AdamW with a maximum learning rate of 1e-5 for 10 epochs and batch size of 4. Peak signal-to-noise-ratio (PSNR), structural similarity index measure (SSIM), and LPIPS metrics were used to assess the performance of the model.
Results:
To validate our method, we tested its performance on independent subsets of the described datasets (except for HBA), as well as an in-house dataset of ex vivo T1w MRIs with similar resolution as independent validation datasets. We downscaled the whole brain volumes by a factor of 2 using Freesurfer's mri_convert function with cubic interpolation. Then, we compared the upscaled images to the ground-truth native-resolution volumes. For the in vivo validations, SSIM, LPIPS, and PSNR values were improved in 4 out of 6, 3 out of 6, and 1 out of 6 datasets respectively when compared to BSpline of order 3 interpolation (see Fig. 2.).
For the ex vivo in-house dataset, we obtained comparable results with the state-of-the-art method NLMUPSAMPLE [8], when upscaled from isotropic voxel size of 1.0mm3 to 0.5mm3. The proposed method yielded 0.0172, 39.6dB, and 0.988 for LPIPS, PSNR, and SSIM metrics respectively. NLMUPSAMPLE performed better in PSNR and SSIM (40.5dB and 0.991) but worse in LPIPS (0.0263). Finally, interpolation obtained 0.0196, 40.6dB, and 0.991 for LPIPS, PSNR and SSIM. The proposed method performed better when upscaling higher-resolution images (Fig. 1). Furthermore, the post-mortem T1w MRI tests showed the generalization capabilities of the proposed model, since the training corpus did not include any post-mortem sample.
For the ex vivo in-house dataset, we obtained comparable results with the state-of-the-art method NLMUPSAMPLE [8], when upscaled from isotropic voxel size of 1.0mm3 to 0.5mm3. The proposed method yielded 0.0172, 39.6dB, and 0.988 for LPIPS, PSNR, and SSIM metrics respectively. NLMUPSAMPLE performed better in PSNR and SSIM (40.5dB and 0.991) but worse in LPIPS (0.0263). Finally, interpolation obtained 0.0196, 40.6dB, and 0.991 for LPIPS, PSNR and SSIM. The proposed method performed better when upscaling higher-resolution images (Fig. 1). Furthermore, the post-mortem T1w MRI tests showed the generalization capabilities of the proposed model, since the training corpus did not include any post-mortem sample.
·Fig. 1.: Visual comparison between ground truth (GT), low resolution (DOWNSCALED), BSpline of order 3 interpolation (CUBIC BSPLINE), and the proposed method (OURS). In-vivo and post-mortem MRIs
·Fig. 2.: Comparison amongst independent samples from 6 of the public datasets used for training the super-resolution network.
Conclusions:
In general, our method improved sharpness and high-frequency details in MRIs compared to cubic BSpline interpolation. It might over-sharpen the softest textures in the lower-resolution/lower-quality MRIs, which translates to lower PSNR values -a metric that favors softer textures-. Furthermore, since the input to the network is the interpolated version of the low resolution image, artifacts caused by the interpolation such as aliasing can get more evident as they become sharper.
Modeling and Analysis Methods:
Methods Development 1
Motion Correction and Preprocessing 2
Keywords:
Aging
Data analysis
MRI
Open-Source Code
Open-Source Software
STRUCTURAL MRI
Other - Deep Learning, Super-resolution
1|2Indicates the priority used for review
Provide references using author date format
[2] Henschel, L. et. al. (2020). FastSurfer—A fast and accurate deep learning based neuroimaging pipeline. NeuroImage, 219, 117012. https://doi.org/10.1016/j.neuroimage.2020.117012
[5] Henschel, L. et. al. (2022). FastSurferVINN: Building resolution-independence into deep learning segmentation methods—A solution for HighRes brain MRI. NeuroImage, 251, 118933. https://doi.org/10.1016/j.neuroimage.2022.118933
[1] Manjón, J. V. et. al. (2015). MRI noise estimation and denoising using non-local PCA. Medical Image Analysis, 22(1), 35–47. https://doi.org/10.1016/j.media.2015.01.004
[8] Manjón, J. V. et. al. (2010). Non-local MRI upsampling. Medical Image Analysis, 14(6), 784–792. https://doi.org/10.1016/j.media.2010.05.010
[3] Shi, W. et. al. (2016). Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1874–1883. https://doi.org/10.1109/CVPR.2016.207
[6] Umer, R. M. et. al. (2020). Deep Generative Adversarial Residual Convolutional Networks for Real-World Super-Resolution (arXiv:2005.00953). arXiv. http://arxiv.org/abs/2005.00953
[7] Zhang, R. et. al. (2018). The Unreasonable Effectiveness of Deep Features as a Perceptual Metric (arXiv:1801.03924). arXiv. http://arxiv.org/abs/1801.03924