3D MRI to PET Translation using Constrained Denoising Diffusion Probabilistic Model

2444 

Abstract Submission 

Minhui Yu¹, Mingxia Liu¹

¹University of North Carolina at Chapel Hill, Chapel Hill, NC

Minhui Yu  
University of North Carolina at Chapel Hill
Chapel Hill, NC

Mingxia Liu  
University of North Carolina at Chapel Hill
Chapel Hill, NC

Brain PET and MRI are non-invasive imaging techniques that play important roles in diagnosing and monitoring neurological conditions such as dementia and epilepsy. However, unlike MRI that is widely accessible in clinical practice, PET is usually unavailable due to its high cost and the complexities associated with radioactive tracers. To address this challenge, it is desirable to generate PET data from MRIs. Numerous prior studies employ generative adversarial networks (GANs) for MRI to PET translation but are generally limited by unstable model training, thus leading to less reliable results. To this end, we propose a novel constrained denoising diffusion probabilistic model (cDDPM) for PET synthesis based on corresponding MRI scans, which can enhance stability of model training while ensuing fidelity of generated PET images. In this framework, we propose to enhance the vanilla DDPM training process [1] by incorporating constraints on not only the noise component, but also the generated outcomes in final timestep derived from this noise. This improvement helps ensure the fidelity of synthesized PET images due to their one-to-one correspondence with input MRIs. This is among the first attempts to translate from 3D structural MRI to PET through a new constrained diffusion model at image level. Experimental results on 293 subjects suggest the superiority of our method in generating PET with high image quality, compared with state-of-the-arts.

The proposed cDDPM is a 3D generative framework based on a DDPM model that incorporates dual constraints during training process. As shown in Fig. 1, our cDDPM contains 3 downsampling blocks, 2 residual blocks, and 3 upsampling blocks, enhanced with cross-attention mechanisms and with the timestep embedded to intermediate convolutional layers. Each 3D MRI is incorporated as a condition, which is concatenated to the perturbed PET image (PET+Gaussion noise) as the input. The training process consists of (1) a forward process that incrementally adds Gaussian noise to the PET image across 1,000 timesteps, and (2) a reverse process that progressively denoises to reconstruct PET. In each timestep, a portion of noise is introduced with its proportion gradually increasing with time according to a predefined schedule, to avoid overly polluting the image at the onset. During training, a noisy image from a random intermediate timestep for each subject is used to predict the noise, to estimate the less noisy image at the preceding timestep. A noise-level constraint is used to encourage the estimated noise to be close to the real noise, and an image-level constraint is proposed to ensure the similarity between the synthetic PET (generated from the estimated noise) and its real image. The model undergoes 40 epochs of training with paired MRI and PET. In the test stage, our model is applied to input MR images along with Gaussian noise to synthesize their corresponding PET scans. Through 1,000 inference timesteps, the model removes the noise and ultimately yields the PET.

The ADNI dataset [2] is used to validate the effectiveness of our method. The dataset contains 293 paired MRI and 18-fluoro-deoxyglucose PET scans of healthy control brains, among which we use approximately 90% of data (263 subjects) for training and 10% (30 subjects) for test. The peak signal noise ratio (PSNR) and structural similarity index measure (SSIM) of synthetic PET generated by our method on the test data is 22.884±0.584 dB and 0.680±0.008, respectively, with an improvement of 50.56% and 44.98% compared with its counterpart without the image-level constraint. We also compare our method with GAN [3] that yields the PSNR of 22.226±0.424 dB and SSIM of 0.676±0.0002. These results demonstrate the strong ability of our proposed model on modality translating from MRI to PET.

The proposed cDDPM translates 3D MRI to PET by introducing an image-level constraint on DDPM that greatly improves the accuracy of the one-to-one image synthesis.

Methods Development ²

PET ¹

MRI

Positron Emission Tomography (PET)