Adversarial Diffusion Models for Medical Image Translation
Poster No:
2412
Submission Type:
Abstract Submission
Authors:
Hanyeol Yang1, Seonggyu Kim2, Yongseon Yoo1, Jihwan Min1, Jong-Min Lee2
Institutions:
1Department of Artificial Intelligence, Hanyang University, Seoul, Korea, Republic of, 2Department of Electronic Engineering, Hanyang University, Seoul, Korea, Republic of
First Author:
Co-Author(s):
Introduction:
Multi-modal imaging provides key insights for assessing the structure and function of the human body. [1]. Augmenting insights through multi-modal imaging enhances diagnostic accuracy and boosts performance in subsequent tasks [2].
However, the practical application of multimodal protocols includes realistic challenges due to labor and economic costs [3]. Medical image synthesis emerges as a solution to this issue, wherein the missing target modality is generated based on the acquired source modality [4,5,6].
In recent years, GAN models have shown remarkable success in image synthesis. Yet, inherent limitations in the quality and diversity of synthesized images persist [7]. To address these limitations, we have adopted diffusion model a recent focal point in generative model research [7,8]. Previous studies on image-to-image translation using existing diffusion models encounter drawbacks, slow sampling, a typical drawback of diffusion model [9,10].
In this context, we introduce an adversarial diffusion model for medical image synthesis. Our approach shows that adversarial loss to enhance sampling speed.
However, the practical application of multimodal protocols includes realistic challenges due to labor and economic costs [3]. Medical image synthesis emerges as a solution to this issue, wherein the missing target modality is generated based on the acquired source modality [4,5,6].
In recent years, GAN models have shown remarkable success in image synthesis. Yet, inherent limitations in the quality and diversity of synthesized images persist [7]. To address these limitations, we have adopted diffusion model a recent focal point in generative model research [7,8]. Previous studies on image-to-image translation using existing diffusion models encounter drawbacks, slow sampling, a typical drawback of diffusion model [9,10].
In this context, we introduce an adversarial diffusion model for medical image synthesis. Our approach shows that adversarial loss to enhance sampling speed.
Methods:
We used T1-weighted and FLAIR images obtained from 128 subjects from hospitals in south Korea. Each image underwent bias field correction, co-registration, and skull stripping. In each subject, 60 axial cross-sections with brain tissue were selected. The collected dataset was divided into 42 subjects for training and 86 subjects for testing purposes. Specifically, 2520 axial slices were used as training and 5160 axial slices were used as test.
Traditional diffusion models undergo training across long time steps. Ongoing research aims to mitigate this inefficiency in both the learning and inference processes. To address this problem, an adversarial learning method is employed to minimize the time step [10]. Unlike the standard diffusion model, which predicts noise at each time step, our model is trained to consistently forecast the image at the 0th time step, regardless of the time sequence. The resultant predicted image is assessed through a discriminator to validate the successful removal of noise corresponding to each time step. For an accurate representation of the biological structure present in the T1w image throughout each time step, the T1w image is utilized as input to the G_θ. The L1 loss between the generated FLAIR and the input FLAIR is computed.
Traditional diffusion models undergo training across long time steps. Ongoing research aims to mitigate this inefficiency in both the learning and inference processes. To address this problem, an adversarial learning method is employed to minimize the time step [10]. Unlike the standard diffusion model, which predicts noise at each time step, our model is trained to consistently forecast the image at the 0th time step, regardless of the time sequence. The resultant predicted image is assessed through a discriminator to validate the successful removal of noise corresponding to each time step. For an accurate representation of the biological structure present in the T1w image throughout each time step, the T1w image is utilized as input to the G_θ. The L1 loss between the generated FLAIR and the input FLAIR is computed.
·Overview of Model and loss function
Results:
Our model achieved 19.22 PSNR and 80.45 SSIM score on the test set. Notably, it employed approximately half the number of parameters compared to previous studies.
Conclusions:
We propose a model that combines diffusion model and density constraints for medical image synthesis. The proposed model improves the sampling speed of diffusion model (1000->2).
Acknowledgements
This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No.2020-0-01373, Artificial Intelligence Graduate School Program(Hanyang University))
Acknowledgements
This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No.2020-0-01373, Artificial Intelligence Graduate School Program(Hanyang University))
Modeling and Analysis Methods:
Other Methods 2
Novel Imaging Acquisition Methods:
Multi-Modal Imaging 1
Keywords:
Machine Learning
STRUCTURAL MRI
Other - Generative model
1|2Indicates the priority used for review
Provide references using author date format
[2] Junghoon Lee, Aaron Carass, Amod Jog, Can Zhao, & Jerry L. Prince (2017). Multi-atlas-based CT synthesis from conventional MRI with patch-based refinement for MRI-based radiotherapy planning. In Medical Imaging 2017: Image Processing (pp. 101331I). SPIE.
[3] Ye, D.H., Zikic, D., Glocker, B., Criminisi, A., Konukoglu, E. (2013). Modality Propagation: Coherent Synthesis of Subject-Specific Scans with Data-Driven Regularization. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2013. MICCAI 2013. Lecture Notes in Computer Science, vol 8149. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40811-3_76
[4] Nie, D. et al. (2017). Medical Image Synthesis with Context-Aware Generative Adversarial Networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D., Duchesne, S. (eds) Medical Image Computing and Computer Assisted Intervention − MICCAI 2017. MICCAI 2017. Lecture Notes in Computer Science(), vol 10435. Springer, Cham. https://doi.org/10.1007/978-3-319-66179-7_48
[5] Huang, Y., Shao, L., & Frangi, A. (2018). Cross-Modality Image Synthesis via Weakly Coupled and Geometry Co-Regularized Joint Dictionary Learning. IEEE Transactions on Medical Imaging, 37(3), 815-827.
[6] Yu, B., Zhou, L., Wang, L., Shi, Y., Fripp, J., & Bourgeat, P. (2019). Ea-GANs: Edge-Aware Generative Adversarial Networks for Cross-Modality MR Image Synthesis. IEEE Transactions on Medical Imaging, 38(7), 1750-1762.
[7] Dhariwal, P., & Nichol, A. (2021). Diffusion Models Beat GANs on Image Synthesis. In Advances in Neural Information Processing Systems (pp. 8780–8794). Curran Associates, Inc..
[8] Ho, J., Jain, A., & Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems (pp. 6840–6851). Curran Associates, Inc..
[9] Özbey, M., Dalmaz, O., Dar, S., Bedel, H., Özturk, Ş., Güngör, A., & Çukur, T. (2023). Unsupervised Medical Image Translation With Adversarial Diffusion Models. IEEE Transactions on Medical Imaging, 42(12), 3524-3539.
[10] Zhisheng Xiao, Karsten Kreis, & Arash Vahdat (2022). Tackling the Generative Learning Trilemma with Denoising Diffusion GANs.ICLR