From Images to Insights: The Role of AI in Radiology Report Generation

Vaishali; Shelly Garg

doi:10.55524/ijirem.2025.12.2.20

Abstract

Radiology reports are important for diagnosing and managing a variety of health conditions, but manually crafting them is always laborious and often tiresome for radiologists. Machine learning has made significant advances and is beginning to show promise to help automate this process in recent years by enabling the production of reports from a clinical image, such as a chest X-ray or CT scan. This paper provides a review of the state of machine learning in Radiology Report Generation. We focus on the ability to use Convolutional Neural Networks (CNNs) to understand image data as well as transformer-based models to produce text data. In addition, we will examine the newer multimodal models which produce output combining image and text for potentially more accurate and contextually aware reports. Furthermore, our review highlights important observations, particularly the growing use of Deep Learning models and large medical datasets (e.g., MIMIC-CXR, CheXpert). Although there has been advancement, challenges remain such as clinically inaccurate results, lack of explainability, and inadequate variety of data. In summary, we conclude that although machine learning has a strong potential to significantly reduce work for radiologist and add consistent reporting, substantial additional work needs to be done to bring these systems to greater trust and reliability for real-world clinical settings.

Keywords

Radiology Report Generation, Machine learning, Deep learning, Convolutional Neural Networks (CNNs), Transformers, Medical Image Analysis, Natural Language Processing (NLP).

References

X. Wang et al., "ChestX-ray8: Hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases," in Proc. CVPR, 2017. Available from: https://doi.org/10.1109/CVPR.2017.369
J. Irvin et al., "CheXpert: A large chest radiograph dataset with uncertainty labels and expert comparison," in Proc. AAAI, 2019. Available from: https://doi.org/10.1609/aaai.v33i01.3301590
Z. Chen et al., "Generating radiology reports via memory-driven transformer," in Proc. EMNLP, 2020. Available from: https://doi.org/10.48550/arXiv.2010.16056
Y. Zhang et al., "Radiology report generation from chest X-ray images: A hybrid retrieval-generation framework," in Proc. AAAI, 2022. Available from: https://doi.org/10.1016/j.media.2023.102798
A. Johnson et al., "MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports," Scientific Data, vol. 6, no. 1, pp. 1–8, 2019. Available from: https://doi.org/10.1038/s41597-019-0322-0
J. Irvin et al., "CheXpert: A large chest radiograph dataset with uncertainty labels and expert comparison," in Proc. AAAI, 2019. Available from: https://doi.org/10.1609/aaai.v33i01.3301590
P. Zhang et al., "A Study on the Integration of Clinical and Demographic Features for Breast Cancer Prediction," Artificial Intelligence in Medicine, vol. 24, no. 3, pp. 120-134, 2021. Available from: http://dx.doi.org/10.4048/jbc.2023.26.e45
X. Wang et al., "ChestX-ray8: Hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases," in Proc. CVPR, 2017. Available from: https://doi.org/10.1109/CVPR.2017.369
X. Wang et al., "TieNet: Text-image embedding network for common thorax disease classification and reporting in chest X-rays," in Proc. CVPR, 2018. Available from: https://doi.org/10.48550/arXiv.1801.04334
Z. Chen et al., "Generating radiology reports via memory-driven transformer," in Proc. EMNLP, 2020. Available from: https://doi.org/10.48550/arXiv.2010.16056
Y. Zhang et al., "Radiology report generation from chest X-ray images: A hybrid retrieval-generation framework," in Proc. AAAI, 2022. Available from: https://doi.org/10.1016/j.media.2023.102798
A. Johnson et al., "MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports," Scientific Data, vol. 6, no. 1, pp. 1–8, 2019. Available from: https://doi.org/10.1038/s41597-019-0322-0
J. Irvin et al., "CheXpert: A large chest radiograph dataset with uncertainty labels and expert comparison," in Proc. AAAI, 2019. Available from: https://doi.org/10.1609/aaai.v33i01.3301590
A. Boag et al., "Evaluation of automated radiology report generation using clinical factual correctness," in Proc. NAACL, 2022. Available from: https://pmc.ncbi.nlm.nih.gov/articles/PMC10499844/
R. Das et al., "Deep learning for medical image analysis: A survey," Comput. Med. Imaging Graph., vol. 66, pp. 27–37, 2018. Available from: https://shorturl.at/cf3Gd
S. Ravi et al., "A comprehensive review of deep learning techniques in medical image analysis," Neurocomputing, vol. 324, pp. 29–43, 2019. Available from: https://doi.org/10.1007/978-3-030-33128-3_1
P. Lakhani and G. Sundaram, "Deep learning at chest radiography: Automated classification of pulmonary tuberculosis by using convolutional neural networks," Radiology, vol. 284, no. 2, pp. 572–578, 2017. Available from: https://doi.org/10.1148/radiol.2017162326
M. Razzak, S. Imran, and G. Xu, "Big data analytics for preventive medicine," Comput. Methods Programs Biomed., vol. 141, pp. 63–71, 2017. Available from: https://doi.org/10.1007/s00521-019-04095-y
R. Liu et al., "Efficient radiological diagnosis of pneumonia with deep convolutional networks," IEEE Access, vol. 6, pp. 5408–5416, 2018. Available from: https://doi.org/10.3390/diagnostics10060417
T. S. A. Hosny et al., "Artificial Intelligence in Radiology: The Application of Deep Learning and Machine Learning to Improve Breast Cancer Detection," IEEE Reviews in Biomedical Engineering, vol. 13, pp. 147-156, 2020. Available from: https://doi.org/10.1053/j.semnuclmed.2025.01.008
He, K., Zhang, X., Ren, S., & Sun, J., "Deep Residual Learning for Image Recognition," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. Available from: https://doi.org/10.1109/CVPR.2016.90
Simonyan, K., & Zisserman, A., "Very Deep Convolutional Networks for Large-Scale Image Recognition," International Conference on Learning Representations (ICLR), 2015. Available from: https://doi.org/10.48550/arXiv.1409.1556
Rajpurkar, P., et al., "CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning," Proceedings of the 30th Conference on Neural Information Processing Systems (NeurIPS), 2017. Available from: https://stanfordmlgroup.github.io/projects/chexnet/
Brown, T., et al., "Language Models are Few-Shot Learners," Proceedings of the 34th International Conference on Neural Information Processing Systems (NeurIPS), 2020. Available from: https://doi.org/10.48550/arXiv.2005.14165
Raffel, C., et al., "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer," Journal of Machine Learning Research, vol. 21, no. 140, pp. 1-67, 2020. Available from: https://jmlr.org/papers/volume21/20-074/20-074.pdf
Li, X., et al., "Visual BERT: A Simple and Performant Baseline for Vision and Language," Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019. Available from: https://doi.org/10.48550/arXiv.1908.03557
Chen, X., et al., "UNITER: Learning Universal Image-Text Representations," Proceedings of the European Conference on Computer Vision (ECCV), 2020. Available from: https://doi.org/10.48550/arXiv.1909.11740

Cites this article as

Vaishali, S. Garg, "From Images to Insights: The Role of AI in Radiology Report Generation", International Journal of Innovative Research in Engineering & Management (IJIREM), Vol-12, Issue-2, Page No-124-129, 2025. Available from: https://doi.org/10.55524/ijirem.2025.12.2.20