Interpretable Deep Learning Models for Transparent Health Decision Support Systems
Keywords:
Healthcare AI, Interpretable Deep Learning, Clinical Decision Support, Explainable AI, Patient SafetyAbstract
The increasing adoption of deep learning in healthcare has created significant opportunities for improving medical diagnosis, patient risk prediction, and clinical decision support. However, many deep learning models remain difficult to interpret, limiting their acceptance in health systems where transparency, accountability, and patient safety are essential. This study aims to develop and evaluate interpretable deep learning models for transparent health decision support systems. The proposed approach integrates convolutional neural networks, deep neural networks, attention mechanisms, SHAP values, and Grad-CAM to explain model predictions across medical imaging and clinical tabular datasets. Public healthcare datasets, including chest X-ray images, electronic health records, and diabetes patient records, are used to represent real clinical decision-making contexts. Model performance is evaluated using accuracy, precision, recall, F1-score, AUC, sensitivity, and specificity, while interpretability is assessed through explanation clarity, feature relevance, decision traceability, and clinical usefulness. The findings indicate that interpretable deep learning models can provide reliable predictive performance while offering clearer explanations for medical decisions. Visual heatmaps and feature attribution outputs help identify relevant disease indicators, patient risk factors, and clinical variables that influence model predictions. This study contributes to the development of trustworthy AI in healthcare by supporting transparent, ethical, and human-centered clinical decision-making. The research is also aligned with SDG 3 by promoting better health outcomes, patient safety, and responsible digital health innovation.
Downloads
References
[1] S. A. Alowais, S. S. Alghamdi, N. Alsuhebany, T. Alqahtani, A. I. Alshaya, S. N. Almohareb, A. Al-dairem, M. Alrashed, K. Bin Saleh, H. A. Badreldin et al., “Revolutionizing healthcare: the role of artificial intelligence in clinical practice,” BMC medical education, vol. 23, no. 1, p. 689, 2023.
[2] S. Maleki Varnosfaderani and M. Forouzanfar, “The role of ai in hospitals and clinics: transforming healthcare in the 21st century,” Bioengineering, vol. 11, no. 4, p. 337, 2024.
[3] P. Goktas and A. Grzybowski, “Shaping the future of healthcare: ethical clinical challenges and pathways to trustworthy ai,” Journal of Clinical Medicine, vol. 14, no. 5, p. 1605, 2025.
[4] E. S¸ AHiN, N. N. Arslan, and D. ¨Ozdemir, “Unlocking the black box: an in-depth review on interpretability, explainability, and reliability in deep learning,” Neural computing and applications, vol. 37, no. 2, pp. 859–965, 2025.
[5] P. Hager, F. Jungmann, R. Holland, K. Bhagat, I. Hubrecht, M. Knauer, J. Vielhauer, M. Makowski, R. Braren, G. Kaissis et al., “Evaluation and mitigation of the limitations of large language models in clinical decision-making,” Nature medicine, vol. 30, no. 9, pp. 2613–2622, 2024.
[6] J. Alderman, R. Riley, D. Parekh, C. Summers, X. Liu, and A. Denniston, “Hidden risks of predictive models in healthcare,” BMJ evidence-based medicine, 2025.
[7] O. Wysocki, J. K. Davies, M. Vigo, A. C. Armstrong, D. Landers, R. Lee, and A. Freitas, “Assessing the communication gap between ai models and healthcare professionals: Explainability, utility and trust in ai-driven clinical decision-making,” Artificial Intelligence, vol. 316, p. 103839, 2023.
[8] M. Ennab and H. Mcheick, “Enhancing interpretability and accuracy of ai models in healthcare: a comprehensive review on challenges and future directions,” Frontiers in Robotics and AI, vol. 11, p. 1444763, 2024.
[9] R. R. Kothinti, “Artificial intelligence in healthcare: Revolutionizing precision medicine, predictive analytics, and ethical considerations in autonomous diagnostics,” World Journal of Advanced Research and Reviews, vol. 19, no. 3, pp. 3395–3406, 2024.
[10] H. Xu and K. M. J. Shuttleworth, “Medical artificial intelligence and the black box problem: a view based on the ethical principle of “do no harm”,” Intelligent Medicine, vol. 4, no. 1, pp. 52–57, 2024.
[11] D. Ueda, T. Kakinuma, S. Fujita, K. Kamagata, Y. Fushimi, R. Ito, Y. Matsui, T. Nozaki, T. Nakaura, N. Fujima et al., “Fairness of artificial intelligence in healthcare: review and recommendations,” Japanese journal of radiology, vol. 42, no. 1, pp. 3–15, 2024.
[12] P. Pandey, S. Chaudhary, and X. Nie, “Deep learning technique for interpretable diagnosis of polycystic ovary syndrome in ultrasound imaging,” Aptisi Transactions on Technopreneurship (ATT), vol. 7, no. 3, pp. 779–792, 2025, https://doi.org/10.34306/att.v7i3.768.
[13] A. A. Adeniran, A. P. Onebunne, and P. William, “Explainable ai (xai) in healthcare: Enhancing trust and transparency in critical decision-making,” World Journal of Advanced Research and Reviews, vol. 23, no. 3, pp. 2647–2658, 2024.
[14] S. Guluwadi et al., “Enhancing brain tumor detection in mri images through explainable ai using grad-cam with resnet 50,” BMC medical imaging, vol. 24, no. 1, pp. 1–19, 2024.
[15] H. Luo, C. Xiang, L. Zeng, S. Li, X. Mei, L. Xiong, Y. Liu, C. Wen, Y. Cui, L. Du et al., “Shap based predictive modeling for 1 year all-cause readmission risk in elderly heart failure patients: feature selection and model interpretation,” Scientific reports, vol. 14, no. 1, p. 17728, 2024.
[16] A. S. Anita, T. Kuusk, G. Nicola, M. Hardini, and U. Rahardja, “Advancements in artificial intelligence and their contributions to sustainable development goals: A multidisciplinary review,” Sundara Advanced Research on Artificial Intelligence, vol. 2, no. 1, pp. 37–47, 2026.
[17] United Nations Department of Economic and Social Affairs, “The 17 goals,” https://sdgs.un.org/goals, 2026, accessed: 2026-03-26.
[18] ——, “Goal 13: Take Urgent Action to Combat Climate Change and Its Impacts,” https://sdgs.un.org/goals/goal13, 2025, accessed: 2026-03-26.
[19] M. Nurkanti, U. Rahardja, M. Lubis, and A. A. M. Shukri, “Deep learning-based biology learning with ethnopedagogy and local wisdom to support sustainable development goals,” Jurnal Pendidikan IPA Indonesia, vol. 15, no. 1, 2026.
[20] R. Miotto, F. Wang, S. Wang, X. Jiang, and J. T. Dudley, “Deep learning for healthcare: review, opportunities and challenges,” Briefings in bioinformatics, vol. 19, no. 6, pp. 1236–1246, 2018.
[21] T. H. Kim, R. Chinthaginjala, A. Srinivasulu, S. P. Tera, and S. O. Rab, “Covid-19 health data prediction: a critical evaluation of cnn-based approaches,” Scientific Reports, vol. 15, no. 1, p. 9121, 2025.
[22] F. Liu, H. Zhou, K. Wang, Y. Yu, Y. Gao, Z. Sun, S. Liu, S. Sun, Z. Zou, Z. Li et al., “Metagp: A generative foundation model integrating electronic health records and multimodal imaging for addressing unmet clinical needs,” Cell Reports Medicine, vol. 6, no. 4, 2025.
[23] N. Azizah, P. A. Sunarya, U. Rahardja, A. B. Mutiara, P. Prihandoko, and C. Pasha, “Improving smear-negative tuberculosis detection using data augmentation and faster r-cnn,” International Journal of Cyber and IT Service Management (IJCITSM), vol. 6, no. 1, pp. 65–77, 2026.
[24] C. C. Yang, “Explainable artificial intelligence for predictive modeling in healthcare,” Journal of health-care informatics research, vol. 6, no. 2, pp. 228–239, 2022.
[25] N. Hassan, R. Slight, G. Morgan, D. W. Bates, S. Gallier, E. Sapey, and S. Slight, “Road map for clinicians to develop and evaluate ai predictive models to inform clinical decision-making,” BMJ Health & Care Informatics, vol. 30, no. 1, p. e100784, 2023.
[26] M. Ennab and H. Mcheick, “Designing an interpretability-based model to explain the artificial intelligence algorithms in healthcare,” Diagnostics, vol. 12, no. 7, p. 1557, 2022.
[27] D. Juliastuti, Y. F. C. P. Meilani, I. N. Hikam, and J. Nathalie, “Integrating clinical intelligence and emotional responsiveness to build patient trust in healthcare delivery,” Health, Empathy, and AI Learning (HEAL), vol. 1, no. 1, pp. 17–25, 2025.
[28] H. Wang, G. Shi, X. Liu, J. Zhang, S. Chen, J. Shi, Y.-a. Li, G. Yue, and Y. Wu, “Explainable ai in medicine: A comprehensive narrative review of methods, applications, and future directions,” Expert Systems, vol. 43, no. 6, p. e70259, 2026.
[29] V. Agarwal, M. Lohani, A. S. Bist, and D. Julianingsih, “Application of voting based approach on deep learning algorithm for lung disease classification,” in 2022 International Conference on Science and Technology (ICOSTECH). IEEE, 2022, pp. 01–07.
[30] J. H. Jeong, K.-S. Lee, S.-M. Park, S. R. Kim, M.-N. Kim, S. C. Chae, S.-H. Hur, I. W. Seong, S. K. Oh, T. H. Ahn et al., “Prediction of longitudinal clinical outcomes after acute myocardial infarction using a dynamic machine learning algorithm,” Frontiers in Cardiovascular Medicine, vol. 11, p. 1340022, 2024.
[31] D. Jin, E. Sergeeva, W.-H. Weng, G. Chauhan, and P. Szolovits, “Explainable deep learning in healthcare: A methodological survey from an attribution view,” WIREs Mechanisms of Disease, vol. 14, no. 3, p. e1548, 2022.
[32] K. Karnawati, D. N. Ramadhan, T. L. Anita, R. Nurmala, and L. Maria, “Designing inclusive companion robots to mitigate bias and enhance empathy in ai-driven care systems,” Journal of Orange Technology, vol. 2, no. 2, pp. 83–92, 2026.
[33] F. Tang, F. Yao, and M. Zhou, “Innovating diagnostic learning: How generative ai explanation styles influence cognitive load and confidence calibration in medical students,” Computers & Education, p. 105619, 2026.
[34] J. Seringa, A. Hirata, A. R. Pedro, R. Santana, and T. Magalh˜aes, “Health care professionals and data scientists’ perspectives on a machine learning system to anticipate and manage the risk of decompensation from patients with heart failure: qualitative interview study,” Journal of Medical Internet Research, vol. 27, p. e54990, 2025.
[35] B. Draghi, Z. Wang, P. Myles, and A. Tucker, “Identifying and handling data bias within primary health-care data using synthetic data generators,” Heliyon, vol. 10, no. 2, 2024.
[36] C. Trocin, P. Mikalef, Z. Papamitsiou, and K. Conboy, “Responsible ai for digital health: a synthesis and a research agenda,” Information Systems Frontiers, vol. 25, no. 6, pp. 2139–2157, 2023.
[37] Q. Abbas, W. Jeong, and S. W. Lee, “Explainable ai in clinical decision support systems: a meta-analysis of methods, applications, and usability challenges,” in Healthcare, vol. 13, no. 17. MDPI, 2025, p. 2154.
[38] M. M. R. Sweet, M. P. Ahmed, S. Akter, and S. A. Tisha, “Integrating deep learning and interpretable regression models for transparent decision support in healthcare diagnostics,” Journal of Medical and Health Studies, vol. 6, no. 5, pp. 17–38, 2025.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Deni Sunaryo, Antonius Ary Setyawan, Po Abas Sunarya, Sondang Visiana Sihotang, Putri Anahera Oganda
This work is licensed under a Creative Commons Attribution 4.0 International License.
