EXPLORATORY DATA ANALYSIS AS A FOUNDATION FOR TRUSTWORTHY CLINICAL MACHINE LEARNING
DOI:
https://doi.org/10.37943/JXDB7524Keywords:
exploratory data analysis, early clinical triage, temporal drift, missing data mechanisms, documentation bias, data-centric machine learning, methodological reliabilityAbstract
Exploratory data analysis is frequently treated as a preliminary or purely descriptive step in clinical machine learning studies. However, many methodological failures of clinical prediction models arise not from algorithm selection, but from unrecognized properties of the data-generating process, including temporal non-stationarity, non-random missingness, and systematic documentation effects. These issues are particularly critical in early clinical triage settings, where risk assessment must be performed at hospital admission using limited and heterogeneous clinical information. In this study, we examine exploratory data analysis as a dedicated methodological stage that precedes and constrains downstream machine learning design in early triage–level mortality risk prediction. Using a large-scale national registry of hospitalized patients with confirmed COVID-19 infection, we conduct a structured exploratory analysis of admission-level clinical data, focusing on temporal drift in outcome prevalence and patient case-mix, feature-level missingness mechanisms, documentation bias in symptom recording, and structural redundancy among clinical indicators. The results demonstrate pronounced temporal non-stationarity across pandemic phases, indicating that commonly used random data splitting strategies may introduce methodological bias even prior to model development. Missingness patterns are shown to be structured and outcome-dependent rather than random, reflecting both clinical severity and organizational processes. In addition, several apparent univariate associations between symptoms and outcomes are attributable to documentation bias rather than underlying clinical effects. These findings provide concrete diagnostic evidence that exploratory analysis directly constrains valid choices related to validation design, handling and representation of missing information, feature specification, and interpretation boundaries. The findings suggest that exploratory data analysis should be regarded as a fundamental methodological stage supporting reliable and trustworthy clinical machine learning in early-stage decision-making settings.
References
Al-Nafjan, A., Aljuhani, A., Alshebel, A., Alharbi, A., Alshehri, A. (2025). Artificial intelligence in predictive healthcare: A systematic review. Journal of Clinical Medicine, 14(19), 6752. https://doi.org/10.3390/jcm14196752
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., King, D. (2019). Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine, 17, 195. https://doi.org/10.1186/s12916-019-1426-2
Ghassemi, M., Naumann, T., Schulam, P., Beam, A. L., Chen, I. Y., et al. (2020). A review of challenges and opportunities in machine learning for health. AMIA Joint Summits on Translational Science Proceedings, 2020, 191–200. https://pubmed.ncbi.nlm.nih.gov/32477638/
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., et al. (2021). WILDS: A benchmark of in-the-wild distribution shifts. Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, 139, 5637–5664. https://proceedings.mlr.press/v139/koh21a.html
Subbaswamy, A., Saria, S. (2020). From development to deployment: Dataset shift, causality, and shift-stable models in health AI. Biostatistics, 21(2), 345–352. https://doi.org/10.1093/biostatistics/kxz041
Wynants, L., Van Calster, B., Collins, G. S., Riley, R. D., Heinze, G., et al. (2020). Prediction models for diagnosis and prognosis of COVID-19: Systematic review and critical appraisal. BMJ, 369, m1328. https://doi.org/10.1136/bmj.m1328
. Rajwa, B., Naved, M. M. A., Adibuzzaman, M., Grama, A. Y., Khan, B. A., et al. (2024). Identification of predictive patient characteristics for assessing the probability of COVID-19 in-hospital mortality. PLOS Digital Health, 3(4), e0000327. https://doi.org/10.1371/journal.pdig.0000327
Ranzani, O. T., Bastos, L. S. L., Gelli, J. G. M., Marchesi, J. F., Baião, F., et al. (2021). Characterisation of the first 250 000 hospital admissions for COVID-19 in Brazil: a retrospective analysis of nationwide data. The Lancet Respiratory Medicine, 9(4), 407–418. https://doi.org/10.1016/S2213-2600(20)30560-9
de Souza F.S.H., Hojo-Souza N.S., Batista B.D.O., da Silva C.M., Guidoni D.L. (2021). On the analysis of mortality risk factors for hospitalized COVID-19 patients in Brazil: Data from the SIVEP-Gripe database. PLOS ONE, 16(3), e0248580. https://doi.org/10.1371/journal.pone.0248580
Guo, L. L., Pfohl, S. R., Fries, J., Johnson, A. E. W., Posada, J., et al. (2022). Evaluation of domain generalization and adaptation on improving model robustness to temporal dataset shift in clinical medicine. Scientific Reports, 12, 2726. https://doi.org/10.1038/s41598-022-06484-1
Lu, S.-C., Song, W., Pfob, A., Gibbons, C. (2025). Assessing the representativeness of large medical data using population stability index. BMC Medical Research Methodology, 25, 44. https://doi.org/10.1186/s12874-025-02474-9
Ren, W., Liu, Z., Wu, Y., Zhang, Z., Hong, S., et al. (2024). Moving beyond medical statistics: A systematic review on missing data handling in electronic health records. Health Data Science, 2024, Article 0176. https://doi.org/10.34133/hds.0176
Afkanpour, M., Hosseinzadeh, E., Tabesh, H. (2024). Identify the most appropriate imputation method for handling missing values in clinical structured datasets: A systematic review. BMC Medical Research Methodology, 24, 188. https://doi.org/10.1186/s12874-024-02310-6
dos Santos Silva, G. F., Barcellos Filho, F. N., Wichmann, R. M., Costa da Silva Junior, F., Chiavegatto Filho, A. D. P. (2025). Strategies for detecting and mitigating dataset shift in machine learning for health predictions: A systematic review. Journal of Biomedical Informatics, 170, 104902. https://doi.org/10.1016/j.jbi.2025.104902
Finlayson, S. G., Subbaswamy, A., Singh, K., Saria, S. (2021). The Clinician and Dataset Shift in Artificial Intelligence. The New England Journal of Medicine, 385(3), 283–286. https://doi.org/10.1056/NEJMc2104626
Guo, L. L., Pfohl, S. R., Fries, J., Posada, J., Fleming, S. L., et al. (2021). Systematic review of approaches to preserve machine learning performance in the presence of temporal dataset shift in clinical medicine. Applied Clinical Informatics, 12(4), 808–815. https://doi.org/10.1055/s-0041-1735184
Vela, D., Sharp, A., Zhang, R., Nguyen, T., Hoang, A., et al. (2022). Temporal quality degradation in AI models. Scientific Reports, 12, 11654. https://doi.org/10.1038/s41598-022-15245-z
Sperrin, M., Martin, G. P., Sisk, R., & Peek, N. (2020). Missing data should be handled differently for prediction than for description or causal explanation. Journal of Clinical Epidemiology, 125, 183–187. https://doi.org/10.1016/j.jclinepi.2020.03.028
Nijman, S. W. J., Leeuwenberg, A. M., Beekers, I., Verkouter, I., Jacobs, J. J. L., et al. (2022). Missing data is poorly handled and reported in prediction model studies using machine learning: A literature review. Journal of Clinical Epidemiology, 142, 218–229. https://doi.org/10.1016/j.jclinepi.2021.11.023
Perets, O., Cohen, S., Morris, A., Benjamins, J., Ben-Israel, D., et al. (2024). Inherent bias in electronic health records: A scoping review of sources of bias. ACM Journal of Data and Information Quality, 15(4), Article 35. https://doi.org/10.1145/3757924
Rahmani, K., Thapa, R., Tsou, P., Casie Chetty, S., Barnes, G., et al. (2023). Assessing the effects of data drift on the performance of machine learning models used in clinical sepsis prediction. International Journal of Medical Informatics, 173, 104930. https://doi.org/10.1016/j.ijmedinf.2022.104930
Tamang, L., Bouadjenek, M. R., Dazeley, R., Aryal, S. (2025). Handling out-of-distribution data: A survey. IEEE Transactions on Knowledge and Data Engineering, 37(10), 5948–5966. https://doi.org/10.1109/TKDE.2025.3592614
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Articles are open access under the Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish a manuscript in this journal agree to the following terms:
- The authors reserve the right to authorship of their work and transfer to the journal the right of first publication under the terms of the Creative Commons Attribution License, which allows others to freely distribute the published work with a mandatory link to the the original work and the first publication of the work in this journal.
- Authors have the right to conclude independent additional agreements that relate to the non-exclusive distribution of the work in the form in which it was published by this journal (for example, to post the work in the electronic repository of the institution or publish as part of a monograph), providing the link to the first publication of the work in this journal.
- Other terms stated in the Copyright Agreement.