Cardiology hospital admission risk prediction: training, internal validation and technical implementation in the electronic health record
Source: PubMed Central Open Access, NCBI / U.S. National Library of Medicine
Abstract Aims Rising healthcare demand is increasingly outpacing available outpatient capacity in cardiology, where follow-up is often scheduled at fixed intervals despite substantial variation in individual patient risk. This uniform follow-up approach contributes to high outpatient workload and inefficient use of clinical resources. Accurate risk estimation using routinely collected electronic health record (EHR) data may support more individualized follow-up planning by identifying patients at very low risk of mortality or unplanned hospitalization, in whom follow-up intervals could be safely extended. s1 Methods and results We developed and validated a machine-learning model as part of the Cardiology Hospital Admission Risk Prediction (CHARP) program. The retrospective baseline cohort comprised 307 792 outpatient visits from 52 989 unique patients at Amsterdam UMC. The primary endpoint was a composite of unplanned cardiac hospitalization or all-cause death within 2 years; the 1-year composite endpoint served as a secondary outcome. Model development and validation were performed in a filtered, leakage-safe prediction cohort using gradient-boosted decision trees (XGBoost) with strict patient-level GroupKFold cross-validation. All predictions and performance metrics were retrospectively evaluated at the outpatient visit (trigger) level. Retrospective model performance was assessed using AUROC, AUPRC, Brier score, calibration curves, and SHAP-based explainability. The final
Abstract
Abstract Aims Rising healthcare demand is increasingly outpacing available outpatient capacity in cardiology, where follow-up is often scheduled at fixed intervals despite substantial variation in individual patient risk. This uniform follow-up approach contributes to high outpatient workload and inefficient use of clinical resources. Accurate risk estimation using routinely collected electronic health record (EHR) data may support more individualized follow-up planning by identifying patients at very low risk of mortality or unplanned hospitalization, in whom follow-up intervals could be safely extended. s1 Methods and results We developed and validated a machine-learning model as part of the Cardiology Hospital Admission Risk Prediction (CHARP) program. The retrospective baseline cohort comprised 307 792 outpatient visits from 52 989 unique patients at Amsterdam UMC. The primary endpoint was a composite of unplanned cardiac hospitalization or all-cause death within 2 years; the 1-year composite endpoint served as a secondary outcome. Model development and validation were performed in a filtered, leakage-safe prediction cohort using gradient-boosted decision trees (XGBoost) with strict patient-level GroupKFold cross-validation. All predictions and performance metrics were retrospectively evaluated at the outpatient visit (trigger) level. Retrospective model performance was assessed using AUROC, AUPRC, Brier score, calibration curves, and SHAP-based explainability. The final model was technically deployed within the electronic health record to allow automated, visit-level risk estimation in a prospective silent-running environment. In the filtered prediction cohort (199 961 visits), the 2-year composite endpoint prevalence was 16.8%. Across five cross-validation folds, the CHARP model achieved a mean AUROC of 0.77 ± 0.00 and AUPRC of 0.42 ± 0.01, with a Brier score of 0.12, indicating strong overall discrimination and good calibration. Key predictors included NT-proBNP, renal function indices, prior hospitalizations, and cardiac function measures. The deployed CHARP pipeline successfully generated daily risk predictions for all scheduled cardiology outpatients in the EHR environment throughout the silent-running period. s2 Conclusion This study shows that machine-learning applied to routine EHR data can deliver clinically meaningful, visit-level risk stratification for cardiology outpatients. The successful EHR integration of CHARP enables prospective evaluation of data-driven follow-up strategies aimed at reducing outpatient clinic burden through safe de-intensification of follow-up for low-risk patients. s3 Graphical Abstract Graphical Abstract Flow diagram titled “CHARP: EHR-based machine learning for low-risk identification in cardiology outpatient care”, arranged in four left-to-right panels. Panel one, “Rising outpatient demand”, lists increasing outpatient workload, uniform follow-up despite variable risk, and limited outpatient capacity. Panel two, “Routine EHR data”, lists labs, vitals, medications, imaging, and prior admissions, drawn from 307,792 outpatient visits across 52,989 patients at two university hospitals. Panel three, “Automated risk estimation in the EHR”, shows a gauge scaling from low through moderate to high risk, with top predictive factors NT-proBNP, renal function, and prior admissions, and notes that the model is EHR-integrated with daily automated scoring, explainable SHAP outputs, an AUROC of 0.77, and an AUPRC of 0.42. Panel four, “Low-risk identification”, shows a calendar icon and states that follow-up can be safely extended for low-risk patients. http://www.w3.org/1999/xlink float portrait ztag109_ga.jpg anchor ztag109_ga portrait graphical