Assessment of Electronic Clinical Monitoring Systems in the Pediatric Intensive Care Unit: Prospective Concordance Study.
Source: PubMed, NCBI / U.S. National Library of Medicine
Electronic health record (EHR) data are being increasingly used for retrospective observational research through large, robust databases, and advanced data extraction tools. We sought to assess the reliability of vital sign, ventilator, and continuous medication data captured in the EHR in a pediatric intensive care unit. We conducted a prospective concordance study of children receiving invasive mechanical ventilation in June 2025. Data sources included (1) a bedside clinical researcher, (2) automated EHR extraction, and (3) a continuous vital sign monitoring system. Vital signs from the EHR were compared to those obtained through the continuous vital sign monitoring system. Ventilator and medication data were compared to the bedside observations. Differences were measured as means with SDs or median differences with IQRs, and a 10% error rate was used as a concordance adequacy threshold. We obtained 110 bedside observations from 27 unique patients. Five of 8 measured vital signs in the EHR met the 10% concordance adequacy threshold (respiratory rate, 2.0/20.4, 9.8%; cuff systolic blood pressure, 8.3/99.4, 8.4%; invasive systolic blood pressure, 4.0/88.0, 4.5%; invasive diastolic blood pressure, 3.8/50.0, 7.6%; and oxygen saturation, 2/96, 2.1%), with heart rate (11.4/113, 10.1%), cuff diastolic blood pressure (9/58.6, 15.3%), and end-tidal carbon dioxide (4.0/36.6, 10.9%) failing to meet concordance. Occasional rare clinically meaningful outliers were observed, such as a sy
Abstract
Electronic health record (EHR) data are being increasingly used for retrospective observational research through large, robust databases, and advanced data extraction tools. We sought to assess the reliability of vital sign, ventilator, and continuous medication data captured in the EHR in a pediatric intensive care unit. We conducted a prospective concordance study of children receiving invasive mechanical ventilation in June 2025. Data sources included (1) a bedside clinical researcher, (2) automated EHR extraction, and (3) a continuous vital sign monitoring system. Vital signs from the EHR were compared to those obtained through the continuous vital sign monitoring system. Ventilator and medication data were compared to the bedside observations. Differences were measured as means with SDs or median differences with IQRs, and a 10% error rate was used as a concordance adequacy threshold. We obtained 110 bedside observations from 27 unique patients. Five of 8 measured vital signs in the EHR met the 10% concordance adequacy threshold (respiratory rate, 2.0/20.4, 9.8%; cuff systolic blood pressure, 8.3/99.4, 8.4%; invasive systolic blood pressure, 4.0/88.0, 4.5%; invasive diastolic blood pressure, 3.8/50.0, 7.6%; and oxygen saturation, 2/96, 2.1%), with heart rate (11.4/113, 10.1%), cuff diastolic blood pressure (9/58.6, 15.3%), and end-tidal carbon dioxide (4.0/36.6, 10.9%) failing to meet concordance. Occasional rare clinically meaningful outliers were observed, such as a systolic blood pressure difference of 31 mm Hg, a heart rate difference of 87 beats per minute, and a respiratory rate difference of 18 breaths per minute. All 6 ventilator settings met concordance adequacy criteria between the EHR and the bedside observations, with a median difference of 0.0 (IQR 0-0). Outliers were less common but included differences such as a respiratory rate of 34 breaths per minute and an inspiratory time of 0.3 seconds. Continuous medication dosing concordance was variable, with an overall low concordance between 30.8% (339.3/11) and 31.4% (345.4/11). EHR data captured in the pediatric intensive care unit in our single-center sample were mostly concordant with other measured observations for vital signs and ventilator settings, but less concordant for continuous medications.