Drift in model calibration performance was detected immediately, with minor fluctuations in discrimination
By Elana Gotkine HealthDay Reporter
MONDAY, Dec. 5, 2022 (HealthDay News) — Dynamically updating models can improve prediction of 28-day survival in hospitalized COVID-19 patients, according to a study published online Nov. 10 in Nature Communications.
Todd J. Levy, from the Feinstein Institutes for Medical Research at Northwell Health in Manhasset, New York, and colleagues developed a framework for continuously monitoring and updating prognostic models and applied it to predict 28-day survival in patients with COVID-19. Demographic, laboratory, and clinical data were obtained from electronic health records of 34,912 hospitalized COVID-19 patients from March 2020 until May 2022. A dynamic self-monitoring, auto-updating approach was employed; a 2,000-patient sliding window incremented at 500-patient intervals was used to monitor the calibration and apply model updating strategies. This framework was applied to three modeling methods of prediction: a custom generalized linear model, logistic regression, and gradient boosted decision trees.
The researchers found that the resulting models maintained good discrimination and calibration throughout the waves of the pandemic, irrespective of model architecture. Drift in model calibration performance was detected immediately, with minor fluctuations in discrimination. The models always outperformed their initial, static, versions.
“COVID-19 was one of the most dynamic diseases we’ve witnessed in modern history and information about how to care for patients was constantly evolving,” a coauthor said in a statement. “By harnessing data and developing a real-time auto-updating clinical tool, we set out to create a tool that accounts for these developments and helps clinicians make the decisions they need to deliver better care.”
Copyright © 2022 HealthDay. All rights reserved.