A team of researchers in San Francisco have developed an EHR-driven deep learning model that’s able to accurately predict the prognosis of patients with rheumatoid arthritis (RA), according to a study published in JAMA Network Open.
First author Beau Norgeot, MS, and colleagues’ aim was to develop an accurate AI model that could forecast complex disease outcomes using EHR data, so RA was a natural pick for their prognostic study. Data from the American College of Rheumatology state 42 percent of patients with the condition had moderate or high disease activity at the time of their last doctor’s visit, and it’s hard to find a blanket treatment for RA since every patient’s disease trajectory is so different.
“Knowing the future condition of a patient would enable a physician to customize current therapeutic options to prevent disease worsening, but predicting that future condition requires sophisticated modeling and information,” Norgeot, of the University of California, San Francisco, and co-authors said in JAMA. “If successful, the ability to forecast disease activity could be clinically used to inform the aggressiveness of treatment on an individualized basis at each clinical visit.”
The researchers developed a novel deep learning model using data from two distinct healthcare systems with different EHRs: a university hospital (UH) and a public safety-net hospital (SNH). Both institutions had “substantially different patient populations and treatment patterns,” according to the study.
The UH provided more than a million unique patient records starting in January 2012, while the SNH had records for 65,000 patients starting in January 2013. Norgeot et al. extracted structured data from the EHRs, including medications, demographics, laboratories and prior measures of disease activity, and built a longitudinal model that could predict disease activity for RA patients at their next rheumatology clinic visit.
A total of 578 UH patients and 242 SNH patients were included in the study. Individuals treated at the UH were seen more frequently than those treated at the SNH (100 days between visits compared to 180 days) and were more frequently prescribed higher-class medications.
At the UH, the team’s deep learning model reached an area under the receiver operating characteristic curve (AUROC) of 0.91 in a test cohort of 116 patients—a favorable result that held somewhat when tested again in an SNH cohort of 117 patients. Despite differing baseline characteristics, the UH-trained model achieved an AUROC of 0.74 in SNH patients.
The authors said their results suggest deep learning models can be successfully trained on cohorts of just a few hundred patients to accurately predict complex disease outcomes, and their work is proof that those models can be generalizable. Still, they acknowledged the accuracy of their results was poorer in the SNH cohort.
“Given the many differences in the demographics and social determinants among the patients in these centers, we believe that the ability of the model to function significantly above random is still promising,” they wrote. “By considering no more than the most recent year of each patient’s history but allowing patients to have as few as four months of history, the model may have utility for patients at all stages of their care, if proven in future prospective studies.
“Although the amount of data that a rheumatologist must synthesize in a single visit to make decisions is large and increasing, the results presented herein suggest that use of artificial intelligence models to assist with predictive tasks in the near future is promising.”
After graduating from Indiana University-Bloomington with a bachelor’s in journalism, Anicka joined TriMed’s Chicago team in 2017 covering cardiology. Close to her heart is long-form journalism, Pilot G-2 pens, dark chocolate and her dog Harper Lee.