The healthcare industry has a lot of hope for machine learning solutions across patient care. However, there are many barriers to widespread adoption keeping machine learning from being implemented into clinical practice.
One of the biggest issues for machine learning is a lack of randomized clinical trials (RCTs), which are typically required before clinical adoption. Plus, there are questions about how machine learning interventions are being incorporated into clinical trials in healthcare.
Researchers from Boston and Connecticut conducted a systematic review of several databases to find relevant articles published, using the search terms “machine learning, clinical decision-making and RCTs.” They published their findings in JAMA Network Open.
The review found a lack of RCTs for medical machine learning interventions, as well as a need for “additional well-designed, transparent, and inclusive RCTs for machine learning interventions to promote use in the clinic,” wrote first author Deborah Plana, BS, of Harvard Medical School, et al. Researchers also noted there is growing concern that machine learning interventions are being released without the gold standard of RCTs.
In fact, researchers found 41 machine learning RCTs. That’s compared to 343 medical AI or machine learning interventions approved by the U.S. Food and Drug Administration (FDA). That may mean most FDA-approved machine learning medical devices have been approved without demonstrating efficacy in an RCT. As such, there may be a lower burden of evidence required for AI or machine learning algorithm clearance compared to pharmaceutical drugs.
In addition, RCTs of machine learning interventions found by researchers did not meet several standards. Of the 41, 37 trials, or 90%, did not share code and data along with study results, 38 (93%) did not analyze poor-quality or unavailable input data and 38 (93%) did not assess performance errors.
“These results suggest that machine learning RCT reporting quality needs improvement,” Pana et al. wrote.
The findings underscore not only a need for more RCTs for machine learning interventions, but also a concern for the quality of those medical machine learning RCTs, with opportunities to improve reporting transparency and inclusivity.
Amy joined TriMed Media as a Senior Writer for HealthExec after covering home care for three years. When not writing about all things healthcare, she fulfills her lifelong dream of becoming a pirate by sailing in regattas and enjoying rum. Fun fact: she sailed 333 miles across Lake Michigan in the Chicago Yacht Club "Race to Mackinac."