AI can detect brain hemorrhages in CT scans more accurately than some radiologists, according to new findings published by Proceedings of the National Academy of Sciences.
Researchers from the University of California San Francisco (UCSF) and University of California Berkeley developed a convolutional neural network (CNN) that could evaluate imaging findings in just one second, training it with more than 4,000 head CT scans. The CNN also traced outlines of each individual abnormality it detected.
The algorithm achieved an area under the ROC curve (AUC) of 0.991 and performed better than two out of four practicing radiologists who participated in the study.
“We wanted something that was practical, and for this technology to be useful clinically, the accuracy level needs to be close to perfect,” corresponding author Esther Yuh, MD, PhD, associate professor of radiology at UCSF, said in a news release from the school. “The performance bar is high for this application, due to the potential consequences of a missed abnormality, and people won’t tolerate less than human performance or accuracy.”
Yuh noted in the news release that AI does still struggle with knowing if exams made up for dozens of different images are “normal” or not.
“Achieving 95% accuracy on a single image, or even 99%, is not OK, because in a series of 30 images, you’ll make an incorrect call on one of every 2 or 3 scans,” Yuh said. “To make this clinically useful, you have to get all 30 images correct—what we call exam level accuracy. If a computer is pointing out a lot of false positives, it will slow the radiologist down, and may lead to more errors.”
One aspect of the team’s research that differentiates it from others looking into AI was that it “fed” just a portion of each image to the algorithm at a time, allowing it to contextualize as it made its evaluation.
“We took the approach of marking out every abnormality—that’s why we had much, much better data,” said Jitendra Malik, PhD, a professor at UC Berkley and co-corresponding author of the study. “Then we made the best use possible of that data. That’s how we achieved success.”
Michael has more than 16 years of experience as a professional writer and editor. He has written at length about cardiology, radiology, artificial intelligence and other key healthcare topics.