AI-aided physicians are better at diagnosing real-world skin cancer than either AI or physicians alone, and the least-experienced clinicians derive the most benefit from the algorithmic assist.
An international team of researchers made the findings, and Nature Medicine has published the study.
The team, whose U.S. members included Allan Helpern, MD, of Memorial Sloan Kettering, first trained a convolutional neural network on dermatologic photos from a publicly available image repository.
They tested their human-machine diagnostic combination using 1,521 self-shot photos of 596 suspicious skin lesions submitted by 93 participating patients.
Participating physician raters numbered 302 representing 41 countries. A little more than half the field, 169 (56%), were board-certified dermatologists. The rest were either dermatology residents (25.5%) or general practitioners (12.6%).
When the researchers had the raters diagnose batches of images with and without AI support, they found decision support with AI-based multiclass probabilities improved the raters’ overall accuracy from 63.6% to 77.0%.
Meanwhile the least experienced rater cohort changed initial diagnosis more often than its expert-level counterpart.
And the latter group benefited from the AI input only marginally—and only if these physicians weren’t confident in their initial diagnosis.
“This finding suggests that, if experts have high confidence in their initial diagnosis, they should ignore AI-based support or not use it at all,” the authors comment in their discussion.
In a news brief published by the University of Queensland in Australia, study co-author Monika Janda, DPhil, underscores the superiority of “human plus machine” over either one alone.
“This is important because AI decision support has slowly started to infiltrate healthcare settings, and yet few studies have tested its performance in real world settings or how clinicians interact with it,” Janda says. “These findings indicate a combined AI-human approach to skin cancer diagnosis may be the most relevant for clinicians in the future.”
The study is available in full for free.
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.