Researchers have piloted a deep learning algorithm that can recognize visual cues of sickness, also known as “clinical gestalt,” in facial photos.
The team trained the model on 126 photos of healthy individuals in the Chicago Face Database and 26 photos that were digitally manipulated to make members of that subset appear acutely ill.
They externally validated the algorithm on photos of 22 volunteers before and after they were injected with lipopolysaccharide (LPS), which safely induces symptoms resembling those of sepsis.
When queried to look for combinations of multiple “sick” facial features such as pale lips, sunken eyes and droopy mouth, the model drew from four distinct convolutional neural networks—and achieved 100% sensitivity.
The technique was significantly weaker at ruling out sickness in healthy faces, notching specificity of only 42%.
The researchers suggest their contribution is showing that medical AI algorithms can be successfully trained on synthetically generated data.
This could offer a useful option for helping diagnose sicknesses with a dearth of real-world datasets, they point out.
José Castela Forte, an MD/PhD at Groningen University Medical Center in the Netherlands, and colleagues describe the work in a study published by Frontiers in Medicine.
“We can say this deep learning algorithm trained on simulated ‘gestalt’ data distinguished between photographs of acutely ill and healthy people above chance level, surpassing the performance of non-experts, but fell below the performance of trained clinicians in other studies of different health conditions,” the authors write.
The high sensitivity warrants further research into the use of clinical gestalt for identifying acute illness in emergency and intensive-care settings, they add.
Forte et al. acknowledge several limitations in their study design, not least the small sample.
They also note that the risk of dissimilarity between training and testing data increases as the size of the dataset increases, while the sheer volume of images would preclude manual verification of each.
For these reasons, the authors write, “the development of methodologies and standards to measure the quality of synthetic data is necessary before it can be used more widely.”
Nevertheless, in the present study:
A deep learning algorithm trained on a synthetic, augmented dataset of facial photographs distinguished between healthy and simulated acutely ill individuals, demonstrating that synthetically generated data can be used to develop algorithms for health conditions in which large datasets are difficult to obtain. These results support the potential of facial feature analysis algorithms to support the diagnosis of acute illness.”
The study is available in full for free.
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.