AI trained mostly on chest x-rays from men will perform poorly when a clinician applies it to women patients. An algorithm for diagnosing skin cancer on dermatologic photos will botch the job if the patient is dark-skinned and most of the training images came from the fair-skinned. And so on.
Examples of this brand of bias are accumulating into an unignorable chink in healthcare AI’s armor.
Granted, gathering AI-suitable training data from widely diverse patient populations is difficult even when it’s doable. But if AI in healthcare is unable to help some of the most vulnerable patient demographics, how can it help the cause of improving America’s health at the population level?
Three Stanford MD/PhDs lay out the best thinking on the subject in an opinion piece published Nov. 17 in Scientific American.
“Bias in AI is a complex issue; simply providing diverse training data does not guarantee elimination of bias,” write Kaushal, Altman and Langlotz. “Several other concerns have been raised—for example, lack of diversity among developers and funders of AI tools; framing of problems from the perspective of majority groups; implicitly biased assumptions about data; and use of outputs of AI tools to perpetuate biases, either inadvertently or explicitly.”
They note that researchers are trying to get around the problem of un-diverse data by building algorithms that can extrapolate broad outputs from limited inputs.
“From these innovations may emerge new ways to decrease AI’s need for huge data sets,” the authors write. “But for now, ensuring diversity of data used to train algorithms is central to our ability to understand and mitigate biases of AI.”
With that they call for building the technical, regulatory, economic and privacy infrastructure needed to glean data that’s not only big but also diverse enough to train medical AI for the benefit of all patients everywhere.
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.