Researchers at the University of California, San Francisco (UCSF) found Google Translate (GT), Google’s AI-based app, can help non-English speaking patients and their providers. However, it's not perfect, according to research published in JAMA Internal Medicine.
Prior studies of GT have shown limited accuracy and one study found GT Spanish translations of patient education materials were only 60 percent accurate. Additionally, 4 percent of translations resulted in serious error.
Google changed its algorithm in 2017 and the researchers, led by Elaine Khoong, MD, of Zuckerberg San Francisco General Hospital and UCSF, sought to evaluate GT for translating emergency department discharge instructions into Spanish and Chinese.
They specifically assessed 100 free-texted ED discharge instructions (i.e. diagnosis and/or results, follow-up instructions, medication instructions) and oversampled for medication changes and common complaints. They analyzed each sentence by content category, use of medical jargon, and the presence of nonstandard English, including spelling and grammar errors, abbreviations and colloquial English.
The researchers translated the instructions into Spanish and Chinese and then bilingual translators translated them back to English. Potential harm from inaccurate translations was assessed by two clinicians using a rating system: clinically nonsignificant, clinically significant, and life-threatening potential harm.
There were 647 sentences in the 100 sets of patient instructions. The researchers found 92 percent of instructions were accurately translated into Spanish and 81 percent were accurately translated to Chinese. Still, there was potential for “clinically significant” harm in mistranslations. This happened in 2 percent of Spanish mistranslations and in 8 percent of Chinese mistranslations.
“While GT can supplement (not replace) written English instructions, machine-translated instructions should include a warning about potentially inaccurate translations,” Khoong and colleagues wrote.
To avoid such harm, providers can have patients read GT translations while receiving verbal instructions, be attentive about spelling and grammar and avoid medical jargon.
“Google Translate is more accurate than a lot of clinicians believe, and I think it’s definitely more useful than not providing anything at all,” Khoong said in a prepared statement. “We cautiously support its use.”
As a senior news writer for TriMed, Subrata covers cardiology, clinical innovation and healthcare business. She has a master’s degree in communication management and 12 years of experience in journalism and public relations.