Generally speaking, clinicians have little idea how AI algorithms at their disposal were trained.
The lack often includes the makeup of the patient data—demographics, relative health status and so on—that went into creating these tools.
As a result, there can be no telling how capable any given algorithm is of diagnosing any particular patient in any given clinician’s care.
For these reasons, healthcare institutions “need to establish a robust AI governance program that addresses all phases of technology adoption and use.”
The observation and advice are from ECRI, which has named “insufficient governance of AI in medical technologies” No. 5 among its top 10 tech health-tech hazards of 2024.
The nonprofit patient-safety organization, which is informally known as the Consumer Reports of healthcare, arrived at its list after poring over data from numerous sources.
In an executive summary of the report, ECRI says the topics it spotlights are not necessarily the most frequently reported problems or the ones associated with the most severe consequences, although its analysts do give some weight to such considerations.
Instead, “the list reflects our judgment about which risks should be given attention now to help care providers, as well as device manufacturers, prioritize their patient safety efforts.”
Here’s the full list.
- Medical devices may pose usability challenges for home users, risking misuse and patient harm.
- Inadequate or onerous device cleaning instructions endanger patients.
- Sterile drug compounding without the use of technological safeguards increases the risk of medication errors.
- Overlooked environmental impacts of patient care endanger public health.
- Insufficient governance of AI used in medical technologies risks inappropriate care decisions.
- Ransomware targeting the healthcare sector remains a critical threat.
- Increased burn risk with single-foil electrosurgical return electrodes.
- Infusion pump damage remains a medication safety concern.
- Poor QC of implantable orthopedic products can lead to surgical delays and patient harm.
- Third-party web analytics software can compromise patient confidentiality.
All these items represent “problems that can be avoided or risks that can be minimized through the careful management of technologies,” ECRI explains. “With the additional content provided in the full report, the list serves as a tool to help technology managers and device users manage the risks efficiently and effectively.”
The executive summary is downloadable in full for free. Accessing the briefing launches a link to the full report, which is available to ECRI subscribers.
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.