Canadian researchers working with Toronto General Hospital-University Health Network have developed a natural language processing (NLP) approach to predicting downstream radiology resource utilization, according to work published in the Journal of the American College of Radiology March 2. The team said their model could be key to streamlining healthcare management and reducing unnecessary costs.
A.D. Brown, MD, MBA, and J.R. Kachura, MD, call radiology a “finite healthcare resource”—one that’s coveted but in high demand at most facilities. Recent increases in the utilization of radiology services as baseline data and to confirm initial findings have even outpaced the workforce in some cases.
“Demand for radiologic services often outstrips capacity, and this mismatch between supply and demand has been associated with degraded healthcare quality and patient safety,” Brown and Kachura wrote in JACR. “Although institutions can implement strategies to respond to fluctuating demand, anticipating the under- or overutilization of radiologic resources is a challenge.”
The authors said an automated method for predicting future imaging resource utilization could help streamline the process, paving the way for capacity management strategies that could help meet the increased but unpredictable demand for radiology services. Using data from all hepatocellular carcinoma (HCC) surveillance CT exams performed at their hospital between 2010 and 2017, they used open-source NLP and machine learning software to parse free-text radiology reports into bag-of-words and term frequency-inverse document frequency (TF-IDF) models.
In NLP, bag-of-words refers to the frequency with which words occur in a report summary, while TF-IDF considers the number of times a word appears in the summary and measures the uniqueness of specific terms in the context of entire report collections. Brown and Kachura also used three machine learning techniques—logistic regression, support vector machine (SVM) and random forest—to make their predictions.
As a whole, the authors found bag-of-words models were somewhat inferior to the TF-IDF approach, with the TF-IDF and SVM combination yielding the most favorable results. The pairing outperformed all other models with an accuracy of 92 percent, a sensitivity of 83 percent, a specificity of 96 percent and an area under the curve of 0.971.
“In this study, all six models demonstrated a high level of accuracy in the classification task,” Brown and Kachura wrote. “These findings suggest that an algorithmic approach to text analysis could be used as a tool to help radiology administrators better predict changes in demand and proactively institute capacity management strategies to address fluctuations in demand.”
The combination of TF-IDF and logistic regression was the second-most accurate model, according to the results, with an accuracy of 91.4 percent and a sensitivity and specificity of 79 percent and 95.9 percent, respectively.
The authors’ work was subject to certain limitations, including the fact that their models were trained on data from a single institution, limiting the generalizability of the tech. They also didn’t consider patients’ clinical history or whether they’d undergone imaging exams before, and their institution lacked a standardized definition for liver lesions during the study period.
“However, despite the limitations of the dataset, the models were able to perform with a high level of accuracy across the six pipelines,” Brown and Kachura said. “NLP-based predictive models had excellent accuracy and predictive value with potential for data-driven applications in healthcare management and health service delivery. NLP models capable of predicting healthcare resource utilization may improve management decision-making, reduce costs and broaden access to care.”
After graduating from Indiana University-Bloomington with a bachelor’s in journalism, Anicka joined TriMed’s Chicago team in 2017 covering cardiology. Close to her heart is long-form journalism, Pilot G-2 pens, dark chocolate and her dog Harper Lee.