Wiro J Niessen is Professor of biomedical image analysis at Erasmus MC in Rotterdam and Delft University of Technology, and Director of the biomedical image analysis platform of the European Organization for Imaging Research. For the past 20 years, he has been working to integrate AI in radiology. He believes the technology can help radiologists in countless scenarios from automating image analysis to picking up disease at pre-clinical stage and predicting treatment outcome. But a number of obstacles must be overcome before it fully enters clinical practice.

Myriad applications for AI in radiology 

One of Niessen’s key responsibilities has been to apply AI to support the Rotterdam study, a population cohort that was launched 15 years ago at Erasmus MC, and in which thousands of images have been generated to collect information on disease development. Niessen has been using machine learning (ML) for the past 15 years, first to automate the analysis of large imaging data sets, and now to find patterns with a prognostic value.

“We started to make the clinical researchers’ and clinicians’ jobs easier by automating the extraction of quantitative information from imaging data, which is tedious and time consuming to do by hand. 

“Now we also try to predict from a CT or MRI scan whether someone is going to respond to a certain therapy or if they are likely to at an early stage in neurodegenerative disease. If you collect a lot of data and follow the patients over time, you see whether they develop a disease or not, and whether they respond to therapy. You can always dive back into the data and see whether there are patterns that can predict disease,” he said. 

Niessen, who talked to Insights at the European Congress of Radiology in Vienna, is involved at every stage of AI technology development, from research all the way up to commercialisation. He notably founded the Erasmus MC spin-off company Quantib, which brings solutions that were initiated in research to the market. 

For a long time, he has only used conventional ML, first selecting relevant features in an image, and then assessing their diagnostic or prognostic value. In oncology imaging, that means for instance capturing the boundaries of a tumour and computing features such as tumour texture, volume, intensity patterns and density, to build classifiers that can help in diagnosis and prognosis. Niessen’s group has worked with common ML tools such as support vector machines, decision trees and random forests. 

Deep learning hype is set to continue

For the past four years, his group has been increasingly working with deep learning (DL), to train systems to perform tasks on their own by optimising efficient networks. DL originates from the vision field, but breakthroughs like neural network architecture or convolutional neural network can perfectly be adapted to medicine, Niessen explained. “This is very much a field in which you can benefit from all these advances. In daily clinical care and research, we are collecting a large number of cases and data, and it is now our challenge to exploit this wealth of information with AI techniques,” he said. 

DL has very much become a trend in medical imaging and the industry is increasingly presenting new solutions. The hype is set to continue in the near future, as more and more DL-based technology will emerge, Niessen predicted.

One important aspect is generalisation, he said. “You need to make sure that the dataset on which you train the algorithm is representative of the environment in which you employ it. For example, population studies provide large opportunities to design proper algorithms for both ML and DL, and therefore are very good to understand preclinical stage of disease,” he said. 

Obstacles to clinical integration

Clinical practice is also a minefield of data ready to be exploited by AI. In prostate cancer, ML could help to know cancer severity and grade, and to base a prognosis on MR images and potentially spare the patient a biopsy, Niessen suggested. “In oncology, it would be of great benefit if we could predict disease progression or therapy response based on imaging data alone. It’s already being used as an early screening and surveillance tool.” Besides cancer, DL can also help to detect white matter lesions in the brain, with the algorithm learning from examples.

Conversely, not many of the ML or DL tools developed in research are ready for clinical practice yet. “A lot of the developments are still in the research stage. Integration into clinical practice is still a challenge,” Niessen said. Perhaps even up to 95% of information used in daily practice is still qualitative information, for instance the radiology report, Niessen estimated. But things are changing in that aspect as well. “Step by step it’s getting better, and structured reporting is becoming more important in radiology, to provide more objective information. If efforts continue in that direction, more data will be available to train ML and DL algorithms, and the structured report could increasingly be filled by algorithms. We’re still in the early days,” he said.

Integrated AI is on the rise

AI is already a market because people are buying software, which increasingly use AI technology, each time they buy a scanner. The level of integrated AI is therefore rising and it will keep on growing. “Every year there will be more AI. In 10, 15 or 20 years, I think it will be the exception if an image is analysed without the help of AI, “ he said. Certain tools are already performing better than humans in some settings, for instance in TB screening. But ultimately, AI tools perform relatively isolated tasks, whereas a radiologist does a combination of different tasks. Therefore integrating ML and DL based solutions in clinical practice remains a major challenge for the years to come, and breakthrough will come if AI proves to be of additional value in clinical practice, Niessen explained.

“Today there aren’t that many solutions that really help in workflow. We must develop tools that make the life easier, improve accuracy and usability, because only then will they be used on a large scale. This in turn will trigger a demand for new products and technical advances. It’s a learning process.”

Boundaries between specialities will reduce

Radiologists, computer scientists, engineers and everyone involved must collaborate to advance these tools. There is no doubt that AI will impregnate every aspect of radiology practice at different levels in the foreseeable future. “First simple tasks will be delegated and then the impact will be more important in some areas than others. I think prostate cancer is really an interesting area, to avoid biopsy. After 15 years of implementation, we’ll see how far we have come. Radiology has evolved tremendously over the past 15 years,” he said. 

AI integration in clinical practice will bring changes inside the hospital and help reduce boundaries between specialties, especially between pathology and imaging. Weekly multidisciplinary meetings and other aspects of medical practice that consume a lot of time and resources will partially be replaced or upgraded by AI technologies and AI supported integrated diagnostic reports will become standard, he predicted. 

“We’ll be able to see what the integrated data says. With AI, we should strive to bring all the data into one common integrated report. In such a diagnostic competence centre, walls between departments will cease to exist. These developments are promising for the development of diagnostic and prognostic reports that facilitate the transition to precision medicine, in which the therapy is optimised for the individual.”