The future of AI and medical imaging, from Nvidia to Harvard

It has been six years since Geoffrey Hinton said “We need to stop training radiologists now,” insisting that “it’s completely obvious that within five years, deep learning is going to do better than radiologists.” Instead, the future of medical imaging, it seems, remains firmly in the hands of radiologists — who have adopted artificial intelligence (AI) as a collaborative tool to boost medical imaging, one of the most essential areas of healthcare that is used throughout the patient journey. 

What is evolving, however, are significant open-source efforts to bring AI models related to medical imaging into clinical settings at scale, as well as making sure the medical imaging data that trains those AI models is robust, diverse and available to all. 

Integrating AI models into clinical workflows

To tackle the former goal, Nvidia announced today at the annual meeting of the Radiology Society of North America (RSNA) that MONAI, an open-source medical-imaging AI framework accelerated by Nvidia, is making it easier to integrate AI models into clinical workflows with MONAI Application Packages (MAPs), delivered through MONAI Deploy. 

Nvidia and King’s College London introduced MONAI in April 2020 to simplify AI medical imaging workflows. This helps transform raw imaging data into interactive digital twins to improve analysis or diagnostics, or guide surgical instruments. The development and adoption of the platform now has over 600,000 downloads, half of these in the last six months. 

Medical-imaging leaders, including UCSF, Cincinnati Children’s Hospital and startup Qure AI, are adopting MONAI Deploy to turn research breakthroughs into clinical impact, Nvidia said in a press release. In addition, all the major cloud providers, including Amazon, Google, Microsoft and Oracle, are supporting MAPs, enabling researchers and companies using MONAI Deploy to run AI applications on their platform, either by using containers or with native app integration.

“MONAI has really established itself in the research and development community as the PyTorch of healthcare,” said David Niewolny, director of healthcare business development at Nvidia, in a press briefing in advance of the announcements. “It’s purpose-built for radiology, but now expanding into pathology and digital surgery, and really tackles the entire AI lifecycle, bridging that gap between this research community and deployment.” 

For example, Cincinnati Children’s Hospital is creating a MAP for an AI model that automates total cardiac volume segmentation from CT images, aiding pediatric heart transplant patients in a project funded by the National Institutes of Health. “It is accelerating decision-making time for pediatric transplant patients,” he said. “It truly has the potential to save a number of children’s lives.” 

Scaling AI and medical imaging to wider audience

The integration of MONAI by all the cloud hyperscalers allows this research to scale beyond one hospital to a much wider audience, Niewolny added. For example, The MAP connector has been integrated with Amazon HealthLake Imaging, which allows clinicians to view, process and segment medical images in real time. And Google Cloud’s Medical Imaging Suite has integrated MONAI into its platform to enable clinicians to deploy AI-assisted annotation tools that help automate the highly manual and repetitive task of labeling medical images. 

In addition, “Oracle Cloud infrastructure has some pretty big things planned,” he added, particularly in light of Oracle’s recent acquisition of Cerner, one of the largest medical record companies in the world. 

“It’s fantastic to see this gap being closed between the model developers and the folks actually doing the clinical deployment,” he said. “That is really turbocharging AI innovation throughout the medical imaging ecosystem.” 

Developing diverse medical image datasets

Of course, even with better hardware and infrastructure, advances in medical imaging, AI and data science require the right medical imaging datasets to make sure that algorithms are not biased. To that end, a Harvard Medical School AI research lab just announced a new initiative, called MAIDA, to develop and share diverse medical image datasets from across the globe. 

According to lab leader Pranav Rajpurkar, assistant professor at Harvard Medical School, the problem they decided to solve is that medical imaging data is rarely shared across institutions due to data security concerns, vendor lock-in and data infrastructure costs. 

In addition, existing data lacks diverse representation. Algorithms for clinical applications are disproportionately trained on a few hospitals, with little to no representation at a national or global level. Populations not adequately represented in the training cohort will likely receive biased results. For example, darker skin is underrepresented in widely used dermatology datasets. 

“There is an urgent need to democratize medical image datasets and ensure diversity in the data that’s being used for data science and AI development,” Rajpurkar told VentureBeat. “The current data that’s in the public domain is, in addition to being a small sliver, it’s a very selective sliver and it’s not diverse and lacking international representation.” 

Around 40 hospitals are already involved in MAIDA’s dataset curation, Rajpurkar said, which is beginning with datasets of chest X-rays, which are the most common imaging exam worldwide. The lab is also working on the development of AI models for other common radiologist tasks — including endotracheal tube placement and pneumonia detection in the emergency room. 

“We expect that MAIDA will be a key ingredient for medical AI and data science, enabling tools to work on more diverse populations than they currently are,” he said.