DISCLAIMER: This code is provided for research and development use only. This code is not intended for use in clinical decision-making or for any other clinical use and the performance of the code for clinical use has not been established.
This repository contains code and instructions to recreate the medical imaging AI pipeline demo shown by Microsoft HLS team at RSNA 2021.
As you navigate the demo, you will be playing several roles in a team that is aiming to build and deploy a chest radiograph classification system using cloud-based tools available in Microsoft Azure.
This demo will walk you through the steps that a data science team would typically undertake and describe the use of Microsoft tools as well as provide some custom code to take you through the steps. We chose a chest X-ray classification scenario since it is a well studied problem for which many great datasets are available. The focus of this demo is not on the algorithms used to build the best performing system, but rather the steps and tools that are needed to get one there. The same tools and principles could be applied to many other types of medical imaging datasets.
Note: In the real world you will likely deal with DICOM data. A follow-up to this demo will cover working with DICOM and the various aspects of handling real-world clinical data. In this demo we use real medical images converted to 16bit grayscale PNGs for simplicity.
The following diagram represents the steps we will be modeling:
- Azure account. Since this demo relies on the use of Azure cloud, you will need to create an account.
- Azure ML workspace. Azure Machine Learning is one of the many services provided on the Azure platform. A Workspace is a separate portal provided by the service with which you will be interacting through the demo. Azure ML Workspace provides controls that allow interacting with compute resources, writing code in shared Notebooks, annotating datasets, managing datasets and more. Once you create your Azure account, you will need to create the workspace. Follow instructions for creating a workspace on the Azure ML Documentation page
With that, you are ready to dive into the data science!
The next steps include:
- Loading Data into the cloud
Here we will load a sample dataset into Azure Blob Storage and register it with Azure Machine Learning for further analysis - Annotate the dataset
Here, we cover how to annotate your medical imaging dataset using tools built into the Azure ML Studio. Note that in this demo, the actual training will use the labels available with our sample dataset so that you won't have to manually label all the images. But we will show you how to leverage existing labels and those that you create through the annotation tool. - Build a model from the annotated dataset
In this portion we will show you how to set up your compute resources and then use Notebooks feature of the Azure ML Studio to submit an experiment, resulting in a machine learning model that is capable of analyzing XRay images. - Deploy the model as a RESTful endpoint and perform explainability analysis
Finally, we will demonstrate how to deploy a model so that it's ready for inferencing, and then how to use this model to gain insights into how the model is making predictions.
If you are looking for advanced algorithms for medical image analysis that harness the power of Azure-based cloud computing - make sure to check out the amazing work done by the InnerEye team from Microsoft Research Cambridge and their open-source repositories:
- InnerEye-DeepLearning for advanced medical image analysis algorithms
- HI-ML - building blocks for AI/ML scenarios for medical applications:
A real world medical imaging AI setup would leverage DICOM data and would benefit from storing DICOM images in the cloud. You can explore the services that Microsoft provides to manage DICOM and FHIR data in the cloud:
As mentioned, this tutorial uses PNG images instead of DICOM images, however, everything in this tutorial can be done with DICOM images. We are planning to amend this tutorial to use DICOM images, in the meantime please contact us if you are having any issues using DICOM images.
Real world scenarios often come with datasets that are partially labeled, or weakly labeled, and need to be reviewed. Azure ML provides ability to import labels through the API. We are planning to provide a tutorial on the best practices to import medical imaging labels, in the meantime contact us to learn more if you are interested.
We are working on an advanced annotation tool that provides more ways of annotating medical imaging datasets. Contact our team to learn more.
Here are some learning materials if you would like to explore some of the Microsoft's AI tools further:
- A 30 day challenge focusing on learning AI fundamentals
- Interactive introduction into Azure ML
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repositories using our CLA.
These are the ways to contribute:
- Submit bugs and help us verify fixes as they are checked in.
- Review the source code changes.
Contact our team at medimagingaiinfo@microsoft.com for any additional questions.