We use machine learning technology to do auto-translation. Click "English" on top navigation bar to check Chinese version.
Large scale AI in digital pathology without the heavy lifting
Pictured: Tissue sample stained with H&E and immunofluorescence (IF) markers side by side.
This is a guest post by Moritz Widmaier, product manager; Martin Shulze, principal consultant for technology and IT; and Florian Leiss, vice president (VP) of product strategy and corporate development, at Ultivue.
Pathology is currently undergoing a transformation. While microscopes still dominate many workflows, digital pathology combined with artificial intelligence (AI) is disrupting the space. Together they unlock the pathway to precision medicine and will be among our strongest weapons to fight cancer. Pathologists have been looking at morphological patterns in patients’ tissue sections highlighted by Hematoxylin and Eosin (H&E) staining for more than a century. They accumulated a comprehensive body of knowledge on how combined insights from tissue and cell morphologies characterize disease and guide treatment selection. AI tools can complement expert assessment with quantitative measurements to enable data-driven medicine. AI companies have formed on the idea that AI can complement the expert assessment of H&E slides by a pathologist.
Tissue analysis in color
With the advent of immune therapy, a more granular characterization of the immune system (and tumor cells) has become necessary to support clinical pathology and diagnostics.
At a high level, our immune system comprises multiple different cell types (T cell, B cell, etc.), which are difficult to distinguish in H&E slides. To complicate things, most of these cell types can be subdivided into a vast number of “specializations” and states of activities that cannot be distinguished from H&E slides anymore. Neither pathologists nor AI can reveal the required information from H&E sections since it was not captured when the tissue section was stained and digitized. Trying to understand the tumor-immuno microenvironment through the eyes of ‘Digital Pathology’ (i.e. only looking at H&E stained sections) can be compared to using black-and-white photography to spot yellow tulips in a red tulip field.
Immunofluorescence is the color photography equivalent of pathology. It allows us to see many more cell phenotypes by adding highly specific staining channels, and with the help of AI, their spatial relationships, or spatial biology.
Ultivue combines expertise in the development of high-quality multiplex immunofluorescence assays and AI-based computational pathology. We aim to deeply characterize immune cells and tumor cell and their spatial relationships in the context of intact tissue, using AI to analyze tissue images at scale. This presents multiple challenges. Our multiplex immunofluorescence images can have five billion pixels and more than five, sometimes 13 channels. These images can get as large as 100 GB. Images at the scale of 10 to 100 GB present a different order of magnitude to what many systems are built for – including systems built to analyze H&E images in pathology. Experts developing these tissue assays and understanding the scientific intricacies in the rich image data mostly aren’t AI engineers and cloud architects. They want algorithms to support the interpretation of the images to generate results quickly, without having to write or read code.
Handling over 10 GB images on Amazon Web Services
Amazon Simple Storage Service (
Figure 1. The Ultivue architecture using Amazon S3, Amazon Web Services Storage Gateway, Amazon SageMaker, and Amazon SageMaker Studio as key components.
For this solution, we use
Amazon SageMaker Studio as a user interface for non-technical users
When handling data at the scale that Ultivue does, the need for utilizing the scalability of cloud services is omnipresent. However, the bio-technology space requires medical and biological experts that cannot be experts in software and IT at the same time. Our user interfaces must simplify the use of highly scalable compute services, such as Amazon SageMaker, and at the same time, offer flexibility to a wide range of users, like AI engineers.
Utilizing custom images and lifecycle configurations allows Ultivue to provide an internal app-store-like experience on top of the highly flexible Amazon SageMaker Studio interface.
Figure 2. Screenshots of the user experience.
In the past, image analysis experts and AI engineers worked with software engineers to integrate algorithms into custom processing environments and connect them to different data sources. Now, processing at scale and data pipelines are reduced to API calls, and image analysis experts can make their algorithms available to pathologists without any intermediary needed.
From pixel to object
Without image analysis, information-rich multi-channel images are just images. Humans, even trained experts, are limited by–in the best case–the presence of only three-color receptors in our retina. Consequently, it becomes increasingly difficult to visually process the information coming from the 5-13 channels that are typically used with Ultivue technology, let alone considering the interactions of potentially dozens, if not a hundred, different cell types across millions of cells present in a tissue section.
AI-based image analysis gives us the opportunity to classify millions of cells and interrogate their frequency and spatial relationships in a data-driven way. This can help find out which specific immune cells need to talk to other immune cells or interact with a subset of tumor cells to predict therapy outcome, and more.
Visit
Learn more about how healthcare and life science organizations use Amazon Web Services at
Pictured: Classified cells overlayed on IF image as image analysis detected them.
Read more about Amazon Web Services for healthcare:
-
How Digithurst and Telepaxx built a secure and scalable radiology solution chain using Amazon Web Services -
AMILI helps advance precision medicine by building microbiome library on Amazon Web Services -
Helping prevent sudden cardiac arrest in young athletes with AI -
How KHUH built a long-term storage solution for medical image data with Amazon Web Services -
Solving medical mysteries in the Amazon Web Services Cloud: Medical data-sharing innovation through the Undiagnosed Diseases Network
The mentioned AWS GenAI Services service names relating to generative AI are only available or previewed in the Global Regions. Amazon Web Services China promotes AWS GenAI Services relating to generative AI solely for China-to-global business purposes and/or advanced technology introduction.