Predicting Endometrial Cancer Subtypes and Molecular Features from Histopathology Images Using Multi-resolution Deep Learning Models

Abstract

Determining endometrial carcinoma histological subtype is a critical diagnostic process that directly affects prognosis and treatment options. Recently, molecular subtyping and mutation status are gaining popularity as they offer more relevant information to evaluate the severity and develop individualized therapies. However, compared to the histopathological approach, the availability of molecular subtyping is limited as it can only be obtained by genomic sequencing, which is more expensive. Here, we implemented deep convolutional neural network models that predict not only the histological subtypes, but also molecular subtypes and 18 common gene mutations based on digitized H&E stained pathological images. Taking advantage of the multiresolution nature of the whole slide images, we introduced a customized architecture, Panoptes, to integrate features of different magnification. The model was trained and evaluated with images from The Cancer Genome Atlas and Clinical Proteomic Tumor Analysis Consortium. Our models achieved an area under the receiver operating characteristic curve (AUROC) of 0.969 in predicting histological subtype and 0.934 to 0.958 in predicting CNV-H molecular subtype. The prediction tasks of 4 mutations and MSI-High molecular subtype also achieved high performance with AUROC ranging from 0.781 to 0.873. Panoptes showed significantly better performance than InceptionResnet in most of these top predicted tasks by up to 18%. Feature extraction and visualization revealed that the model relied on human-interpretable patterns. Our results suggest that Panoptes can help pathologists determine molecular subtypes and mutations without sequencing, and is generally applicable to any cancer type.

Artificial Intelligence for Histology-Based Detection of Microsatellite Instability and Prediction of Response to Immunotherapy in Colorectal Cancer

“…While the complex architecture showed strong performance in predicting many histological and molecular features, MSI was best predicted by the existing InceptionResnetV1 architecture, with an AUC of 0.827 (Table 2), which outperformed Kather's previously described ResNet-18 architecture (AUC 0.75). The inclusion of clinical data did not seem to improve the model's performance: when the age and BMI of the patient were added into the model, its performance did not significantly improve [71]. Predicted MSI was correlated with certain histological features, including intratumoral and peritumoral lymphocytic infiltrates.…”

“…However, given that TIL density can vary across tumor area, this study using surgical specimens likely yielded a greater AUC than would be achieved with smaller biopsy specimens, such as those typically available from sites of metastasis. (64)(65)(66)(67)(68)(69)(70)(71)(72)(73)(74)(75)(76)(77)(78) [27] 82 (80)(81)(82)(83)(84)(85) [27] 9.1 (5.9-14.1) [27] 21 [30] 97 [30] 9.8 (3.5-28.5) [30] 60 [36] 78 (76)(77)(78)(79) (48)(49)(50)(51)(52)(53)(54)(55)(56)(57)(58)(59)(60)(61)(62)(63) [27] 77 (74)(75)(76)…”

“…Focusing on endometrial cancer, a recent study available as a preprint generated CNNs that had three branches of an InceptionResNet architecture, each analyzing tiles at a different resolution [71]. An optional fully connected layer incorporating clinical features was also evaluated as a fourth branch.…”

“…Specific somatic mutations may affect tumor response to ICIs. Previous success in predicting genomic data from histology [58,60,71] suggests that it would be possible to perform for other genes, but validation for individual genes would be required. For example, mutations in the DNA polymerase epsilon (POLE) gene, which codes for an enzyme involved in DNA proofreading during replication, can lead to a very high mutational burden without MSI [104].…”

Using Machine Learning Algorithms to Predict Immunotherapy Response in Patients with Advanced Melanoma

“…22). First, we performed class activation mapping (CAM) to identify regions within each tile that the neural network uses to generate predictions (23,24). To do this, we analyzed a set of tiles that were classified as POD with high probability (POD probability above 0.75; 136,109 "POD" tiles) and another set of tiles classified as Response with high probability (POD probability below 0.25; 51,220 "Response" tiles).…”