In the past years, advancements in artificial intelligence (AI) have had a significant influence on many aspects of our lives. It is especially tempting to use AI for the study of histopathological tissue sections, which requires highly trained pathologists to evaluate pictures with a gigapixel resolution or more. In the context of rare cancers, the rarity of cases may lead to diagnosis uncertainty, delay, or even misdiagnosis. AI has the potential to make diagnostic procedures more effective, repeatable, and reliable.

The aim of this transversal project is to translate our multi-omics tumor profiling into the clinical setting without the need to generate costly and complex-to-analyze molecular data. To do this, we are exploring how different deep-learning computer vision algorithms can detect morphological features that pathologists will be able to recognize, with the ultimate goal to improve the diagnosis and treatment of rare cancers.

Anomaly detection deep learning algorithms are a type of unsupervised machine learning algorithm that are used to identify patterns in images that are unusual or unexpected. They do not require any prior knowledge of what an anomaly is, and simply need to be trained on “normal” images. This “normal” class can be defined in different ways, e.g. good prognosis patients, a specific molecular group etc. allowing for the identification of discriminating regions for tumor proliferation or aggressiveness (see Figure below). 

We are also developing deep learning algorithms to detect mitosis and high proliferation areas based on immunohistochemistry whole slide images. Computer vision algorithms on whole slide images (WSI) used together with molecular data can uncover morphological features with the potential to reconcile discordant molecular and histopathological classifications.  

This technology has the potential to simplify and enrich clinical decision-making processes, but requires rigorous external validation in clinical settings. For this purpose, the Rare Cancers Genomics initiative maintains a rich network of collaborators (ENETS, EPIC, MESOBANK, EURACAN).

Unsupervised anomaly detection pipeline for whole-slide images of lung carcinoids (WSI): A) Preprocessing steps. A.1) Tiling of the WSI. A.2) Vahadane color normalization (Vahadane et al. 2016). B) Deep learning method for anomaly detection. B.1) Unsupervised anomaly detection model C-Flow via normalization flow (Gudovskiy et al. 2022). B.2) Pixel-level tiled anomaly map. C) Preliminary results. C.1) Segmentation of WSI tumors, via a C-Flow model trained on tumor images. The bluer a tile is, the more it falls outside the learned distribution, and thus the more it is non-tumoral. C.2) Map of WSI abnormalities at the pixel scale, obtained via a model trained on typical carcinoid (TC) samples. The more abnormal a pixel is, i.e. reddish, the more it denotes a deviation from the TC distribution. C.3) Aggregate abnormality score by patients, correlated to their “molecular aggressiveness” defined using multi-omics data.

Assessment of the current and emerging criteria for the histopathological classification of lung neuroendocrine tumours in the lungNENomics project. Mathian E, Drouet Y, Sexton-Oates, Papotti MG, Pelosi G, Vignaud J-M, Brcic L, Mansuet-Lupo A, Damiola F, Altun C, Berthet J-P, Blanc Fournier C, Brustugun OT, Centonze G, Chalabreysse L, De Montpréville VT, Di Micco CM, Fadel E, Gadot N, Graziano P, Hofman P, Hofman V, Lacomme S, Lund-Iversen M, Mangiante L, Milione M, Muscarella LA, Perrin C, Planchard G, Popper H, Rousseau N, Roz L, Sabella G, Tabone-Eglinger S, Voegele C, Volante M, Walter T, Dingemans A-M, Moonen L, Jan Speel E, Derks J, Girard N, Chen L, Alcala N, Fernandez-Cuesta L, Sylvie Lantuejoul S, Foll M. ESMO Open. Accepted

HaloAE: A Local Transformer Auto-Encoder for Anomaly Detection and Localization Based on HaloNet. E Mathian, H Liu, L Fernandez-Cuesta, D Samaras, M Foll, L Chen. Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) – Volume 5: VISAPP; ISBN 978-989-758-634-7; ISSN 2184-4321, SciTePress, pages 325-337. DOI: 10.5220/0011865900003417

Members of the Rare Cancers Genomics Team at IARC have participated as invited speaker, invited discussant, or been selected for oral presentations in the following international renown conferences:

– 18th International Conference on Computer Vision Theory and Applications VISAPP