From Tissue Image to Transcriptomics: Dual Modality Deep Learning Model for Classification of Cancer Using Histopathological Images and Gene Expression Pattern

Abstract

A primary method for cancer detection involves the examination of histopathological images. However, traditional approaches to analyze these images are time-consuming and prone to errors. With recent advancements in deep learning, researchers are increasingly leveraging these models to enhance the accuracy and efficiency of histopathological image analysis. In this study, a deep learning-based model is proposed for multi-class cancer classification that integrates gene expression prediction with histopathological image analysis to enhance diagnostic precision and streamline the detection process. The model is intended for use in real-world clinical settings where it is a challenge to classify data correctly because predefined sequences of gene expression and cancer labels are often missing. Fourier Neural Networks (FNN) and EfficientNetB0 are utilized to get a full set of spatial and frequency-based features from Whole Slide Images (WSIs). To select the best set of features, Incremental Principal Component Analysis (IPCA) was used, and the resulting representations were subsequently reconstructed from patch-level features to WSI representations. A DeepONet model, one of the advanced deep learning models, was selected for mapping the generated histopathological image features to predict gene expression patterns. After training, the model achieved 93% accuracy in classifying cancer types with a 0.92 precision value indicating acceptable performance with respect to multiple cancer classifications. Additionally, the model achieved strong performance in gene expression prediction, with a Mean Absolute Error (MAE) of 0.033 and an R² score of 0.765, demonstrating its reliability in capturing gene expression patterns. Addressing real-world challenges such as missing gene expression data or ambiguous cancer-type classifications, the proposed Dual Deep Learning Model enhances cancer diagnosis by improving accuracy in oncology. By integrating histopathological image analysis with gene expression prediction, the model enables automated clinical decision-making, offering a robust solution for distinguishing between cancerous and non-cancerous cases.

Received: 11 June 2025 | Revised: 15 August 2025 | Accepted: 13 September 2025

Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.

Data Availability Statement

Data are available from the corresponding author upon reasonable request.

Author Contribution Statement

Anju Das: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Visualization. Neelima Nizampatnam: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft. Somnath Ganguly: Investigation, Data curation, Writing – review & editing, Visualization, Supervision, Project administration. Joon Ho Choi: Validation, Investigation, Writing – review & editing, Visualization, Supervision, Project administration.