Beyond origin: multimodal AI synthesis to resolve cancers of unknown primary

A revolution in silos: the rise of unimodal predictors

The past decade has witnessed a remarkable, albeit fragmented, revolution in the ability to infer a tumor’s tissue of origin (TOO). AI and machine learning have been the driving force enabling the extraction of subtle, predictive patterns from complex biological data that were previously invisible to human experts.

In genomics, comprehensive profiling has moved beyond identifying single driver mutations to providing rich molecular fingerprints of individual tumors. Gene expression profiling and next-generation sequencing have become foundational. AI models such as CUP-AI-Dx have demonstrated high accuracy in predicting TOO from RNA expression data⁶. More recently, a new generation of deep learning models such as OncoNPC and GDD-ENS has enabled patterns in somatic mutations from clinically standard gene panels to be leveraged to classify tumor types with impressive performance, thus offering a more accessible route to genomic diagnosis^7,8. The emergence of large language models such as OncoChat, trained on vast genomic datasets, has further advanced the frontier of classification and diagnostic support⁹. These tools are beginning to deliver on the promise of molecularly guided therapy. The landmark CUPISCO and Fudan CUP-001 clinical trials have recently demonstrated that using molecular information to guide treatment, through either histology-agnostic targeted therapy or site-specific therapy, results in longer progression-free survival than standard empirical chemotherapy, thus providing definitive evidence of the clinical utility of this approach^4,10,11.

Simultaneously, a parallel revolution has unfolded in pathology. Deep learning models have shown that cellular morphology and tissue architecture within standard images of hematoxylin and eosin staining contain extensive information regarding tumor lineage. Algorithms such as TOAD and TORCH have achieved high accuracy in TOO prediction from pathology images alone, thus demonstrating the untapped diagnostic potential of routine histology^12,13. This capability is being markedly scaled by new pathology foundation models. Systems such as CONCH and PathChat, which integrate visual and language data, can interpret and even discuss complex histopathological features in natural language, thus providing a powerful new interface that augments pathologists’ diagnostic capabilities^14,15.

Radiomics applies AI to extract thousands of quantitative features from medical images such as computed tomography (CT) and positron emission tomography (PET) scans, thereby revealing textural and spatial patterns that correlate with tumor biology and origin. In parallel, liquid biopsies offer a window into tumors through the bloodstream. Analysis of circulating tumor DNA (ctDNA) methylation patterns or fragmentation profiles can provide crucial clues regarding the TOO without a need for tissue biopsy, thus conferring a critical advantage when tissue is scarce or inaccessible^16,17. Multi-analyte tests such as CancerSEEK combine ctDNA analysis with protein biomarkers to enhance both detection and localization¹⁸. In each of these domains, AI is essential for separating the faint, tumor-derived signal from the overwhelming background noise of the body, as exemplified by language model-based tools such as ACID for cell-free DNA (cfDNA) analysis¹⁹.

The need for synthesis: from data points to a biological portrait

Each of these unimodal advances represents a major step forward, yet each operates within the confines of its own data type and has intrinsic limitations. Traditional pathology, the cornerstone of diagnosis, falters with poorly differentiated tumors, in which morphological clues are lost. Genomic profiling, although powerful, requires sufficient high-quality tumor tissue, is costly, and provides no information regarding the tumor’s spatial architecture or macroscopic presentation. Imaging captures this spatial context but lacks molecular depth, whereas liquid biopsies have drawbacks of low signal-to-noise ratios and a complete absence of anatomical information.

In essence, clinicians receive a series of disconnected, often contradictory “eyewitness” accounts of the tumor’s identity. The challenge is not to choose the most reliable witness but to synthesize these accounts into a single, coherent narrative. A cancer’s identity is not encoded solely in its DNA, its morphology, its metabolic activity, or its secreted biomarkers but instead is expressed through the complex interplay of all these features. A specific genomic alteration in a tumor may manifest as a subtle textural pattern on a PET-CT scan, a distinct cellular arrangement on a hematoxylin and eosin slide, or a characteristic methylation signature in the blood. To date, tools to learn and interpret this complex, cross-modal language have been lacking. Therefore, the necessary paradigm shift will lie in moving from unimodal classification to multimodal synthesis. The field must transition from developing isolated AI predictors to building integrated systems that can construct a comprehensive biological portrait of a patient’s cancer from all available data (Table 1).

The dawn of the multimodal AI clinician

This vision is no longer speculative: agentic multimodal AI can integrate pathology, imaging, and molecular data to support evidence-based diagnostic prediction and decision-making. Architectures inspired by models such as Contrastive Language-Image Pre-training (CLIP) have demonstrated remarkable ability to learn shared representations between images and text, thus creating a common language for disparate data types²⁰. This concept is now being extended into the medical domain. Models such as Med-PaLM Multimodal, and the forthcoming Med-Gemini are being trained on vast, diverse datasets of biomedical information, and are learning to connect clinical language, imaging, and genomics within a single, unified framework²¹.

These models represent a fundamental leap beyond their unimodal predecessors. Architecturally, this advancement has been achieved by projecting disparate data types, such as genomic signatures and pathology image embeddings, into a shared latent space, thus allowing the model to learn a common, cross-modal language of cancer biology. Instead of merely classifying, these models synthesize data. A multimodal system for CUP would not just process a pathology image but interpret that image in the context of the patient’s genomic alterations and radiological findings. For example, it could learn that a specific pattern of poorly differentiated adenocarcinoma on a liver biopsy, when combined with a KRAS G12D mutation and a characteristic pattern of fluorodeoxyglucose (FDG) uptake on a PET scan, confers a 95% probability of pancreatic primary cancer. The system would view genomic alterations not as isolated events but as features whose diagnostic significance is modulated by the complete clinical picture. This integrated analysis enables a far more robust and nuanced probabilistic scoring for the tumor’s origin than any single modality could ever achieve. For example, our TORCH model¹³ provides direct clinical evidence for CUP by combining cytological image representations with clinical and molecular features via a transformer-based multimodal attention architecture. In a cohort of 391 patients with CUP, those who received treatment concordant with their TORCH-predicted tissue of origin achieved significantly longer overall survival than discordant cases [median 27 vs. 17 months; hazard ratio (HR) ≈ 0.53]. These findings demonstrated that multimodal synthesis can inform real-world therapeutic decision-making and translate into measurable patient benefit.

The ultimate evolution of this paradigm is the AI agent, an autonomous system that not only integrates data but also actively performs reasoning regarding the data. Powered by large language models and equipped with specialized tools, these agents can navigate complex diagnostic pathways, plan workflows, and even generate novel hypotheses^22,23. An oncologist could engage in a multi-turn dialogue with such an agent, by querying the integrated data to refine diagnostic hypotheses in real time. For example, one could ask, “Given the ctDNA methylation profile and the absence of a lesion on the chest CT, what is the likelihood of a gastro-esophageal primary vs. a biliary primary cancer?” The agent could then access specialized radiology and pathology foundation models such as RadFM or PathChat to provide an evidence-based answer and cite the specific features in each data stream that support its conclusion^15,24,25. This collaborative human-AI interaction moves beyond static predictions to a dynamic process of diagnostic discovery, by augmenting clinicians’ expertise and surpassing the limits of unassisted human cognition.

A new grand challenge: MTI

Translating this transformative potential into clinical practice requires a concerted effort to overcome major hurdles, most notably data scarcity. Training robust multimodal models requires large, high-quality, and ethically sourced datasets that link genomic, pathologic, radiologic, and clinical outcomes data from the same patient. The rarity of CUP makes assembling such cohorts challenging and necessitates multi-institutional and international collaboration. Furthermore, the “black box” nature of complex deep learning models poses a barrier to clinical trust. Further development of Explainable AI (XAI) is necessary to ensure transparency. Finally, the risks of algorithmic bias must be proactively addressed to ensure that these tools are equitable across diverse patient populations.

However, these challenges should not be considered roadblocks but engineering problems to be solved on the path to a much larger vision. The goal should not be merely to build a better tool for predicting a tumor’s origin but to question the primacy of origin itself. We propose that the field should focus on a new grand challenge: the creation and validation of a MTI score. Although multimodal prediction frameworks already exist, such as CUP-AI-Dx, OncoNPC, and more general biomedical models such as Med-Gemini, these systems are ultimately designed to produce a single tissue-of-origin label. In contrast, MTI is conceptualized as a unified latent representation that jointly encodes molecular, morphological, radiologic, cfDNA, and clinical characteristics. Rather than providing only an origin classification, MTI defines a modality-agnostic tumor identity space capable of supporting multiple downstream clinical tasks, including origin inference, therapeutic prioritization, and risk stratification (Figure 2). This AI-generated signature, derived from the synthesis of all available biological data, would represent a holistic, quantitative measure of a tumor’s biological state.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Conceptual distinction between multimodal prediction and MTI. (A) Existing multimodal prediction frameworks integrate features from one or several data types, such as pathology, genomics, radiology, or cfDNA, to generate a single categorical tissue-of-origin label. These systems operate within a supervised classification paradigm and are optimized for origin inference alone. (B) The proposed MTI framework synthesizes pathology, genomic, radiologic, liquid biopsy, and clinical inputs into a unified latent identity representation. Rather than producing only an origin label, this representation is designed to support multiple downstream clinical tasks, including origin inference, therapeutic prioritization, and risk stratification. MTI therefore represents a shift from classification toward integrated tumor identity modeling. AI, artificial intelligence; cfDNA, cell-free DNA; CUP, cancer of unknown primary; MTI, multimodal tumor identity.

For CUP, and eventually other cancers, this MTI could entirely supersede the traditional tissue-of-origin classification. Instead of forcing a tumor into an “anatomical box” that it might no longer resemble, the MTI would define it according to integrated molecular, morphological, and microenvironmental properties. This perspective does not erase the fundamental importance of developmental lineage but instead reframes it as one of many features in a more holistic and clinically actionable biological identity. This concept aligns with emerging clinical evidence from trials such as CUPISCO, wherein histology-agnostic, molecularly guided therapy was found to be superior to standard care, thereby suggesting that a tumor’s actionable biology is more clinically relevant than its origin^10,26. The MTI would serve as a fundamental new classification system to directly guide therapy according to a comprehensive biological profile rather than an inferred anatomical history. Although this long-term vision moves beyond origin-based frameworks, early clinical use of MTI will necessarily be transitional. MTI can function as a secondary classifier that refines origin assessment and highlights cross-modal discordance, and its integrated representations may support therapy prioritization by identifying actionable biological programs even when the primary site remains uncertain. These transitional roles position MTI as an augmentative layer within current CUP workflows that enables clinically compatible, incremental adoption. To operationalize this framework, we outline a concise implementation roadmap including data linkage, model architecture, validation strategy, and clinical deployment (Figure 3). Simultaneously, MTI faces practical limitations including the risk of cross-modal overfitting, biases introduced during data harmonization, and the challenge of ensuring that latent identity representations align with clinically actionable endpoints. Recognizing these constraints will be essential as MTI progresses toward broader clinical integration.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

End-to-end workflow for the development, validation, and clinical deployment of a MTI system. (A) Multimodal data linkage and cohort assembly. Pathology images, genomic sequencing data, radiologic imaging, liquid biopsy assays, and structured clinical variables are linked at the patient level under harmonized metadata standards and quality-control procedures, thus forming integrated CUP/cancer cohorts suitable for model training. (B) MTI model architecture and training. A multimodal encoder (e.g., transformer-based) learns a shared cross-modal latent space and generates an MTI embedding that captures the joint morphological, molecular, radiologic, and clinical characteristics of each tumor. (C) Validation strategy across scales. Model performance is assessed through internal validation (training-validation splits, cross-validation, and site-held-out testing), followed by prospective evaluation and workflow-embedded assessment within defined clinical settings. (D) Clinical deployment and feedback loop. MTI outputs, such as probabilities for tissue of origin, risk profiles, and therapy prioritization, are integrated into a clinical decision support system. Real-world outcomes and performance monitoring enable iterative model refinement and ensure safe, reliable clinical implementation. cfDNA, cell-free DNA; CT, computed tomography; CUP, cancer of unknown primary; EHR, electronic health record; MRI, magnetic resonance imaging; MTI, multimodal tumor identity; NGS, next-generation sequencing; PET, positron emission tomography; WSI, whole slide imaging.

Developing MTI systems across institutions will require privacy-preserving data infrastructures. Federated learning offers a feasible approach by enabling joint model training on distributed datasets while keeping raw patient data local, thereby facilitating multi-center collaboration without compromising confidentiality. Ensuring robustness will also require systematic cross-population generalizability testing, evaluating performance across demographic groups, geographic regions, and institutional workflows to identify and mitigate potential sources of algorithmic bias. Finally, clinical deployment must align with emerging regulatory pathways. Current FDA and EMA guidance for AI/ML-based medical devices emphasizes transparency, reproducibility, and post-deployment monitoring. These considerations will be essential for MTI systems intended to support oncology decision-making.

To achieve this vision, we call for a “CUP Moonshot”: a global, collaborative effort to build the federated datasets and foundational models required to establish and validate the MTI as a new clinical standard.

CUP has long been emblematic of limitations in oncology and a diagnostic dead end. However, by forcing us to look beyond anatomical origin and embrace a more holistic, data-driven definition of cancer, it may paradoxically reveal a path forward. The multimodal AI synthesis required to solve the CUP enigma provides a blueprint for the future of cancer diagnostics, by moving from a series of disconnected tests to a single, unified understanding of a patient’s disease. In resolving the identity of this one enigmatic cancer, tools will be developed to better understand all cancers. Ultimately, clinical encounters will be transformed by replacing diagnoses of exclusion and uncertainty with clear, actionable biological identities that offer patients not only targeted therapies but also a concrete understanding of their diseases.

Author contributions

Conceived and designed the analysis: Xiangchun Li, Hongru Shen.

Wrote the paper: Xiangchun Li, Hongru Shen.