A multimodal foundation model for molecular and morphologic data in oncology

Nature
Read full story· 20 min
Researchers developed THREADS, a multimodal foundation model integrating molecular and morphological data to improve oncology analysis. The model was evaluated using multiple publicly available cancer imaging and transcriptomics datasets.

What happened

A team of researchers created THREADS, a foundation model that combines molecular and morphological data to assist with cancer analysis. They tested THREADS using a range of publicly available datasets that include both cancer imaging and transcriptomics information. The model and its code have been made accessible online for further use and evaluation by the scientific community.

Key details

  • THREADS integrates both molecular (such as transcriptomics) and morphological (imaging) data.
  • Evaluation was performed using datasets from sources like TCGA, GTEx, and multiple cancer imaging collections.
  • The model code is available on GitHub, and model weights are hosted on HuggingFace.
  • Preprocessing tools and benchmark code are also publicly accessible.
  • Curated labels and data splits used in the study are available through HuggingFace.

Why it matters

Researchers, clinicians, and developers in oncology and computational pathology now have access to a new tool that can analyze both genetic and imaging data together. This enables more comprehensive analysis of cancer datasets, potentially improving research workflows and diagnostic approaches. The public release of code, data, and benchmarks allows others in the field to validate, adapt, or build upon the model for their own cancer studies.


More in Healthcare