Skip to content

Latest commit

 

History

29 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GigaTIME: Multimodal AI generates virtual population for tumor microenvironment modeling (Cell)

Paper Model License PyTorch Microsoft

Official implementation of GigaTIME

📄 Paper🤗 Model CardAzure AI Foundry

⚡ New: GigaTIME-Flash

GigaTIME-Flash is built on top of GigaPath-Flash and delivers better prediction quality, 6× faster inference, and 8× less GPU memory than the original GigaTIME.

➡️ Model card: 🤗 prov-gigatime/GigaTIME-flash

➡️ Try it: scripts/gigatime_flash_testing.ipynb

➡️ Whole-slide inference: tile a full TCGA slide, run GigaTIME-Flash, and stitch a slide-level virtual mIF map across all channels — scripts/gigatime_flash_tcga_wsi_inference.ipynb

Model Family

Model Description Paper License
GigaPath Whole-slide foundation model Nature (2024) Apache-2.0
GigaPath-Flash Efficient whole-slide foundation model arXiv (2026) Apache-2.0
GigaTIME Spatial proteomics prediction from H&E Cell (2026) Apache-2.0
GigaTIME-Flash Efficient spatial proteomics prediction from H&E arXiv (2026) Apache-2.0

Environment Setup

We recommend using Conda for environment management. The codebase has been tested with Python 3.11 using A100 GPUs for optimal reproducibility. Before creating the environment, ensure that the torch version specified in environment.yml matches your GPU and CUDA driver setup.

To set up the environment, run:

conda env create -f environment.yml

This will create a Conda environment named gigatime. Activate it with:

conda activate gigatime

Data

A set of 50 paired H&E and mIF patches from the test set is available for evaluation. Download the sample data from Dropbox.

After downloading, unzip the folder and place it in the data directory:

unzip sample_test_data.zip -d ./data/

Make sure the extracted folder are located in ./data/.

Pre-trained Model

Model card available in HuggingFace. GigaTIME is also available in Azure AI Foundry.

You need to agree to the terms to access the models. Once you have the necessary access, set your HuggingFace read-only token as an environment variable:

export HF_TOKEN=<huggingface read-only token>

If you don’t set the token, you might encounter the following error:

ValueError: We have no connection or you passed local_files_only, so force_download is not an accepted option.

Once that is done, you can load your model like this:

from huggingface_hub import snapshot_download
import torch

repo_id = "prov-gigatime/GigaTIME"
local_dir = snapshot_download(repo_id=repo_id)

weights_path = os.path.join(local_dir, "model.pth")
state_dict = torch.load(weights_path, map_location="cpu")
model.load_state_dict(state_dict)

Tutorials

  • Inference Tutorial:

Learn how to load the model and run predictions on sample patches: scripts/gigatime_testing.ipynb

  • ⚡ GigaTIME-Flash Inference Tutorial:

Run inference with the new, efficient GigaTIME-Flash model (6× faster, 8× less GPU memory, and higher prediction quality than the original CNN-based GigaTIME): scripts/gigatime_flash_testing.ipynb

  • Training Tutorial:

Understand the training workflow with a one-epoch demo: scripts/gigatime_training.ipynb

Training GigaTIME cross-modal translator

We also release the script needed to train the GigaTIME model here.

To train the model:

python scripts/db_train.py --arch gigatime   --tiling_dir "gigatime_training_path"  --window_size 256       --batch_size 32     --sampling_prob 1     --name GigaTIME_model    --output_dir "Output_Directory"    --epoch 300 --input_h 512 --input_w 512 --lr 0.001 --loss BCEDiceLoss --val_sampling_prob 1 --num_workers 12 --gpu_ids 0 1 2 3 4 5 6 7 --crop True --metadata "Gigatime metadata file"

Model Uses

Intended Use

The data, code, and model checkpoints are intended to be used solely for (I) future research on pathology AI models and (II) reproducibility of the experimental results reported in the reference paper. The data, code, and model checkpoints are not intended to be used in clinical care or for any clinical decision-making purposes.

Primary Intended Use

The primary intended use is to support AI researchers reproducing and building on top of this work. GigaTIME should be helpful for generating virtual mIF profiles from routine H&E pathology slides.

Out-of-Scope Use

Any deployed use case of the model --- commercial or otherwise --- is out of scope. Although we evaluated the models using a broad set of publicly-available research benchmarks, the models and evaluations are intended for research use only and not intended for deployed use cases.

License Notice

The model is not intended or made available for clinical use as a medical device, clinical support, diagnostic tool, or other technology intended to be used in the diagnosis, cure, mitigation, treatment, or prevention of disease or other conditions. The model is not designed or intended to be a substitute for professional medical advice, diagnosis, treatment, or judgment and should not be used as such. All users are responsible for reviewing the output of the developed model to determine whether the model meets the user’s needs and for validating and evaluating the model before any clinical use.

Citation

@article{valanarasu2025multimodal,
  title={Multimodal AI generates virtual population for tumor microenvironment modeling},
  author={Valanarasu, Jeya Maria Jose and Xu, Hanwen and Usuyama, Naoto and Kim, Chanwoo and Wong, Cliff and Argaw, Peniel and Shimol, Racheli Ben and Crabtree, Angela and Matlock, Kevin and Bartlett, Alexandra Q and others},
  journal={Cell},
  year={2025},
  publisher={Elsevier}
}

@article{usuyama2026gigapathflash,
  title={{GigaPath-Flash} and {GigaTIME-Flash}: Efficient Pathology Foundation Models for Whole-Slide and Tumor Microenvironment Analysis},
  author={Usuyama, Naoto and Valanarasu, Jeya Maria Jose and Yao, Sicong and Xu, Hanwen and Bagga, Jaspreet and Qin, Guanghui and Kramer, Robert E. and Wong, Cliff and Lee, Soohee and Qiu, Hao and Zhao, Theodore Zhengde and Ben Shimol, Racheli and Crabtree, Angela and Matlock, Kevin and Lozano Garcia, Eduardo Alejandro and Sangani, Naiteek and Santamaria-Pang, Alberto and Rokuss, Maximilian and Hasija, Yashna and Patel, Naisargi Manishkumar and Entenmann, Jason and Bartlett, Alexandra Q. and Wright, Bill J. and Fox, Bernard A. and Piening, Brian and Zhang, Sheng and Wang, Sheng and Naumann, Tristan and Bifulco, Carlo and Poon, Hoifung},
  journal={arXiv preprint arXiv:2607.18218},
  year={2026},
  doi={10.48550/arXiv.2607.18218},
  url={https://arxiv.org/abs/2607.18218}
}

About

GigaTIME: Multimodal AI generates virtual population for tumor microenvironment modeling (Cell)

Topics

Resources

Stars

404 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors

Languages