This repository contains the code for Self Supervised Multi-View Graph Representation Learning in Digital Pathology submitted to GRAIL 2023. Please refer to the paper for more details.
The shortage of annotated data in digital pathology presents a significant challenge for training GNNs. Inspired by pathologists who take multiple views of a histology slide under a microscope for exhaustive analysis, this project performs graph representation learning using self-supervision. The methodology leverages multiple graph views constructed from a given histology image to capture diverse information by maximizing mutual information across nodes and graph representations of different graph views, resulting in a comprehensive graph representation. The trained graph encoder weights for Feature Extractor (SimCLR),Infograph Cell, Infograph Patch and MultiView Graph is shared at this link.
opencv
dgl (only for farthest sampling method used while building cell graph)
torch-geometric
trainer - https://github.com/Vishwesh4/TrainerCode
torchmetrics
wandb
networkx
sklearn
torchvision
The experiments was performed on BRACS dataset previous version downloaded from the website.
For performing training on histology images using the proposed methodology, the images were converted into cell graph and patch graph.
For generating feature embedding for the nodes for both the graphs, Resnet34 was trained on BRACS training dataset using SimCLR. The model weight is shared in the same drive link.
For generating cell graph, the cells were identified using hovernet using panuke pretrained model link. The generated .mat files and histology images were used for generating cell graph using
python utils/build_cellgraph_full.py
For generating patch graph use
python utils/build_patchgraph_full.py
The infograph model for both cell graph and patch graph can be trained using infomax using
python train_infograph.py -c ./configs/infograph_[cell/patch].yml -s [SEED]
The training hyperparameters can be directly modifed from the config files present in the config folder
The cell and patch graph encoders can be trained using same-view and cross-view mutual information maximization using
python train_multiview.py -c ./configs/multiview.yml -s [SEED]
The trained weights can be used for transfer learning like shown in ./test_bracs.py
- Vishwesh Ramanathan (@Vishwesh4)
If you want to contact, you can reach the authors by raising an issue or email at vishwesh.ramanathan@mail.utoronto.ca
- Code was inspired from Infograph repository and Pytorch Geometric repository