Memory Issues

[HYLPRC]

Hi, thank you for your excellent work and for open-sourcing the BioPathNet code!

I am currently trying to reproduce the drug repositioning experiments on the PrimeKG dataset. However, I encountered some GPU memory issues when training the model.

I would like to ask about the experimental setup used in your paper:

What hardware configuration did you use for training on PrimeKG (e.g., GPU type and memory)?
What batch size and number of negative samples were used in your experiments?
Did you apply any specific strategies to reduce memory consumption (e.g., gradient accumulation, subgraph sampling, etc.)?

I am currently using a single RTX 2080 (around 10GB GPU memory), and it seems difficult to run the model with the default configuration. I would really appreciate it if you could share some guidance on suitable settings for limited GPU resources.

Thanks again for your great work!

[emyyue]

Hi,

yeah indeed the PrimeKG graph is really big. The config is as uploaded https://github.com/emyyue/BioPathNet/tree/b051a9003a1363b5655cdb7dfdad60609c0e4fe3/config/primekg, e.g. batch_size of 16 and num_negative 64
and here is an excerpt from the sbatch script:

#!/bin/bash
#SBATCH --job-name=primekg
#SBATCH --output=./slurm_out/cp_%j.txt
#SBATCH --error=./slurm_out/cp_%j.txt
#SBATCH --time=12:00:00
#SBATCH --mem=64Gb
#SBATCH -c 4
#SBATCH --gres=gpu:1
#SBATCH --constraint=a100_80gb

So as you can see, we needed 80 Gb to fit the whole graph into memory...

Perhaps this can also help: gradient_interval to be inserted alongside of batch_size performs a gradient update every n batches.

https://github.com/DeepGraphLearning/torchdrug/blob/6066fbd82360abb5f270cba1eca560af01b8cc90/torchdrug/core/engine.py#L53

Best

[HYLPRC]

Hi,

Thank you very much for your detailed reply and for sharing the configuration and sbatch script. This is extremely helpful, especially the note about the 80GB memory requirement on PrimeKG.

I also appreciate the suggestion regarding gradient_interval. I’ll explore that as well.

I’ll take some time to investigate how to reduce the memory and computational overhead, possibly through more efficient batching or pruning strategies on my side.

Thanks again for your help!

Best regards