Inquiry of increasing the job time limit

[yuzc19]

Hi! Thanks for providing the HPC resources!

I would like to know if you can increase the job time limit from 24 to 96 hours because when I pre-train a model, it often takes more than one day to finish the whole experiment. Right now, I need to resume my previous training occasionally, which is really inconvenient. I would really appreciate it if you could increase the slurm job time limit. Thank you!

[koomie]

Hi there. Can you expand on your restart process? Are you able to pick back up where the previous training stopped without incident? Is the inconvenience just related to submitting an additional job? To aid in that, SLURM supports job dependencies so you can submit multiple jobs up from and they will schedule to run sequentially, one after another. I

[yuzc19]

Thank you! I think job dependencies fix the issue for me.