Training very slow

[hungk64it1x]

Hi minivision team,

I clone your repo and I did not change anything from the original code, I only modified image folders for my own dataset.
My dataset has about 240k images and I trained with image size 80x80.
But I found in the training the loader for dataset was very slowly and it mostly didn't use GPU core for training.
Have any idea about this issue?

Thank you.

[zhuyingSeu]

If you have one gpu, python train.py --device_ids 0.

[amilkcar]

Hi @hungk64it1x , do you have an example on how to crop and resize the images... i have like 10k images but i didnt understand how to crop and resize the original images

[chau25102001]

Hi minivision team,

I clone your repo and I did not change anything from the original code, I only modified image folders for my own dataset. My dataset has about 240k images and I trained with image size 80x80. But I found in the training the loader for dataset was very slowly and it mostly didn't use GPU core for training. Have any idea about this issue?

Thank you.

It's probably due to the time cost for reading large images and generating FT images during training. Maybe you should consider resize your images and generate/save FT images before hand, then read them in getitem() function of the dataset instead of generating FT on the fly. Hope this helps!