minimum GPU memory required for running vision_transformer

[amiltonwong]

Hi, authors,

What are the minimum GPU memory required for running vision_transformer during inference and training, respectively?

[andsteing]

The flag settings in the README were tested with a host having 8 V100 GPUs connected, so that would be for 16G of RAM. If you have fewer GPUs or less memory, you can adapt the --accum_steps flag accordingly (e.g. --accum_steps=$((8*16)) if you have a single GPU with 16G of RAM). For inference you'll need less RAM, so you can try to decrease the --accum_steps flag until you run out of memory. Note that you can also change the --batch_size flag, but for training you will then also need to find a new value for the learning rate.