EfficientNet-b0 training

[sefibk]

Thanks for the amazing repo!
I am trying to train Enet-b0 according to your recipe:
./distributed_train.sh 2 /imagenet/ --model efficientnet_b0 -b 384 --sched step --epochs 450 --decay-epochs 2.4 --decay-rate .97 --opt rmsproptf --opt-eps .001 -j 8 --warmup-lr 1e-6 --weight-decay 1e-5 --drop 0.2 --drop-connect 0.2 --model-ema --model-ema-decay 0.9999 --aa rand-m9-mstd0.5 --remode pixel --reprob 0.2 --amp --lr .048
However:

1. Batch size 384 doesn't fit on V100x4 GPUs. How can I follow the recipe anyway?
2. When I run with -b 320 it seems (from the summary.csv) the accuracy is not improving. Is there a TensorBoard I can view to try and follow the training and debug?

[rwightman]

just rescale your learning rate when you change the batchsize ( .048 / (2 * 384)) * 4 * 320.

with ema enabled the EMA validation scores will be lagged by many epochs depending on the constant, the logging output shows the ema and non-ema output so you can verify non-ema numbers are improving... but csv only records the ema when enabled

[sefibk]

Thanks a lot!!