EfficientNetV2 self tested imagenet accuracy

[leondgarse]

• Just showing how different parameters affecting model accuracy.

• Init tfds imagenet2012 following Tensorflow imagenet2012.

• Create model using Official efficientnetv2 publication or this one is no difference.

  # Official EfficientNetV2S
  sys.path.append('../automl/efficientnetv2/')
  import datasets as orign_datasets
  import effnetv2_model as orign_effnetv2_model
  
  dataset = "imagenetft"
  model_type = "s"
  load_weights = "imagenet21k-ft1k"
  cc = orign_datasets.get_dataset_config(dataset)
  if cc.get("model", None):
      cc.model.num_classes = cc.data.num_classes
  else:
      cc['model'] = None
  model = orign_effnetv2_model.get_model('efficientnetv2-{}'.format(model_type), model_config=cc.model, weights=load_weights)

  # This one
  from keras_cv_attention_models import efficientnet
  model = efficientnet.EfficientNetV2S(pretrained='imagenet21k-ft1k')

• Run testing.

  • rescale_mode torch means (image - [0.485, 0.456, 0.406]) / [[0.229, 0.224, 0.225]], tf means (image - 0.5) / 0.5. [128, 128] means mean=128, std=128 -> (image in (0, 255) - 128) / 128.
  • central_crop=-1 means resize image directly without crop.

  from keras_cv_attention_models import efficientnet, imagenet
  model = efficientnet.EfficientNetV2S(pretrained='imagenet21k-ft1k')
  imagenet.evaluation(model, rescale_mode=[128, 128], resize_method="bicubic", antialias=True, central_crop=-1)
  # >>>> Load pretrained from: /home/tdtest/.keras/models/efficientnetv2/efficientnetv2-s-21k-ft1k.h5
  # Evaluating: 100%|█████████████████████████████████████████████████| 782/782 [07:52<00:00,  1.66it/s]
  # >>>> Accuracy top1: 0.84804 top5: 0.97514

• Small model ImageNet pretained results

  model	Input Resolution	rescale_mode	resize_method	central_crop	top 1	top 5
  EffV2B0	224	tf	bilinear	0.875	0.75304	0.92606
  EffV2B0	224	tf	bicubic	0.875	0.7581	0.92858
  EffV2B0	224	torch	bilinear	0.875	0.78594	0.94262
  EffV2B0	224	torch	bicubic	0.875	0.78748	0.94386
  EffV2B0	224	torch	bicubic antialias	0.875	0.7833	0.94038
  EffV2B0	224	torch	bicubic	0.87	0.787	0.9436
  EffV2B0	224	torch	bicubic	0.95	0.78732	0.94412
  EffV2B0	224	torch	bicubic antialias	0.95	0.78288	0.93998
  EffV2B0	224	torch	bicubic	1.0	0.78536	0.94404

  model	Input Resolution	rescale_mode	resize_method	central_crop	top 1	top 5
  EffV2B1	240	torch	bicubic	0.87	0.79722	0.94938
  EffV2B1	240	torch	bicubic	240/272 (0.882)	0.79788	0.94924
  EffV2B1	240	torch	bicubic	0.95	0.7987	0.94936
  EffV2B2	260	torch	bicubic	260/292 (0.890)	0.80428	0.95184
  EffV2B2	260	torch	bicubic	0.95	0.80642	0.95262
  EffV2B3	300	torch	bicubic	300/332 (0.9036)	0.81974	0.95818
  EffV2B3	300	torch	bicubic	0.95	0.82098	0.95896
  EffV2B3	300	torch	bicubic	1.0	0.82	0.9587
  EffV2T	288	torch	bicubic	0.99	0.82186	0.96112
  EffV2T	288	torch	bicubic antialias	0.99	0.82324	0.96114
  EffV2T	288	torch	bicubic antialias	1.0	0.82272	0.96206
  EffV2T	288	torch	bicubic antialias	0.95	0.82338	0.96072
  EffV2T	320	torch	bicubic antialias	0.99	0.82588	0.96238
  EffV2T	320	torch	bicubic antialias	0.95	0.82594	0.962
  EffV2T_GC	288	torch	bicubic antialias	0.95	0.82404	0.96236
  EffV2T_GC	288	torch	bicubic antialias	0.99	0.82394	0.96282
  EffV2T_GC	288	torch	bicubic antialias	1.0	0.82458	0.96306
  EffV2T_GC	320	torch	bicubic antialias	1.0	0.82676	0.96372

• Small model ImageNet21k pretained results

  model	Input Resolution	rescale_mode	resize_method	central_crop	top 1	top 5
  EffV2B2,21k	260	torch	bicubic	0.95	0.19478	0.31922
  EffV2B2,21k	260	tf	bicubic	0.95	0.79118	0.94922
  EffV2B2,21k	260	tf	bicubic antialias	0.95	0.79254	0.94992
  EffV2B2,21k	260	tf	bicubic antialias	0.99	0.79482	0.95022
  EffV2B1,21k	240	tf	bicubic antialias	0.99	0.79034	0.94978
  EffV2B1,21k	240	tf	bicubic antialias	0.87	0.7819	0.9436
  EffV2B0,21k	224	tf	bicubic antialias	0.95	0.77418	0.94134
  EffV2B0,21k	224	tf	bicubic antialias	0.875	0.76786	0.93682
  EffV2B0,21k	224	tf	bicubic antialias	0.99	0.7755	0.94208
  EffV2B3,21k	300	tf	bicubic antialias	0.99	0.82462	0.96524

• Large model results

  model	Input Resolution	rescale_mode	resize_method	central_crop	top 1	top 5
  EffV2S	384	tf	bicubic	0.95	0.83788	0.96602
  EffV2S	384	torch	bicubic	0.95	0.29858	0.46904
  EffV2S	384	tf	bicubic	0.99	0.8386	0.967
  EffV2S	384	tf	bicubic antialias	0.99	0.8387	0.967
  EffV2S	384	tf	bicubic antialias	-1	0.83764	0.96652
  EffV2S	384	[128, 128]	bicubic antialias	0.99	0.8389	0.96712
  EffV2S	384	[128, 128]	bicubic antialias	1.0	0.83892	0.96726
  EffV2S ft1k	384	tf	bicubic	0.94	0.84006	0.97118
  EffV2S ft1k	384	torch	bicubic	0.94	0.13104	0.21656
  EffV2S ft1k	384	tf	bicubic	0.95	0.84076	0.97134
  EffV2S ft1k	384	tf	bicubic	0.96	0.8417	0.97176
  EffV2S ft1k	384	tf	bicubic	0.97	0.84188	0.972
  EffV2S ft1k	384	tf	bicubic	0.98	0.84302	0.9727
  EffV2S ft1k	384	tf	bicubic	0.99	0.84328	0.97254
  EffV2S ft1k	384	tf	bicubic antialias	0.99	0.84336	0.97256
  EffV2S ft1k	384	tf	bicubic	1.0	0.84312	0.97292
  EffV2S ft1k	384	tf	bicubic antialias	1.0	0.84348	0.97288
  EffV2S ft1k	384	tf	bicubic	-1	0.84794	0.97494
  EffV2S ft1k	384	[128, 128]	bicubic	-1	0.84796	0.9751
  EffV2S ft1k	384	tf	bicubic antialias	-1	0.84802	0.97504
  EffV2S ft1k	384	[128, 128]	bicubic antialias	-1	0.84804	0.97514

  model	Input Resolution	rescale_mode	resize_method	central_crop	top 1	top 5
  EffV2M	480	tf	bicubic	0.99	0.8509	0.973
  EffV2M	480	tf	bicubic antialias	0.99	0.85086	0.9731
  EffV2M	480	[128, 128]	bicubic antialias	0.99	0.8508	0.97314
  EffV2M	480	[128, 128]	bicubic antialias	1.0	0.8503	0.9728
  EffV2L	480	tf	bicubic	0.99	0.855	0.97324
  EffV2L	480	tf	bicubic antialias	0.99	0.85478	0.9732
  EffV2M ft1k	480	tf	bicubic	0.99	0.85606	0.9775
  EffV2M ft1k	480	tf	bicubic	-1	0.86082	0.9795
  EffV2M ft1k	480	[128, 128]	bicubic	-1	0.8609	0.97952
  EffV2M ft1k	480	[128, 128]	bicubic antialias	-1	0.86068	0.97952
  EffV2L ft1k	480	tf	bicubic	0.99	0.86294	0.9799
  EffV2L ft1k	480	tf	bicubic	-1	0.86824	0.98162
  EffV2L ft1k	480	tf	bicubic antialias	-1	0.86824	0.98148
  EffV2L ft1k	480	[128, 128]	bicubic	-1	0.86818	0.98162
  EffV2L ft1k	480	[128, 128]	bicubic antialias	-1	0.8683	0.9815
  EffV2XL ft1k	512	tf	bicubic	0.99	0.86532	0.97866
  EffV2XL ft1k	512	tf	bicubic	-1	0.86772	0.98066
  EffV2XL ft1k	512	[128, 128]	bicubic	-1	0.86782	0.9806
  EffV2XL ft1k	512	[128, 128]	bicubic antialias	-1	0.86788	0.9806

[Medicmind]

Awesome you have added EfficientNet as is one of the best classifiers. Will have to try it out.

I have tried ViT as well from another repository which is supposed to be even better than efficientnet but had the issue that it was terrible with training with small datasets. Overfitting was terrible. I think it can be an issue with transformers.

[leondgarse]

Ya, VOLO also not well on small datasets. EfficientNetV2 is now included in official keras.application. I just merged my keras_efficientnet_v2 into here.

[Medicmind]

Yes, VOLO was the worst classifier I tried from your repository and was using a small dataset.

[Medicmind]

EfficientNetV2S worked well and got the best results of overall with the classifiers I've tried with my own datasets. Though EfficientNetV1L2 with image size 512 ran out of memory even using a 16GB Tesla T4 GPU with a batch of 1

python train_script.py -m efficientnet.EfficientNetV1L2 -d diabetic_oedema2 -b 1 --pretrained imagenet -i 512

Might need an NVidia v100 to train it

[leondgarse]

Yes, EfficientNetV1L2 is such a large model, even Keras themselves not holding it. Also EfficientNetV2XL is missing in TF 2.8.0-rc0 keras.applications. In some of my practices, EfficientNetV2M performs almost similar with V2S. Those large models may also need longer training.