Keras float16 vs float32

Requirements

keras==2.1.0

tensorflow==1.8.0

opencv==3.2.0

pycocotools, BeautifulSoup4, lxml, tqdm

How to set float16 compute mode.

from keras import backend as K

K.set_floatx('float16)

Image Classification Part

Usage

Firstly, download ImageNet val data and model pre-trained weights file.

An example for testing vgg16 with float16.

python eval_image_classification.py --model='vgg' --dtype='float16'

An example for testing mobilenet with a width multiplier 1.0.

python eval_image_classification.py --model='mobilenet' --dtype='float16' --alpha=1.0

ImageNet Datatset

ImageNet val data provided by aaron-xichen, sincerely thanks to aaron-xichen for sharing this processed ImageNet val data.

Results

TOP1 acc and TOP5 acc on ImageNet val data.

Pre-trained weight files are downloaded from deep learning models, DenseNet121 weight file is downloaded from DenseNet-Keras Squeezenet weight file is downloaded from keras-squeezenet

There is a backup of weights in baiduyun（百度云）

	float32		float16		diff
Model	Top1 acc	Top5 acc	Top1 acc	Top5 acc	Top1 acc	Top5 acc
VGG16	0.70786	0.89794	0.7082	0.89802	0.00034	0.00008
ResNet50	0.74366	0.91806	0.70508	0.89466	-0.03858	-0.0234
Inceptionv3	0.76518	0.92854	0.765	0.92854	-0.00018	0.00
Inception-ResNet	0.789	0.94426	0.7888	0.94436	-0.0002	0.0001
DenseNet121	0.74234	0.91868	0.74206	0.91868	-0.00028	0.000
Xception	0.77446	0.93618	0.77392	0.93596	-0.00054	-0.00049
Squeezenet	0.52294	0.76312	0.52172	0.76226	-0.00122	-0.00086
MobileNet-1-0	0.69856	0.89174	0.6966	0.8898	-0.00196	-0.00194
MobileNet-7-5	0.67726	0.87838	0.6726	0.87652	-0.00466	-0.00186
MobileNet-5-0	0.6352	0.85006	0.62944	0.84644	-0.00576	-0.00362
MobileNet-2-5	0.5134	0.75546	0.50272	0.74648	-0.01068	-0.00898

Object Detection Part

Usage

Firstly, download VOC2007 test set and COCO2017 val set, COCO2017 val set annotations datasets, then extract them and modify the path in script.

Secondly, download SSD pre-trained weights and put them in 'weights' directory.

SSD300 VOC weights, SSD300 COCO weights, SSD512 VOC weights, SSD512 COCO weights

The method for converting the original YOLOv3 model to a keras model can be found in this repo.

An example for evaluating SSD300 on VOC2007 test set

python eval_object_detection.py --model='ssd300' --dtype='float16' --eval-dataset='voc2007'

Results

Notice that SSD models suffer significant accuracy loss.

SSD results on VOC2007 test set

	mAP
Model	float32	float16	diff
SSD300	0.782	0.769	-0.013
SSD512	0.91	0.868	-0.042

The AP of each category can be found in this doc

SSD and YOLOv3 results on COCO val2017.

	mAP
Model	float32	float16	diff
SSD300	0.424	0.374	-0.050
SSD512	0.481	0.448	-0.033
YOLO320	to do	to do	to do
YOLO416	to do	to do	to do
YOLO608	to do	to do	to do

Semantic Segmentation Part

In this part, I evaluate semantic segmentation with float16 dtype.

U-net is adopted in this test.

HumanParsing-Dataset is adopted in this test.

The tested models are trained by my-self. Training details can be found in this repo: Person-Segmentation-Keras.

Usage

For person segmentation (binary classification) task.

python eval_segmentation.py --model='unet' --dtype='float16' --nClasses=2

For human parsing (multi-class classification) task.

python eval_segmentation.py --model='unet' --dtype='float16' --nClasses=5

Results

Person segmentation

	mIoU
Model	float32	float16	diff
Unet	0.8920	0.8918	-0.0002

Human parsing

		mIoU
	Part	float32	float16	diff
Unet	head	0.66476	0.66463	-0.00013
	upper body	0.48639	0.48640	0.00001
	both hands	0.27016	0.27005	-0.00011
	lower body	0.66536	0.66520	-0.00016
	mean	0.52167	0.52157	-0.0001

PointNet Evaluation

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

In this evaluation, I use a pre-trained PointNet reference to PointNet-Keras.

Results

3D classification

	acc
Model	float32	float16	diff
PointNet_cls	0.87824	0.87784	-0.0004

Reference

deep learing models

DenseNet-Keras

keras-squeezenet

ssd_keras

keras-yolo3

SegNet-Tutorial

Tensorflow-SegNet

image-segmentation-keras