Skip to content

Mixed precision support#6309

Merged
tfboyd merged 2 commits intotensorflow:masterfrom
reedwm:fp16_keras
Mar 6, 2019
Merged

Mixed precision support#6309
tfboyd merged 2 commits intotensorflow:masterfrom
reedwm:fp16_keras

Conversation

@reedwm
Copy link
Contributor

@reedwm reedwm commented Mar 6, 2019

Add mixed precision support to the Keras Resnet model.

This uses the experimental tf.keras mixed precision API. The API will eventually allow users to use mixed precision with a single line of code, but currently it requires several changes to model code.

I tested mixed precision converges with the command line:

python keras_imagenet_main.py --clean --num_gpus=8 --data_dir=/raid/imagenet-2012-tfrecord --batch_size=1024 --train_epochs=90 --model_dir ~/fp16_keras_mod --dtype=fp16 --enable_eager=true --datasets_num_private_threads=14

The final validation accuracy is 76.35%. I ran tensorflow at commit tensorflow/tensorflow@23ec33a.

@reedwm reedwm requested a review from fchollet March 6, 2019 02:13
@reedwm reedwm requested review from a team and karmel as code owners March 6, 2019 02:13
@reedwm reedwm requested review from haoyuz and tfboyd March 6, 2019 20:50
@tfboyd tfboyd removed request for a team and karmel March 6, 2019 20:55
Copy link
Member

@tfboyd tfboyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@haoyuz haoyuz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tfboyd tfboyd merged commit e4a046e into tensorflow:master Mar 6, 2019
wlongxiang pushed a commit to wlongxiang/models that referenced this pull request Apr 24, 2019
* Mixed precision support

* Add TODOs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants