Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not densify sparse gradients in LossScaleOptimizer #37566

Conversation

@guillaumekln
Copy link
Contributor

guillaumekln commented Mar 13, 2020

The method get_unscaled_gradients is currently converting sparse gradients to dense gradients. We can avoid this implicit conversion to save on memory.

@tensorflow-bot tensorflow-bot bot added the size:S label Mar 13, 2020
@googlebot googlebot added the cla: yes label Mar 13, 2020
@gbaned gbaned self-assigned this Mar 13, 2020
@gbaned gbaned added the comp:keras label Mar 13, 2020
@gbaned gbaned added this to Assigned Reviewer in PR Queue via automation Mar 13, 2020
@gbaned gbaned requested a review from pavithrasv Mar 13, 2020
@pavithrasv pavithrasv requested review from reedwm and removed request for pavithrasv Mar 13, 2020
@reedwm
reedwm approved these changes Mar 13, 2020
PR Queue automation moved this from Assigned Reviewer to Approved by Reviewer Mar 13, 2020
@reedwm

This comment has been minimized.

Copy link
Member

reedwm commented Mar 13, 2020

Thanks for the PR!

@tensorflow-copybara tensorflow-copybara merged commit d1dd08d into tensorflow:master Mar 13, 2020
10 checks passed
10 checks passed
Android Demo App Internal CI build successful
Details
Linux GPU Internal CI build successful
Details
MacOS CPU Python3 Internal CI build successful
Details
MacOS Python2 and CC Internal CI build successful
Details
Ubuntu CPU Internal CI build successful
Details
Ubuntu Sanity Internal CI build successful
Details
Windows Bazel Internal CI build successful
Details
Windows Bazel GPU Internal CI build successful
Details
cla/google All necessary CLAs are signed
import/copybara Change imported to the internal review system
Details
PR Queue automation moved this from Approved by Reviewer to Merged Mar 13, 2020
@guillaumekln guillaumekln deleted the guillaumekln:do-not-densify-scaled-sparse-gradients branch Mar 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
PR Queue
  
Merged
Linked issues

Successfully merging this pull request may close these issues.

None yet

6 participants
You can’t perform that action at this time.