Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Do not batch reduce as keras optimzer already does. #183

Merged
merged 1 commit into from
Dec 6, 2021
Merged

Do not batch reduce as keras optimzer already does. #183

merged 1 commit into from
Dec 6, 2021

Conversation

acmore
Copy link
Contributor

@acmore acmore commented Nov 8, 2021

Description

Brief Description of the PR:

This pr is to remove batch_reduce_to call on the gradients for optimizer v2. In the optimzer v2 base class, the gradients are aggregated already.

Type of change

  • Bug fix
  • New Tutorial
  • Updated or additional documentation
  • Additional Testing
  • New Feature

Checklist:

  • I've properly formatted my code according to the guidelines
    • By running yapf
    • By running clang-format
  • This PR addresses an already submitted issue for TensorFlow Recommenders-Addons
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works

How Has This Been Tested?

If you're adding a bugfix or new feature please describe the tests that you ran to verify your changes:
*

@acmore acmore requested a review from rhdong as a code owner November 8, 2021 02:59
@rhdong rhdong requested a review from Lifann November 9, 2021 11:07
@MoFHeka
Copy link
Contributor

MoFHeka commented Nov 11, 2021

Did you check for tf.compat.v1.train.optimzer?

@acmore
Copy link
Contributor Author

acmore commented Nov 12, 2021

tf.compat.v1.train.optimzer

Correct me if I am wrong. I think this _distributed_apply was replaced only in keras optimizer?

@MoFHeka MoFHeka closed this Nov 13, 2021
@MoFHeka
Copy link
Contributor

MoFHeka commented Nov 13, 2021 •
edited

tf.compat.v1.train.optimzer

Correct me if I am wrong. I think this _distributed_apply was replaced only in keras optimizer?

Sorry, you're right. And this bug seems may cause a problem if we don't assign specific local device when using keras model.
Please run "find ./ -name *.py -exec yapf --style=./.yapf -ir {} ;" for formatting.

@MoFHeka MoFHeka reopened this Nov 13, 2021
@rhdong rhdong requested a review from MoFHeka November 14, 2021 07:46
@acmore
Copy link
Contributor Author

acmore commented Nov 15, 2021

tf.compat.v1.train.optimzer

Correct me if I am wrong. I think this _distributed_apply was replaced only in keras optimizer?

Sorry, you're right. And this bug seems may cause a problem if we don't assign specific local device when using eras model. Please run "find ./ -name *.py -exec yapf --style=./.yapf -ir {} ;" for formatting.

done

@Lifann
Copy link
Member

Lifann commented Nov 15, 2021 •
edited

It does call batch_reduce_to twice on one call.

Lifann
Lifann approved these changes Nov 15, 2021
View reviewed changes
Copy link
Member

@Lifann Lifann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Lifann Lifann self-requested a review November 15, 2021 04:48
MoFHeka
MoFHeka previously approved these changes Nov 15, 2021
View reviewed changes
Copy link
Contributor

@MoFHeka MoFHeka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Lifann
Lifann previously approved these changes Nov 22, 2021
View reviewed changes
rhdong
rhdong previously approved these changes Dec 2, 2021
View reviewed changes
Copy link
Member

@rhdong rhdong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rhdong
Copy link
Member

rhdong commented Dec 2, 2021

Hi @acmore, thank you for the fix, and please squash the 3 commits into one, after that I will merge it.

@acmore acmore dismissed stale reviews from rhdong, Lifann, and MoFHeka via cf594dd December 5, 2021 09:54
@acmore acmore closed this Dec 5, 2021
@acmore acmore reopened this Dec 5, 2021
@acmore
Copy link
Contributor Author

acmore commented Dec 5, 2021

Hi @acmore, thank you for the fix, and please squash the 3 commits into one, after that I will merge it.

Thanks. I have squashed them into one.

@rhdong rhdong merged commit c768a2f into tensorflow:master Dec 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rhdong rhdong rhdong left review comments

@Lifann Lifann Lifann left review comments

@MoFHeka MoFHeka MoFHeka left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@acmore @MoFHeka @Lifann @rhdong