Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add GeM layer #16747

Closed
wants to merge 12 commits into from
Closed

Add GeM layer #16747

wants to merge 12 commits into from

Conversation

innat
Copy link

@innat innat commented Jul 3, 2022 •
edited

@innat innat marked this pull request as ready for review July 3, 2022 17:25
@gbaned gbaned added this to Assigned Reviewer in PR Queue via automation Jul 4, 2022
@gbaned gbaned requested a review from fchollet July 4, 2022 15:06
@google-ml-butler google-ml-butler bot added the keras-team-review-pending Pending review by a Keras team member. label Jul 4, 2022
@old-school-kid
Copy link
Contributor

The 4th page of the paper states

The pooling parameter pk can be manually set or learned since this operation is differentiable and can be part of the back-propagation.

I think we can set the 'power' as a learnable parameter instead of treating it as a hyperparameter, which is again demonstrated in the experiments in the paper itself.

fchollet
fchollet reviewed Jul 4, 2022
View reviewed changes
Copy link
Member

@fchollet fchollet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! IMO we can introduce a single GeMPooling layer that works for 1D/2D/3D inputs. No need for a base class either.

keras/layers/pooling/base_generalized_pooling1d.py Outdated Show resolved Hide resolved
keras/layers/pooling/base_generalized_pooling2d.py Outdated Show resolved Hide resolved
@gbaned gbaned removed the keras-team-review-pending Pending review by a Keras team member. label Jul 6, 2022
@innat
Copy link
Author

innat commented Jul 7, 2022

The 4th page of the paper states

The pooling parameter pk can be manually set or learned since this operation is differentiable and can be part of the back-propagation.

I think we can set the 'power' as a learnable parameter instead of treating it as a hyperparameter, which is again demonstrated in the experiments in the paper itself.

@old-school-kid
I see your point. We can set manually or as a learnable parameter. But I think, we need to select one option here. I mostly followed tf/models, and tf/similarity approaches, and they both choose manual passing.

@fchollet
Copy link
Member

IMO it seems fine to keep power as a static constructor argument.

@innat innat requested a review from fchollet July 24, 2022 17:25
@google-ml-butler google-ml-butler bot added the keras-team-review-pending Pending review by a Keras team member. label Jul 24, 2022
@mihirparadkar mihirparadkar removed the keras-team-review-pending Pending review by a Keras team member. label Jul 28, 2022
@gbaned gbaned requested review from fchollet and removed request for fchollet August 5, 2022 07:58
@google-ml-butler google-ml-butler bot added the keras-team-review-pending Pending review by a Keras team member. label Aug 5, 2022
fchollet
fchollet reviewed Aug 8, 2022
View reviewed changes
Copy link
Member

@fchollet fchollet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update!

keras/layers/__init__.py Outdated Show resolved Hide resolved
keras/layers/pooling/BUILD Outdated Show resolved Hide resolved
keras/layers/pooling/__init__.py Outdated Show resolved Hide resolved
keras/layers/pooling/base_generalized_pooling.py Outdated Show resolved Hide resolved
keras/layers/pooling/base_generalized_pooling.py Outdated Show resolved Hide resolved
keras/layers/pooling/base_generalized_pooling.py Outdated Show resolved Hide resolved
keras/layers/pooling/generalized_mean_pooling1d.py Outdated Show resolved Hide resolved
keras/layers/pooling/generalized_mean_pooling1d.py Outdated Show resolved Hide resolved
keras/layers/pooling/generalized_mean_pooling2d.py Outdated Show resolved Hide resolved
keras/layers/pooling/generalized_mean_pooling3d.py Outdated Show resolved Hide resolved
@fchollet fchollet added stat:awaiting response from contributor and removed keras-team-review-pending Pending review by a Keras team member. labels Aug 8, 2022
@gbaned gbaned requested a review from fchollet October 14, 2022 12:06
@google-ml-butler google-ml-butler bot added the keras-team-review-pending Pending review by a Keras team member. label Oct 14, 2022
@gbaned gbaned requested review from fchollet and removed request for fchollet November 24, 2022 19:05
@google-ml-butler google-ml-butler bot added the keras-team-review-pending Pending review by a Keras team member. label Nov 24, 2022
@qlzh727 qlzh727 added stat:awaiting keras-eng Awaiting response from Keras engineer and removed keras-team-review-pending Pending review by a Keras team member. labels Dec 1, 2022
@gbaned
Copy link
Collaborator

gbaned commented Dec 16, 2022

Hi @qlzh727 / @fchollet Any update on this PR? Please. Thank you!

1 similar comment
@gbaned
Copy link
Collaborator

gbaned commented Mar 21, 2023

Hi @qlzh727 / @fchollet Any update on this PR? Please. Thank you!

@innat innat closed this Jun 19, 2023
PR Queue automation moved this from Assigned Reviewer to Closed/Rejected Jun 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fchollet fchollet Awaiting requested review from fchollet

Assignees

@gbaned gbaned

Labels
size:L stat:awaiting keras-eng Awaiting response from Keras engineer
Projects
PR Queue
Closed/Rejected
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add Generalized Mean Pooling layer
6 participants
@innat @old-school-kid @fchollet @gbaned @qlzh727 @mihirparadkar