Skip to content

Add fused add_relu op. #39342

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 11 commits into from
Closed

Add fused add_relu op. #39342

wants to merge 11 commits into from

Conversation

@kimishpatel
Copy link
Contributor

@kimishpatel kimishpatel commented Jun 1, 2020 •
edited
Loading

Stack from ghstack:

Summary:
Many networks such as resnet have adds followed by relu. This op is the
first step in enabling this fused implementation.
Once we have the fused add_relu op, a JIT pass will be written to
replace add + relu patterns with add_relu.

Test Plan:
python test/test_nn.py TestAddRelu

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D21822397

@kimishpatel
Add fused add_relu op.
7c6e6a0 
Summary:
Many networks such as resnet have adds followed by relu. This op is the
first step in enabling this fused implementation.
Once we have the fused add_relu op, a JIT pass will be written to
replace add + relu patterns with add_relu.

Test Plan:
python test/test_nn.py TestAddRelu

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
This was referenced Jun 1, 2020
Closed
Closed
@dr-ci
Copy link

dr-ci bot commented Jun 1, 2020 •
edited
Loading

💊 CI failures summary and remediations

As of commit a5c46a5 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 51 times.

@kimishpatel
Update on "Add fused add_relu op."
3941f9d 
Summary:
Many networks such as resnet have adds followed by relu. This op is the
first step in enabling this fused implementation.
Once we have the fused add_relu op, a JIT pass will be written to
replace add + relu patterns with add_relu.

Test Plan:
python test/test_nn.py TestAddRelu

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D21822397](https://our.internmc.facebook.com/intern/diff/D21822397)

[ghstack-poisoned]
dreiss
dreiss reviewed Jun 4, 2020
View reviewed changes
aten/src/ATen/native/cpu/BinaryOpsKernel.cpp Outdated
}

void add_clamp_kernel(TensorIterator& iter, Scalar alpha_scalar, Scalar min_val, Scalar max_val) {
AT_DISPATCH_ALL_TYPES(iter.dtype(), "add_relu_cpu", [&]() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"add_clamp_cpu"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Will fix it.

@kimishpatel
Update on "Add fused add_relu op."
9509b1b 
Summary:
Many networks such as resnet have adds followed by relu. This op is the
first step in enabling this fused implementation.
Once we have the fused add_relu op, a JIT pass will be written to
replace add + relu patterns with add_relu.

Test Plan:
python test/test_nn.py TestAddRelu

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D21822397](https://our.internmc.facebook.com/intern/diff/D21822397)

[ghstack-poisoned]
@kimishpatel kimishpatel mentioned this pull request Jun 15, 2020
Closed
@kimishpatel
Update on "Add fused add_relu op."
22e497d 
Summary:
Many networks such as resnet have adds followed by relu. This op is the
first step in enabling this fused implementation.
Once we have the fused add_relu op, a JIT pass will be written to
replace add + relu patterns with add_relu.

Test Plan:
python test/test_nn.py TestAddRelu

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D21822397](https://our.internmc.facebook.com/intern/diff/D21822397)

[ghstack-poisoned]
@dreiss
Copy link
Contributor

dreiss commented Jun 17, 2020

Do you have any numbers on how much this improves resnet performance compared to unfused (but vec256-ized) add then relu?

@dreiss
Copy link
Contributor

dreiss commented Jun 17, 2020

(or any other model like unet)

@kimishpatel
Copy link
Contributor Author

Do you have any numbers on how much this improves resnet performance compared to unfused (but vec256-ized) add then relu?

I havent tried on resnet. For some reason on unet it did not make any difference. I did not dig further yet to find out why. I will try it out on resnet.

@kimishpatel
Copy link
Contributor Author

kimishpatel commented Jun 18, 2020 •
edited
Loading

@dreiss, I checked perf on unet model on s9 phone.
Before this stack: 19ms
After this stack and applying add_relu fusion pass (since I am yet to add this to optimizeForMobile, will stack another diff on top for that): 15ms

@kimishpatel
Update on "Add fused add_relu op."
40f260b 
Summary:
Many networks such as resnet have adds followed by relu. This op is the
first step in enabling this fused implementation.
Once we have the fused add_relu op, a JIT pass will be written to
replace add + relu patterns with add_relu.

Test Plan:
python test/test_nn.py TestAddRelu

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D21822397](https://our.internmc.facebook.com/intern/diff/D21822397)

[ghstack-poisoned]
@kimishpatel
Update on "Add fused add_relu op."
5a80fc3 
Summary:
Many networks such as resnet have adds followed by relu. This op is the
first step in enabling this fused implementation.
Once we have the fused add_relu op, a JIT pass will be written to
replace add + relu patterns with add_relu.

Test Plan:
python test/test_nn.py TestAddRelu

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D21822397](https://our.internmc.facebook.com/intern/diff/D21822397)

[ghstack-poisoned]
@kimishpatel kimishpatel mentioned this pull request Jun 18, 2020
Closed
@kimishpatel kimishpatel requested a review from dreiss June 19, 2020 18:13
@kimishpatel
Update on "Add fused add_relu op."
2a4d312 
Summary:
Many networks such as resnet have adds followed by relu. This op is the
first step in enabling this fused implementation.
Once we have the fused add_relu op, a JIT pass will be written to
replace add + relu patterns with add_relu.

Test Plan:
python test/test_nn.py TestAddRelu

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D21822397](https://our.internmc.facebook.com/intern/diff/D21822397)

[ghstack-poisoned]
@kimishpatel
Update on "Add fused add_relu op."
ec7da97 
Summary:
Many networks such as resnet have adds followed by relu. This op is the
first step in enabling this fused implementation.
Once we have the fused add_relu op, a JIT pass will be written to
replace add + relu patterns with add_relu.

Test Plan:
python test/test_nn.py TestAddRelu

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D21822397](https://our.internmc.facebook.com/intern/diff/D21822397)

[ghstack-poisoned]
@kimishpatel
Update on "Add fused add_relu op."
e780328 
Summary:
Many networks such as resnet have adds followed by relu. This op is the
first step in enabling this fused implementation.
Once we have the fused add_relu op, a JIT pass will be written to
replace add + relu patterns with add_relu.

Test Plan:
python test/test_nn.py TestAddRelu

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D21822397](https://our.internmc.facebook.com/intern/diff/D21822397)

[ghstack-poisoned]
@kimishpatel
Update on "Add fused add_relu op."
2be4b66 
Summary:
Many networks such as resnet have adds followed by relu. This op is the
first step in enabling this fused implementation.
Once we have the fused add_relu op, a JIT pass will be written to
replace add + relu patterns with add_relu.

Test Plan:
python test/test_nn.py TestAddRelu

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D21822397](https://our.internmc.facebook.com/intern/diff/D21822397)

[ghstack-poisoned]
@kimishpatel
Update on "Add fused add_relu op."
a5c46a5 
Summary:
Many networks such as resnet have adds followed by relu. This op is the
first step in enabling this fused implementation.
Once we have the fused add_relu op, a JIT pass will be written to
replace add + relu patterns with add_relu.

Test Plan:
python test/test_nn.py TestAddRelu

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D21822397](https://our.internmc.facebook.com/intern/diff/D21822397)

[ghstack-poisoned]
@facebook-github-bot
Copy link
Contributor

This pull request has been merged in 82c9f79.

@facebook-github-bot facebook-github-bot deleted the gh/kimishpatel/28/head branch July 13, 2020 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dreiss dreiss dreiss approved these changes

@AshkanAliabadi AshkanAliabadi Awaiting requested review from AshkanAliabadi

Assignees

No one assigned

Labels

Merged

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@kimishpatel @dreiss @facebook-github-bot @mruberry