Move unboxed dispatch decision into dispatcher #28251

smessmer · 2019-10-17T21:23:34Z

Stack from ghstack:

Implement backend level fallback for c10 #28494 [wip] Implement backend level fallback for c10
Simplify c10 dispatcher #28314 Simplify c10 dispatcher
Move unboxed dispatch decision into dispatcher #28251 Move unboxed dispatch decision into dispatcher

Before, the dispatch key for unboxed operators from native_functions.yaml was generated in codegen and passed to the c10 dispatcher.
Now, we generate it inside of the dispatcher, right next to where the same thing happens for boxed calls.

Differential Revision: D17989951

Before, the dispatch key for unboxed operators from native_functions.yaml was generated in codegen and passed to the c10 dispatcher. Now, we generate it inside of the dispatcher, right next to where the same thing happens for boxed calls. Differential Revision: [D17989951](https://our.internmc.facebook.com/intern/diff/D17989951/) [ghstack-poisoned]

Before, the dispatch key for unboxed operators from native_functions.yaml was generated in codegen and passed to the c10 dispatcher. Now, we generate it inside of the dispatcher, right next to where the same thing happens for boxed calls. Differential Revision: [D17989951](https://our.internmc.facebook.com/intern/diff/D17989951/) ghstack-source-id: 92136929 Pull Request resolved: #28251

ezyang · 2019-10-18T14:03:30Z

aten/src/ATen/core/dispatch/DispatchTable.h

+   template<class... Args>
+   const KernelFunction& lookup(const Args&... args) const {
+     return lookup_([&] () -> c10::optional<TensorTypeId> {
+       if (!dispatch_strategy_.is_valid_) {


Still not sure why we need to indirect through a dispatch strategy

I'm currently working on another simplification diff that will probably address that

Before, the dispatch key for unboxed operators from native_functions.yaml was generated in codegen and passed to the c10 dispatcher. Now, we generate it inside of the dispatcher, right next to where the same thing happens for boxed calls. Differential Revision: [D17989951](https://our.internmc.facebook.com/intern/diff/D17989951/) [ghstack-poisoned]

Pull Request resolved: #28251 Before, the dispatch key for unboxed operators from native_functions.yaml was generated in codegen and passed to the c10 dispatcher. Now, we generate it inside of the dispatcher, right next to where the same thing happens for boxed calls. ghstack-source-id: 92211517 Differential Revision: [D17989951](https://our.internmc.facebook.com/intern/diff/D17989951/)

Before, the dispatch key for unboxed operators from native_functions.yaml was generated in codegen and passed to the c10 dispatcher. Now, we generate it inside of the dispatcher, right next to where the same thing happens for boxed calls. Differential Revision: [D17989951](https://our.internmc.facebook.com/intern/diff/D17989951/) [ghstack-poisoned]

dzhulgakov

nice and clean!

Before, the dispatch key for unboxed operators from native_functions.yaml was generated in codegen and passed to the c10 dispatcher. Now, we generate it inside of the dispatcher, right next to where the same thing happens for boxed calls. Differential Revision: [D17989951](https://our.internmc.facebook.com/intern/diff/D17989951/) [ghstack-poisoned]

Differential Revision: [D18212009](https://our.internmc.facebook.com/intern/diff/D18212009/) [ghstack-poisoned]

Differential Revision: [D18212009](https://our.internmc.facebook.com/intern/diff/D18212009/) ghstack-source-id: 92856230 Pull Request resolved: #28851

Before, the dispatch key for unboxed operators from native_functions.yaml was generated in codegen and passed to the c10 dispatcher. Now, we generate it inside of the dispatcher, right next to where the same thing happens for boxed calls. Differential Revision: [D17989951](https://our.internmc.facebook.com/intern/diff/D17989951/) [ghstack-poisoned]

Differential Revision: [D18212009](https://our.internmc.facebook.com/intern/diff/D18212009/) [ghstack-poisoned]

Pull Request resolved: #28851 ghstack-source-id: 92965835 Differential Revision: [D18212009](https://our.internmc.facebook.com/intern/diff/D18212009/)

kostmo · 2019-10-31T05:24:53Z

CircleCI build failures summary

As of commit 4e5398d:

0/5 flaky
1/5 broken upstream

Here are the reasons each build failed.

This comment was automatically generated by Dr. CI.
Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

This comment has been revised 2 time(s).

Differential Revision: [D18212009](https://our.internmc.facebook.com/intern/diff/D18212009/) [ghstack-poisoned]

Pull Request resolved: #28851 ghstack-source-id: 93054489 Differential Revision: [D18212009](https://our.internmc.facebook.com/intern/diff/D18212009/)

Before, the dispatch key for unboxed operators from native_functions.yaml was generated in codegen and passed to the c10 dispatcher. Now, we generate it inside of the dispatcher, right next to where the same thing happens for boxed calls. Differential Revision: [D17989951](https://our.internmc.facebook.com/intern/diff/D17989951/) [ghstack-poisoned]

Summary: Pull Request resolved: pytorch/pytorch#28251 Before, the dispatch key for unboxed operators from native_functions.yaml was generated in codegen and passed to the c10 dispatcher. Now, we generate it inside of the dispatcher, right next to where the same thing happens for boxed calls. ghstack-source-id: 93085152 Test Plan: unit tests Differential Revision: D17989951 fbshipit-source-id: b343d9650debc62bfcff84cf4d6bdaf9dacc9d16

facebook-github-bot · 2019-11-01T19:08:16Z

This pull request has been merged in 88a34ef.

ezyang · 2019-11-01T21:06:14Z

This diff appears to reliably cause Windows to run out of memory when linking:

19:18:16    Creating library LIB\torch.lib and object LIB\torch.exp
19:18:16 LINK : fatal error LNK1102: out of memory
19:18:16 ninja: build stopped: subcommand failed.
19:18:16 -- Building version 1.4.0a0+88a34ef

I verified the last three CI results on the PR all failed this way. I am reverting it to give us more headroom on master.

smessmer requested review from dzhulgakov and ezyang October 17, 2019 21:25

ezyang approved these changes Oct 18, 2019

View reviewed changes

ezyang reviewed Oct 18, 2019

View reviewed changes

smessmer mentioned this pull request Oct 18, 2019

Simplify c10 dispatcher #28314

Closed

dzhulgakov approved these changes Oct 21, 2019

View reviewed changes

smessmer added 4 commits October 21, 2019 15:23

smessmer mentioned this pull request Oct 23, 2019

Implement backend level fallback for c10 #28494

Closed

This was referenced Oct 24, 2019

Remove c10->ATen registration forwarding #28186

Closed

Delete ATenDispatch #28468

Closed

smessmer added 5 commits October 24, 2019 15:31

smessmer mentioned this pull request Oct 26, 2019

Allow dispatch based on ScalarType, Layout, Device #28723

Closed

smessmer added 3 commits October 28, 2019 11:52

smessmer added a commit that referenced this pull request Oct 29, 2019

[no-review] test #28251 without stack

08d51ca

Differential Revision: [D18212009](https://our.internmc.facebook.com/intern/diff/D18212009/) [ghstack-poisoned]

smessmer mentioned this pull request Oct 29, 2019

[no-review] test https://github.com/pytorch/pytorch/pull/28251 without stack #28851

Closed

smessmer added a commit that referenced this pull request Oct 29, 2019

[no-review] test #28251 without stack

a57aa64

Differential Revision: [D18212009](https://our.internmc.facebook.com/intern/diff/D18212009/) ghstack-source-id: 92856230 Pull Request resolved: #28851

smessmer added a commit that referenced this pull request Oct 31, 2019

Update on "[no-review] test #28251 without stack"

6038e83

Differential Revision: [D18212009](https://our.internmc.facebook.com/intern/diff/D18212009/) [ghstack-poisoned]

smessmer added a commit that referenced this pull request Oct 31, 2019

[no-review] test #28251 without stack

294abd2

Pull Request resolved: #28851 ghstack-source-id: 92965835 Differential Revision: [D18212009](https://our.internmc.facebook.com/intern/diff/D18212009/)

smessmer added a commit that referenced this pull request Nov 1, 2019

Update on "[no-review] test #28251 without stack"

9aa90cd

Differential Revision: [D18212009](https://our.internmc.facebook.com/intern/diff/D18212009/) [ghstack-poisoned]

smessmer added a commit that referenced this pull request Nov 1, 2019

[no-review] test #28251 without stack

a9d4484

Pull Request resolved: #28851 ghstack-source-id: 93054489 Differential Revision: [D18212009](https://our.internmc.facebook.com/intern/diff/D18212009/)

smessmer added 2 commits October 31, 2019 18:00

facebook-github-bot closed this in 88a34ef Nov 1, 2019

facebook-github-bot added the merged label Nov 1, 2019

ezyang mentioned this pull request Nov 1, 2019

LINK : fatal error LNK1102: out of memory (Windows CUDA CI) LNK #27215

Closed

facebook-github-bot deleted the gh/smessmer/93/head branch November 5, 2019 15:18

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Move unboxed dispatch decision into dispatcher #28251

Move unboxed dispatch decision into dispatcher #28251

Uh oh!

smessmer commented Oct 17, 2019 •

edited

Loading

Uh oh!

ezyang Oct 18, 2019

Uh oh!

smessmer Oct 18, 2019

Uh oh!

dzhulgakov left a comment

Uh oh!

kostmo commented Oct 31, 2019 •

edited

Loading

Uh oh!

facebook-github-bot commented Nov 1, 2019

Uh oh!

ezyang commented Nov 1, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Move unboxed dispatch decision into dispatcher #28251

Move unboxed dispatch decision into dispatcher #28251

Uh oh!

Conversation

smessmer commented Oct 17, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ezyang Oct 18, 2019

Choose a reason for hiding this comment

Uh oh!

smessmer Oct 18, 2019

Choose a reason for hiding this comment

Uh oh!

dzhulgakov left a comment

Choose a reason for hiding this comment

Uh oh!

kostmo commented Oct 31, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CircleCI build failures summary

Uh oh!

facebook-github-bot commented Nov 1, 2019

Uh oh!

ezyang commented Nov 1, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

smessmer commented Oct 17, 2019 •

edited

Loading

kostmo commented Oct 31, 2019 •

edited

Loading