C function wrapper uniqueness #1912

albanD · 2017-06-26T15:47:18Z

Ideas of this PR:

If a THPCppFunction exists for a Function, it owns the function and is responsible to delete it.
To make sure that a Function used in the autograd graph is not freed while still in use in the cpp side, the Function holds one reference to the THPCppFunction that wraps it. This reference is released when the shared_ptr used in the graph is freed (see FunctionDeleter).
Every Function that will be used in the graph should be created with SharedFunctionMaker. Functions that are not used in the graph can be created regularly. In particular cpp functions that are used only for forward do not need associated shared pointers.

colesbury · 2017-07-05T15:55:57Z

This looks generally good, but introduces a new pattern for connecting C++ objects and Python wrappers. The pattern in variable.h/python_variable.h is to have a "weak" pointer to the PyObject* and the Python object has a "strong" (owning) reference back to the C++ object.

There's a few small differences between these patterns. For example, in the variable pattern, the wrapper object can be freed once there are no Python references to it. I'm not sure one pattern is overall better than the other.

Mostly, I'm ambivalent about adding a new pattern for connecting C++ objects and their wrappers, when doing something different is not strictly necessary.

albanD · 2017-07-11T15:01:57Z

This difference between the two wrapper is exactly what I was looking for. (Actually possibly changing the variable wrapper to have the same property may be interesting in the future).
The problem with the current implementation (and possibly the variable one as well) is that a single C++ variable may have different python wrapper. Which breaks things like this visualization script.

Which one of the following two pattern you think we should use (and I agree that we should change both variable and function for simplicity):

uniqueness of the wrapper for the whole lifetime of the C++ object (new function behavior with this PR)?
uniqueness of the wrapper as long as it is still referenced from python (current variable implementation) but a new wrapper will be created as soon as the previous one goes out of scope in python (this can lead to subtle issue like in the visualization tool above where id() is used to check wrapper uniqueness even though the PyObject* actually goes out of scope, or impossibility to store data in the python object because you may get another one next time)?

colesbury · 2017-07-19T21:58:13Z

@albanD, can you change this to use the variable pattern? I'm not sure that preserving wrapper objects even after they're no longer referenced from Python is necessary. In the mentioned script, for example, objects are stored in the seen set so id()'s are consistent.

The downside of this is that the C++ object and its wrapper are tightly coupled. This will make it harder to have a useful C++ autograd library. (In the Variable pattern, the C++ object only needs space for an extra pointer, otherwise it doesn't need to know anything about the wrapper).

@soumith wants to cut a new release very soon. Do you have time to update the PR? If not, I can do it.

albanD · 2017-07-20T00:11:50Z

The problem i have with the variable behavior is that you cannot store temporary informations in a variable object in python side. Or you have to make sure that the python object stays in scope.
For example, for better graph printing i was planning on adding a dict property on the python object so that we can easily store arbitrary metadata for a Function.

But i agree that it requires a tight coupling which we might not want.

Changing Function to have same behavior as Variables is a useful, maybe we can delay the question for full persistence of the python object/how to store arbitrary metadata for later?

colesbury · 2017-07-20T00:22:23Z

Yeah, I think we may want to revisit this in the future.

As you point out, if we want to store dict properties on Functions/Variables then this isn't ideal. However, I'm not sure we should even allow that. Numpy arrays and Python numbers and strings, for example, don't allow storing dict properties. You can still associate data with a Variable by using a dictionary with the Variable/Function as the key.

albanD · 2017-07-20T00:53:21Z

Sounds good to me to change Function behavior to be the same as Variables.
I am currently in transit so if you need this quickly you can do it. Otherwise i will take care of this tomorrow.

soumith · 2017-07-21T18:40:21Z

haven't heard back from you @albanD, so i'm asking @colesbury to finish this.

albanD · 2017-07-22T02:03:55Z

@colesbury I implemented the change so that Functions now match the Variable behavior.
Could you please take a final look at this ?

torch/csrc/autograd/python_cpp_function.cpp

@@ -78,13 +78,13 @@ int THPCppFunction_traverse(PyObject* self, visitproc visit, void *arg)

 int THPCppFunction_clear(PyObject* self)
 {
-  ((THPCppFunction*)self)->cdata.reset();
+  ((THPCppFunction*)self)->cdata.~shared_ptr();


torch/csrc/autograd/python_cpp_function.cpp

+    f->cdata = cdata;
+
+    cdata->pyobj = obj.release();
+    Py_INCREF(cdata->pyobj);


torch/csrc/autograd/python_cpp_function.h

@@ -22,7 +22,7 @@ PyObject* CppFunction_pynew(PyTypeObject *type, PyObject *args, PyObject *kwds)
  if (!obj) return NULL;
  THPCppFunction* f = (THPCppFunction*)obj.get();
  HANDLE_TH_ERRORS
-  new (&f->cdata) std::shared_ptr<Function>(Ctor()(args));
+  f->cdata.reset(Ctor()(args));


apaszke

It seems that the weakref is never cleared. It should be reset to NULL in THPFunction's destructor. Also, it seems that pyobj works like a strong reference, in contrast to what the comment says.

torch/csrc/autograd/python_cpp_function.cpp

+    new (&f->cdata) std::shared_ptr<Function>(cdata);
+
+    cdata->pyobj = obj.release();
+    Py_INCREF(cdata->pyobj);


soumith · 2017-07-24T22:33:08Z

@albanD all contbuilds now fail on this. Looks like a segfault?

albanD · 2017-07-25T02:36:34Z

Hi,
Yes, even though the implementation seems correct, there are some refcount that goes below 0.
@colesbury any idea what could cause this?

torch/csrc/autograd/python_cpp_function.cpp

-  ((THPCppFunction*)self)->cdata.reset();
+  auto f = (THPCppFunction*)self;
+  // Remove the weak ref of the c++ object if it exist
+  f->cdata->pyobj = nullptr;


colesbury

lgtm

soumith · 2017-07-25T07:43:20Z

awesome! thanks for getting this in alban!

xmoddroid · 2017-07-31T21:04:01Z

thanks you brother

Syncing nvfuser devel branch to upstream master. https://github.com/csarofeen/pytorch/ Codegen changes include: - codegen improvement: i. improved view support on pointwise and transpose scheduler ii. grouped grid welford added for better outer-norm grid persistence in normalization - misc: i. new composite ops added: variance_mean , arange, ii. fixes misaligned address for transpose scheduler iii. refactor on separation of compilation API from execution API to prepare us for async compilation iv. double type support on expression evaluator v. PYTORCH_NVFUSER_DUMP refactor to save PTX and CUBIN Commits that's in this PR from the devel branch: ``` 89330aa Tensor factories must set the output shape as its input (#1939) b2fd01e arange support (#1933) 56c00fd Double support on all expression evaluators (#1937) 371f282 Improve trivial reduction merge support (#1931) 1d0c267 Test `rand` in a fusion with zero tensor input (#1932) 0dab160 Fix softmax bwd sizes. (#1890) ef98f36 Fix a bug (#1936) 63132a0 Propagate permissive mapping information into indexing pass (#1929) b4ac2c8 Map IterationDomains through view operations. (#1919) c0a187a do not use deprecated functions (#1935) 88de85e Upstream cherry pick fixes 0811 (#1934) b247dcf Separate kernel compilation API from kernel execution API (#1914) b34e3b9 Fix `ir_utils::hasBlockSync` + misc fixes in transpose scheduler (#1924) 14a53e6 Nullary RNGOp (#1892) 3c3c89e Misc fixes/tuning for transpose scheduler (#1912) 20cf109 Grouped grid welford (#1921) 6cf7eb0 Transpose scheduler small dim sizes better support (#1910) 9341ea9 Disabled ViewPersistentShmoo sizes that results in NAN (#1922) 057237f Fix CUDA driver error: misaligned address for transpose scheduler (#1918) 3fb3d80 Add variance_mean function using Welford (#1907) 98febf6 Remove DisableOption::UnrollWithRng (#1913) ee8ef33 Minor fix for the debug interface of using PTX directly (#1917) 6e8f953 Add PYTORCH_NVFUSER_DUMP options to save PTX and CUBIN (#1916) 5eefa9a dopt is only available since nvrtc 11.7 (#1915) 2ec8fc7 Kill computeAtBetween (#1911) d0d106a Improve view support on pointwise and transpose scheduler (#1906) e71e1ec Fix name clash of RNG with shared memory (#1904) 3381793 Fix mutator and sameAs for expanded IterDomain (#1902) ``` RUN_TORCHBENCH: nvfuser Differential Revision: [D39324552](https://our.internmc.facebook.com/intern/diff/D39324552) [ghstack-poisoned]

Syncing nvfuser devel branch to upstream master. https://github.com/csarofeen/pytorch/ Codegen changes include: - codegen improvement: i. improved view support on pointwise and transpose scheduler ii. grouped grid welford added for better outer-norm grid persistence in normalization - misc: i. new composite ops added: variance_mean , arange, ii. fixes misaligned address for transpose scheduler iii. refactor on separation of compilation API from execution API to prepare us for async compilation iv. double type support on expression evaluator v. PYTORCH_NVFUSER_DUMP refactor to save PTX and CUBIN Commits that's in this PR from the devel branch: ``` 89330aa Tensor factories must set the output shape as its input (#1939) b2fd01e arange support (#1933) 56c00fd Double support on all expression evaluators (#1937) 371f282 Improve trivial reduction merge support (#1931) 1d0c267 Test `rand` in a fusion with zero tensor input (#1932) 0dab160 Fix softmax bwd sizes. (#1890) ef98f36 Fix a bug (#1936) 63132a0 Propagate permissive mapping information into indexing pass (#1929) b4ac2c8 Map IterationDomains through view operations. (#1919) c0a187a do not use deprecated functions (#1935) 88de85e Upstream cherry pick fixes 0811 (#1934) b247dcf Separate kernel compilation API from kernel execution API (#1914) b34e3b9 Fix `ir_utils::hasBlockSync` + misc fixes in transpose scheduler (#1924) 14a53e6 Nullary RNGOp (#1892) 3c3c89e Misc fixes/tuning for transpose scheduler (#1912) 20cf109 Grouped grid welford (#1921) 6cf7eb0 Transpose scheduler small dim sizes better support (#1910) 9341ea9 Disabled ViewPersistentShmoo sizes that results in NAN (#1922) 057237f Fix CUDA driver error: misaligned address for transpose scheduler (#1918) 3fb3d80 Add variance_mean function using Welford (#1907) 98febf6 Remove DisableOption::UnrollWithRng (#1913) ee8ef33 Minor fix for the debug interface of using PTX directly (#1917) 6e8f953 Add PYTORCH_NVFUSER_DUMP options to save PTX and CUBIN (#1916) 5eefa9a dopt is only available since nvrtc 11.7 (#1915) 2ec8fc7 Kill computeAtBetween (#1911) d0d106a Improve view support on pointwise and transpose scheduler (#1906) e71e1ec Fix name clash of RNG with shared memory (#1904) 3381793 Fix mutator and sameAs for expanded IterDomain (#1902) ``` RUN_TORCHBENCH: nvfuser ghstack-source-id: d0d88cff0c908b2f0ebf6defaab10bc3e7b437b5 Pull Request resolved: #84626

Syncing nvfuser devel branch to upstream master. https://github.com/csarofeen/pytorch/ Codegen changes include: - codegen improvement: i. improved view support on pointwise and transpose scheduler ii. grouped grid welford added for better outer-norm grid persistence in normalization - misc: i. new composite ops added: variance_mean , arange, ii. fixes misaligned address for transpose scheduler iii. refactor on separation of compilation API from execution API to prepare us for async compilation iv. double type support on expression evaluator v. PYTORCH_NVFUSER_DUMP refactor to save PTX and CUBIN Commits that's in this PR from the devel branch: ``` 89330aa Tensor factories must set the output shape as its input (#1939) b2fd01e arange support (#1933) 56c00fd Double support on all expression evaluators (#1937) 371f282 Improve trivial reduction merge support (#1931) 1d0c267 Test `rand` in a fusion with zero tensor input (#1932) 0dab160 Fix softmax bwd sizes. (#1890) ef98f36 Fix a bug (#1936) 63132a0 Propagate permissive mapping information into indexing pass (#1929) b4ac2c8 Map IterationDomains through view operations. (#1919) c0a187a do not use deprecated functions (#1935) 88de85e Upstream cherry pick fixes 0811 (#1934) b247dcf Separate kernel compilation API from kernel execution API (#1914) b34e3b9 Fix `ir_utils::hasBlockSync` + misc fixes in transpose scheduler (#1924) 14a53e6 Nullary RNGOp (#1892) 3c3c89e Misc fixes/tuning for transpose scheduler (#1912) 20cf109 Grouped grid welford (#1921) 6cf7eb0 Transpose scheduler small dim sizes better support (#1910) 9341ea9 Disabled ViewPersistentShmoo sizes that results in NAN (#1922) 057237f Fix CUDA driver error: misaligned address for transpose scheduler (#1918) 3fb3d80 Add variance_mean function using Welford (#1907) 98febf6 Remove DisableOption::UnrollWithRng (#1913) ee8ef33 Minor fix for the debug interface of using PTX directly (#1917) 6e8f953 Add PYTORCH_NVFUSER_DUMP options to save PTX and CUBIN (#1916) 5eefa9a dopt is only available since nvrtc 11.7 (#1915) 2ec8fc7 Kill computeAtBetween (#1911) d0d106a Improve view support on pointwise and transpose scheduler (#1906) e71e1ec Fix name clash of RNG with shared memory (#1904) 3381793 Fix mutator and sameAs for expanded IterDomain (#1902) ``` RUN_TORCHBENCH: nvfuser ghstack-source-id: dec022a19dc6d511da1008b35d88ea1789e933a4 Pull Request resolved: #84626

Syncing nvfuser devel branch to upstream master. https://github.com/csarofeen/pytorch/ Codegen changes include: - codegen improvement: i. improved view support on pointwise and transpose scheduler ii. grouped grid welford added for better outer-norm grid persistence in normalization - misc: i. new composite ops added: variance_mean , arange, ii. fixes misaligned address for transpose scheduler iii. refactor on separation of compilation API from execution API to prepare us for async compilation iv. double type support on expression evaluator v. PYTORCH_NVFUSER_DUMP refactor to save PTX and CUBIN Commits that's in this PR from the devel branch: ``` 89330aa Tensor factories must set the output shape as its input (#1939) b2fd01e arange support (#1933) 56c00fd Double support on all expression evaluators (#1937) 371f282 Improve trivial reduction merge support (#1931) 1d0c267 Test `rand` in a fusion with zero tensor input (#1932) 0dab160 Fix softmax bwd sizes. (#1890) ef98f36 Fix a bug (#1936) 63132a0 Propagate permissive mapping information into indexing pass (#1929) b4ac2c8 Map IterationDomains through view operations. (#1919) c0a187a do not use deprecated functions (#1935) 88de85e Upstream cherry pick fixes 0811 (#1934) b247dcf Separate kernel compilation API from kernel execution API (#1914) b34e3b9 Fix `ir_utils::hasBlockSync` + misc fixes in transpose scheduler (#1924) 14a53e6 Nullary RNGOp (#1892) 3c3c89e Misc fixes/tuning for transpose scheduler (#1912) 20cf109 Grouped grid welford (#1921) 6cf7eb0 Transpose scheduler small dim sizes better support (#1910) 9341ea9 Disabled ViewPersistentShmoo sizes that results in NAN (#1922) 057237f Fix CUDA driver error: misaligned address for transpose scheduler (#1918) 3fb3d80 Add variance_mean function using Welford (#1907) 98febf6 Remove DisableOption::UnrollWithRng (#1913) ee8ef33 Minor fix for the debug interface of using PTX directly (#1917) 6e8f953 Add PYTORCH_NVFUSER_DUMP options to save PTX and CUBIN (#1916) 5eefa9a dopt is only available since nvrtc 11.7 (#1915) 2ec8fc7 Kill computeAtBetween (#1911) d0d106a Improve view support on pointwise and transpose scheduler (#1906) e71e1ec Fix name clash of RNG with shared memory (#1904) 3381793 Fix mutator and sameAs for expanded IterDomain (#1902) ``` RUN_TORCHBENCH: nvfuser Differential Revision: [D39324552](https://our.internmc.facebook.com/intern/diff/D39324552) [ghstack-poisoned]

Syncing nvfuser devel branch to upstream master. https://github.com/csarofeen/pytorch/ Codegen changes include: - codegen improvement: i. improved view support on pointwise and transpose scheduler ii. grouped grid welford added for better outer-norm grid persistence in normalization - misc: i. new composite ops added: variance_mean , arange, ii. fixes misaligned address for transpose scheduler iii. refactor on separation of compilation API from execution API to prepare us for async compilation iv. double type support on expression evaluator v. PYTORCH_NVFUSER_DUMP refactor to save PTX and CUBIN Commits that's in this PR from the devel branch: ``` 89330aa Tensor factories must set the output shape as its input (#1939) b2fd01e arange support (#1933) 56c00fd Double support on all expression evaluators (#1937) 371f282 Improve trivial reduction merge support (#1931) 1d0c267 Test `rand` in a fusion with zero tensor input (#1932) 0dab160 Fix softmax bwd sizes. (#1890) ef98f36 Fix a bug (#1936) 63132a0 Propagate permissive mapping information into indexing pass (#1929) b4ac2c8 Map IterationDomains through view operations. (#1919) c0a187a do not use deprecated functions (#1935) 88de85e Upstream cherry pick fixes 0811 (#1934) b247dcf Separate kernel compilation API from kernel execution API (#1914) b34e3b9 Fix `ir_utils::hasBlockSync` + misc fixes in transpose scheduler (#1924) 14a53e6 Nullary RNGOp (#1892) 3c3c89e Misc fixes/tuning for transpose scheduler (#1912) 20cf109 Grouped grid welford (#1921) 6cf7eb0 Transpose scheduler small dim sizes better support (#1910) 9341ea9 Disabled ViewPersistentShmoo sizes that results in NAN (#1922) 057237f Fix CUDA driver error: misaligned address for transpose scheduler (#1918) 3fb3d80 Add variance_mean function using Welford (#1907) 98febf6 Remove DisableOption::UnrollWithRng (#1913) ee8ef33 Minor fix for the debug interface of using PTX directly (#1917) 6e8f953 Add PYTORCH_NVFUSER_DUMP options to save PTX and CUBIN (#1916) 5eefa9a dopt is only available since nvrtc 11.7 (#1915) 2ec8fc7 Kill computeAtBetween (#1911) d0d106a Improve view support on pointwise and transpose scheduler (#1906) e71e1ec Fix name clash of RNG with shared memory (#1904) 3381793 Fix mutator and sameAs for expanded IterDomain (#1902) ``` RUN_TORCHBENCH: nvfuser ghstack-source-id: 0443a2a479aa4867cdd42dacb6bbecd7e5e43fd8 Pull Request resolved: #84626

Syncing nvfuser devel branch to upstream master. https://github.com/csarofeen/pytorch/ Codegen changes include: - codegen improvement: i. improved view support on pointwise and transpose scheduler ii. grouped grid welford added for better outer-norm grid persistence in normalization - misc: i. new composite ops added: variance_mean , arange, ii. fixes misaligned address for transpose scheduler iii. refactor on separation of compilation API from execution API to prepare us for async compilation iv. double type support on expression evaluator v. PYTORCH_NVFUSER_DUMP refactor to save PTX and CUBIN Commits that's in this PR from the devel branch: ``` 89330aa Tensor factories must set the output shape as its input (#1939) b2fd01e arange support (#1933) 56c00fd Double support on all expression evaluators (#1937) 371f282 Improve trivial reduction merge support (#1931) 1d0c267 Test `rand` in a fusion with zero tensor input (#1932) 0dab160 Fix softmax bwd sizes. (#1890) ef98f36 Fix a bug (#1936) 63132a0 Propagate permissive mapping information into indexing pass (#1929) b4ac2c8 Map IterationDomains through view operations. (#1919) c0a187a do not use deprecated functions (#1935) 88de85e Upstream cherry pick fixes 0811 (#1934) b247dcf Separate kernel compilation API from kernel execution API (#1914) b34e3b9 Fix `ir_utils::hasBlockSync` + misc fixes in transpose scheduler (#1924) 14a53e6 Nullary RNGOp (#1892) 3c3c89e Misc fixes/tuning for transpose scheduler (#1912) 20cf109 Grouped grid welford (#1921) 6cf7eb0 Transpose scheduler small dim sizes better support (#1910) 9341ea9 Disabled ViewPersistentShmoo sizes that results in NAN (#1922) 057237f Fix CUDA driver error: misaligned address for transpose scheduler (#1918) 3fb3d80 Add variance_mean function using Welford (#1907) 98febf6 Remove DisableOption::UnrollWithRng (#1913) ee8ef33 Minor fix for the debug interface of using PTX directly (#1917) 6e8f953 Add PYTORCH_NVFUSER_DUMP options to save PTX and CUBIN (#1916) 5eefa9a dopt is only available since nvrtc 11.7 (#1915) 2ec8fc7 Kill computeAtBetween (#1911) d0d106a Improve view support on pointwise and transpose scheduler (#1906) e71e1ec Fix name clash of RNG with shared memory (#1904) 3381793 Fix mutator and sameAs for expanded IterDomain (#1902) ``` RUN_TORCHBENCH: nvfuser Differential Revision: [D39324552](https://our.internmc.facebook.com/intern/diff/D39324552) [ghstack-poisoned]

Syncing nvfuser devel branch to upstream master. https://github.com/csarofeen/pytorch/ Codegen changes include: - codegen improvement: i. improved view support on pointwise and transpose scheduler ii. grouped grid welford added for better outer-norm grid persistence in normalization - misc: i. new composite ops added: variance_mean , arange, ii. fixes misaligned address for transpose scheduler iii. refactor on separation of compilation API from execution API to prepare us for async compilation iv. double type support on expression evaluator v. PYTORCH_NVFUSER_DUMP refactor to save PTX and CUBIN Commits that's in this PR from the devel branch: ``` 89330aa Tensor factories must set the output shape as its input (#1939) b2fd01e arange support (#1933) 56c00fd Double support on all expression evaluators (#1937) 371f282 Improve trivial reduction merge support (#1931) 1d0c267 Test `rand` in a fusion with zero tensor input (#1932) 0dab160 Fix softmax bwd sizes. (#1890) ef98f36 Fix a bug (#1936) 63132a0 Propagate permissive mapping information into indexing pass (#1929) b4ac2c8 Map IterationDomains through view operations. (#1919) c0a187a do not use deprecated functions (#1935) 88de85e Upstream cherry pick fixes 0811 (#1934) b247dcf Separate kernel compilation API from kernel execution API (#1914) b34e3b9 Fix `ir_utils::hasBlockSync` + misc fixes in transpose scheduler (#1924) 14a53e6 Nullary RNGOp (#1892) 3c3c89e Misc fixes/tuning for transpose scheduler (#1912) 20cf109 Grouped grid welford (#1921) 6cf7eb0 Transpose scheduler small dim sizes better support (#1910) 9341ea9 Disabled ViewPersistentShmoo sizes that results in NAN (#1922) 057237f Fix CUDA driver error: misaligned address for transpose scheduler (#1918) 3fb3d80 Add variance_mean function using Welford (#1907) 98febf6 Remove DisableOption::UnrollWithRng (#1913) ee8ef33 Minor fix for the debug interface of using PTX directly (#1917) 6e8f953 Add PYTORCH_NVFUSER_DUMP options to save PTX and CUBIN (#1916) 5eefa9a dopt is only available since nvrtc 11.7 (#1915) 2ec8fc7 Kill computeAtBetween (#1911) d0d106a Improve view support on pointwise and transpose scheduler (#1906) e71e1ec Fix name clash of RNG with shared memory (#1904) 3381793 Fix mutator and sameAs for expanded IterDomain (#1902) ``` RUN_TORCHBENCH: nvfuser ghstack-source-id: 34c0b924cd95c85c61480be1aa5b755b51187b07 Pull Request resolved: #84626

Syncing nvfuser devel branch to upstream master. https://github.com/csarofeen/pytorch/ Codegen changes include: - codegen improvement: i. improved view support on pointwise and transpose scheduler ii. grouped grid welford added for better outer-norm grid persistence in normalization - misc: i. new composite ops added: variance_mean , arange, ii. fixes misaligned address for transpose scheduler iii. refactor on separation of compilation API from execution API to prepare us for async compilation iv. double type support on expression evaluator v. PYTORCH_NVFUSER_DUMP refactor to save PTX and CUBIN Commits that's in this PR from the devel branch: ``` 89330aa Tensor factories must set the output shape as its input (#1939) b2fd01e arange support (#1933) 56c00fd Double support on all expression evaluators (#1937) 371f282 Improve trivial reduction merge support (#1931) 1d0c267 Test `rand` in a fusion with zero tensor input (#1932) 0dab160 Fix softmax bwd sizes. (#1890) ef98f36 Fix a bug (#1936) 63132a0 Propagate permissive mapping information into indexing pass (#1929) b4ac2c8 Map IterationDomains through view operations. (#1919) c0a187a do not use deprecated functions (#1935) 88de85e Upstream cherry pick fixes 0811 (#1934) b247dcf Separate kernel compilation API from kernel execution API (#1914) b34e3b9 Fix `ir_utils::hasBlockSync` + misc fixes in transpose scheduler (#1924) 14a53e6 Nullary RNGOp (#1892) 3c3c89e Misc fixes/tuning for transpose scheduler (#1912) 20cf109 Grouped grid welford (#1921) 6cf7eb0 Transpose scheduler small dim sizes better support (#1910) 9341ea9 Disabled ViewPersistentShmoo sizes that results in NAN (#1922) 057237f Fix CUDA driver error: misaligned address for transpose scheduler (#1918) 3fb3d80 Add variance_mean function using Welford (#1907) 98febf6 Remove DisableOption::UnrollWithRng (#1913) ee8ef33 Minor fix for the debug interface of using PTX directly (#1917) 6e8f953 Add PYTORCH_NVFUSER_DUMP options to save PTX and CUBIN (#1916) 5eefa9a dopt is only available since nvrtc 11.7 (#1915) 2ec8fc7 Kill computeAtBetween (#1911) d0d106a Improve view support on pointwise and transpose scheduler (#1906) e71e1ec Fix name clash of RNG with shared memory (#1904) 3381793 Fix mutator and sameAs for expanded IterDomain (#1902) ``` RUN_TORCHBENCH: nvfuser Differential Revision: [D39324552](https://our.internmc.facebook.com/intern/diff/D39324552) Pull Request resolved: #84626 Approved by: https://github.com/malfet

Syncing nvfuser devel branch to upstream master. https://github.com/csarofeen/pytorch/ Codegen changes include: - codegen improvement: i. improved view support on pointwise and transpose scheduler ii. grouped grid welford added for better outer-norm grid persistence in normalization - misc: i. new composite ops added: variance_mean , arange, ii. fixes misaligned address for transpose scheduler iii. refactor on separation of compilation API from execution API to prepare us for async compilation iv. double type support on expression evaluator v. PYTORCH_NVFUSER_DUMP refactor to save PTX and CUBIN Commits that's in this PR from the devel branch: ``` 89330aa Tensor factories must set the output shape as its input (pytorch#1939) b2fd01e arange support (pytorch#1933) 56c00fd Double support on all expression evaluators (pytorch#1937) 371f282 Improve trivial reduction merge support (pytorch#1931) 1d0c267 Test `rand` in a fusion with zero tensor input (pytorch#1932) 0dab160 Fix softmax bwd sizes. (pytorch#1890) ef98f36 Fix a bug (pytorch#1936) 63132a0 Propagate permissive mapping information into indexing pass (pytorch#1929) b4ac2c8 Map IterationDomains through view operations. (pytorch#1919) c0a187a do not use deprecated functions (pytorch#1935) 88de85e Upstream cherry pick fixes 0811 (pytorch#1934) b247dcf Separate kernel compilation API from kernel execution API (pytorch#1914) b34e3b9 Fix `ir_utils::hasBlockSync` + misc fixes in transpose scheduler (pytorch#1924) 14a53e6 Nullary RNGOp (pytorch#1892) 3c3c89e Misc fixes/tuning for transpose scheduler (pytorch#1912) 20cf109 Grouped grid welford (pytorch#1921) 6cf7eb0 Transpose scheduler small dim sizes better support (pytorch#1910) 9341ea9 Disabled ViewPersistentShmoo sizes that results in NAN (pytorch#1922) 057237f Fix CUDA driver error: misaligned address for transpose scheduler (pytorch#1918) 3fb3d80 Add variance_mean function using Welford (pytorch#1907) 98febf6 Remove DisableOption::UnrollWithRng (pytorch#1913) ee8ef33 Minor fix for the debug interface of using PTX directly (pytorch#1917) 6e8f953 Add PYTORCH_NVFUSER_DUMP options to save PTX and CUBIN (pytorch#1916) 5eefa9a dopt is only available since nvrtc 11.7 (pytorch#1915) 2ec8fc7 Kill computeAtBetween (pytorch#1911) d0d106a Improve view support on pointwise and transpose scheduler (pytorch#1906) e71e1ec Fix name clash of RNG with shared memory (pytorch#1904) 3381793 Fix mutator and sameAs for expanded IterDomain (pytorch#1902) ``` RUN_TORCHBENCH: nvfuser Differential Revision: [D39324552](https://our.internmc.facebook.com/intern/diff/D39324552) Pull Request resolved: pytorch#84626 Approved by: https://github.com/malfet

soumith added this to Ready in PR Status Jul 3, 2017

soumith moved this from Ready for Review to Newly Ready for Review in PR Status Jul 4, 2017

albanD and others added 3 commits July 22, 2017 01:30

add SharedFunctionMaker to create Function shared in the graph

2ac5670

Clean shared_ptr usage for only function that will be used in the graph

d613077

make Function binding match Varible one

6039735

albanD force-pushed the CFunction_uniqueness branch from f6dde25 to 6039735 Compare July 22, 2017 01:33

remove unnecessary changes

b4a2d7d

colesbury reviewed Jul 22, 2017

View reviewed changes

fix comments

14574ca

apaszke reviewed Jul 23, 2017

View reviewed changes

torch/csrc/autograd/python_cpp_function.cpp Outdated

new (&f->cdata) std::shared_ptr<Function>(cdata);

cdata->pyobj = obj.release();

Py_INCREF(cdata->pyobj);

This comment was marked as off-topic.

Sign in to view

This comment was marked as off-topic.

Sign in to view

proper weakref implementation

35a6da6

colesbury reviewed Jul 25, 2017

View reviewed changes

add call to clear in dealloc

4ce8a27

colesbury approved these changes Jul 25, 2017

View reviewed changes

soumith merged commit 626840a into pytorch:master Jul 25, 2017

soumith removed this from Newly Ready for Review in PR Status Aug 23, 2017

ezyang added the open source label Jun 24, 2019

jjsjann123 pushed a commit to jjsjann123/pytorch that referenced this pull request Aug 30, 2022

Misc fixes/tuning for transpose scheduler (pytorch#1912)

3c3c89e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

C function wrapper uniqueness #1912

C function wrapper uniqueness #1912

albanD commented Jun 26, 2017

colesbury commented Jul 5, 2017 •

edited

albanD commented Jul 11, 2017

colesbury commented Jul 19, 2017

albanD commented Jul 20, 2017

colesbury commented Jul 20, 2017

albanD commented Jul 20, 2017

soumith commented Jul 21, 2017

albanD commented Jul 22, 2017

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

apaszke left a comment

This comment was marked as off-topic.

This comment was marked as off-topic.

soumith commented Jul 24, 2017

albanD commented Jul 25, 2017

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

colesbury left a comment

soumith commented Jul 25, 2017

xmoddroid commented Jul 31, 2017

C function wrapper uniqueness #1912

C function wrapper uniqueness #1912

Conversation

albanD commented Jun 26, 2017

colesbury commented Jul 5, 2017 • edited

albanD commented Jul 11, 2017

colesbury commented Jul 19, 2017

albanD commented Jul 20, 2017

colesbury commented Jul 20, 2017

albanD commented Jul 20, 2017

soumith commented Jul 21, 2017

albanD commented Jul 22, 2017

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

apaszke left a comment

Choose a reason for hiding this comment

This comment was marked as off-topic.

This comment was marked as off-topic.

soumith commented Jul 24, 2017

albanD commented Jul 25, 2017

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

colesbury left a comment

Choose a reason for hiding this comment

soumith commented Jul 25, 2017

xmoddroid commented Jul 31, 2017

colesbury commented Jul 5, 2017 •

edited