New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace thpp::Tensor with ATen Tensor in autograd csrc #2170

Merged

apaszke merged 6 commits into pytorch:master from killeent:aten-perturbation

Jul 28, 2017

Contributor

killeent commented Jul 20, 2017 •

edited

Loading

As title - moves to using the ATen tensor to wrap TH Tensors for use in autograd. Includes some fixes to ATen discovered by this refactor.

Contributor

ezyang commented Jul 20, 2017

Mucho thanks!

ezyang referenced this pull request in ezyang/pytorch-unattached


          Fix ATen failure to retain memory.

2f2e40e

CC @killeent, I just sliced this off #2170.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

soumith added this to In Progresss in PR Status

ezyang referenced this pull request in ezyang/pytorch-unattached


          Fix ATen failure to retain memory.

3a9f399

CC @killeent, I just sliced this off #2170.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

killeent force-pushed the aten-perturbation branch 4 times, most recently from 686f119 to b656a67 Compare

July 26, 2017 19:36

killeent closed this

killeent reopened this

ezyang referenced this pull request in ezyang/pytorch-unattached


          Fix ATen failure to retain memory.

e2b04bb

CC @killeent, I just sliced this off #2170.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

killeent force-pushed the aten-perturbation branch from b656a67 to f919a21 Compare

July 26, 2017 20:03

apaszke reviewed

View reviewed changes

Contributor

apaszke left a comment

I reviewed the autograd changes and they look good. I don't know why did you remove all these std::moves. They are no longer necessary with at::Tensors, but they should still be valid, and improve performance

torch/csrc/autograd/functions/batch_normalization.cpp Outdated

    
                      training,

                      momentum,

                      eps);

                }

                auto outputs = as_tensor_list(std::move(output));

                auto outputs = as_tensor_list(output);

This comment was marked as off-topic.

Sign in to view

torch/csrc/autograd/functions/convolution.cpp Outdated

    
            @@ -20,24 +21,24 @@ using namespace torch::cudnn;
          
              #endif

              using namespace torch::nn;

              using thpp::Tensor;

              // using thpp::Tensor;

This comment was marked as off-topic.

Sign in to view

torch/csrc/autograd/functions/convolution.cpp Outdated

    
              static at::Tensor subtensor(at::Tensor& tensor, int dim, int groups, int g) {

                if (!tensor.defined()) {

                  at::Tensor empty;

                  return empty;

This comment was marked as off-topic.

Sign in to view

This comment was marked as off-topic.

Sign in to view

torch/csrc/autograd/functions/convolution.cpp Outdated

    
                int num_inputs = tensors.size();

                if (num_inputs == 0) {

                  return nullptr;

                  at::Tensor empty;

                  return empty;

This comment was marked as off-topic.

Sign in to view

torch/csrc/autograd/functions/convolution.cpp Outdated

    
                }

                std::vector<Tensor*> ptrs(num_inputs);

                std::vector<at::Tensor> ptrs(num_inputs);

This comment was marked as off-topic.

Sign in to view

torch/csrc/autograd/functions/convolution.cpp

    
                std::unique_ptr<thpp::Tensor> bias {bias_var ? bias_var->data->clone_shallow() : nullptr};

                auto input = input_var->data;

                auto weight = weight_var->data;

                auto bias = bias_var ? bias_var->data : at::Tensor();

This comment was marked as off-topic.

Sign in to view

This comment was marked as off-topic.

Sign in to view

This comment was marked as off-topic.

Sign in to view

torch/csrc/autograd/functions/tensor.cpp Outdated

    
                return wrap_outputs(inputs, as_tensor_list(std::move(output)), [&](FunctionFlags f) {

                return wrap_outputs(inputs, as_tensor_list(output), [&](FunctionFlags f) {

This comment was marked as off-topic.

Sign in to view

zdevito approved these changes

View reviewed changes

Contributor

zdevito left a comment

I reviewed the aten-specific changes needed here and they look good to me! The sparse handling is good enough to get us started and will serve as a good base for adding more support later.

torch/lib/ATen/templates/TypeDerived.cpp Outdated

    
              // Calls retain on th_pointer

              Tensor ${Type}::unsafeTensorFromTH(void * th_pointer, bool retain) {

                if (retain)

                  ${THTensor}_retain(${state,} (${THTensor}*) th_pointer);

                return Tensor(new ${Tensor}(context,(${THTensor}*)(th_pointer)));

This comment was marked as off-topic.

Sign in to view


          aten in autograd

37ca04e

killeent force-pushed the aten-perturbation branch 3 times, most recently from ae9be8b to 04a91ec Compare

July 27, 2017 13:56


          address pr comments

0135c81

killeent force-pushed the aten-perturbation branch from 04a91ec to 0135c81 Compare

July 27, 2017 14:20

apaszke and others added 2 commits

July 27, 2017 12:20


          Fix OS X build error

0ab247d


          fix sparse dense add

d484637

killeent changed the title ~~[WIP] [DO NOT REVIEW] aten in autograd~~ Replace thpp::Tensor with ATen Tensor in autograd csrc


          autopep8

47c5517

Contributor

ezyang commented Jul 27, 2017

I think the easiest way to make the Travis build pass is if we revert #2214 and investigate why Trusty gcc is flaking like this out-of-band. We've seen this error intermittently when developing on trusty in the jit branch and haven't nailed down why.

apaszke force-pushed the aten-perturbation branch 5 times, most recently from 808fee4 to 81ff9e9 Compare

July 27, 2017 19:26


          Fix travis OOM issues

3e4ce98

apaszke force-pushed the aten-perturbation branch from 81ff9e9 to 3e4ce98 Compare

July 27, 2017 19:43

Contributor

ezyang commented Jul 27, 2017

Uh oh, python 3 Jenkins has an intermittent segfault :(

apaszke closed this

apaszke reopened this

apaszke merged commit c304d04 into pytorch:master

ruotianluo added a commit to ruotianluo/pytorch-1 that referenced this pull request


          Merge commit '8262920b72374b1d9643f35057663ab02ab20330' into use_omp2

d678803

* commit '8262920b72374b1d9643f35057663ab02ab20330':
  Add ATen overload to AutoGPU. (pytorch#2234)
  Add comments for default value (pytorch#2242)
  Remove dead THPP code that has been replaced with ATen objects. (pytorch#2235)
  fix a bug where an uninitialized at::Tensor was passed to createPyObject (pytorch#2239)
  Replace thpp::Tensor with ATen Tensor in autograd csrc (pytorch#2170)
  Added aarch64 support (pytorch#2226)

ruotianluo added a commit to ruotianluo/pytorch-1 that referenced this pull request


          Merge commit '8262920b72374b1d9643f35057663ab02ab20330' into use_omp3

b231ead

* commit '8262920b72374b1d9643f35057663ab02ab20330': (272 commits)
  Add ATen overload to AutoGPU. (pytorch#2234)
  Add comments for default value (pytorch#2242)
  Remove dead THPP code that has been replaced with ATen objects. (pytorch#2235)
  fix a bug where an uninitialized at::Tensor was passed to createPyObject (pytorch#2239)
  Replace thpp::Tensor with ATen Tensor in autograd csrc (pytorch#2170)
  Added aarch64 support (pytorch#2226)
  Increase tol. for float tensor qr big test.
  Improve Variable.retain_grad
  add `retain_grad` method, to variable, so gradient gets stored during backpop, on non-user variables
  Implement BatchNorm double backwards (pytorch#2207)
  [bugfix] in bce_with_logits logsumexp calculation (pytorch#2221)
  fix for ATen API Change
  Opt into Trusty builds. (pytorch#2214)
  allow retain to be specified for unsafeTensorFromTH
  Deduplicate THPUtils_checkLong/THPUtils_unpackLong (pytorch#2218)
  fix osx build errors related to long/int64_t
  Note [Undefined-dim versus 0-dim]
  Remove __func__ hack in auto nn.
  Enable Conv groups gradgradchecks. (pytorch#2216)
  fix a bug where some scalars were getting truncated to integers incorrectly.
  ...

ruotianluo added a commit to ruotianluo/pytorch-1 that referenced this pull request


          Merge commit '8262920b72374b1d9643f35057663ab02ab20330' into use_omp1

8e92db3

* commit '8262920b72374b1d9643f35057663ab02ab20330': (272 commits)
  Add ATen overload to AutoGPU. (pytorch#2234)
  Add comments for default value (pytorch#2242)
  Remove dead THPP code that has been replaced with ATen objects. (pytorch#2235)
  fix a bug where an uninitialized at::Tensor was passed to createPyObject (pytorch#2239)
  Replace thpp::Tensor with ATen Tensor in autograd csrc (pytorch#2170)
  Added aarch64 support (pytorch#2226)
  Increase tol. for float tensor qr big test.
  Improve Variable.retain_grad
  add `retain_grad` method, to variable, so gradient gets stored during backpop, on non-user variables
  Implement BatchNorm double backwards (pytorch#2207)
  [bugfix] in bce_with_logits logsumexp calculation (pytorch#2221)
  fix for ATen API Change
  Opt into Trusty builds. (pytorch#2214)
  allow retain to be specified for unsafeTensorFromTH
  Deduplicate THPUtils_checkLong/THPUtils_unpackLong (pytorch#2218)
  fix osx build errors related to long/int64_t
  Note [Undefined-dim versus 0-dim]
  Remove __func__ hack in auto nn.
  Enable Conv groups gradgradchecks. (pytorch#2216)
  fix a bug where some scalars were getting truncated to integers incorrectly.
  ...

csarofeen mentioned this pull request

RuntimeError: BatchNormalization_updateOutput is not implemented for type CUDAHalfType #2435

Closed

soumith removed this from In Progresss in PR Status

ezyang added the open source label

houseroad mentioned this pull request

Automatic update of fbcode/onnx to 707064980b9825b8705b9d1c9aad34d8b022d5dd #22981

Closed

houseroad added a commit to houseroad/pytorch that referenced this pull request


          Automatic update of fbcode/onnx to 707064980b9825b8705b9d1c9aad34d8b0…

8dd4819

…22d5dd

Summary:
Previous import was 806aa863020fa180e57f576cb032ec44ce8ddcca

Included changes:
- **[70706498](onnx/onnx@70706498)**: TensorProto::INT8 & INT16 were missed here (pytorch#2164) <ZINEKS>
- **[8218a4ea](onnx/onnx@8218a4ea)**: Fix LabelEncoder's shape inference (pytorch#2170) <Wei-Sheng Chin>
- **[0f1a9a1c](onnx/onnx@0f1a9a1c)**: Fixing a unit test in Cumsum Operator (pytorch#2157) <Jeff Saremi>
- **[2c03cff0](onnx/onnx@2c03cff0)**: [New Operator] CumSum (pytorch#2030) <Jeff Saremi>
- **[220b8300](onnx/onnx@220b8300)**: Fix globalpool output shape (pytorch#2147) <daquexian>

Differential Revision: D16341736

fbshipit-source-id: fcd136f20eadfd3a78ae2cd928a4d6eb7c709f4c

facebook-github-bot pushed a commit that referenced this pull request


          Automatic update of fbcode/onnx to 707064980b9825b8705b9d1c9aad34d8b0…

796a39b

…22d5dd (#22981)

Summary:
Pull Request resolved: #22981

Previous import was 806aa863020fa180e57f576cb032ec44ce8ddcca

Included changes:
- **[70706498](onnx/onnx@70706498)**: TensorProto::INT8 & INT16 were missed here (#2164) <ZINEKS>
- **[8218a4ea](onnx/onnx@8218a4ea)**: Fix LabelEncoder's shape inference (#2170) <Wei-Sheng Chin>
- **[0f1a9a1c](onnx/onnx@0f1a9a1c)**: Fixing a unit test in Cumsum Operator (#2157) <Jeff Saremi>
- **[2c03cff0](onnx/onnx@2c03cff0)**: [New Operator] CumSum (#2030) <Jeff Saremi>
- **[220b8300](onnx/onnx@220b8300)**: Fix globalpool output shape (#2147) <daquexian>

Reviewed By: benoitsteiner

Differential Revision: D16341736

fbshipit-source-id: 7e7a2684d8c821991231bfd6558f9f6cb4fb05fb

azazhu pushed a commit to azazhu/pytorch that referenced this pull request


          Refactor lower_alias_memory.cpp (pytorch#2170)

f588435

* Refactoring of lower_alias_memory

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment