Add support for aten::remainder.Tensor_out for MPS backend #87582

yash-dani · 2022-10-24T01:54:59Z

Fixes #86806

pytorch-bot · 2022-10-24T01:55:03Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/87582

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 Failures

As of commit 5219485:

The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

linux-foundation-easycla · 2022-10-24T01:55:03Z

❌ - login: @yash-dani / name: yash . The commit (5219485) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.

gpomeranz · 2022-11-06T12:42:41Z

Hi @yash-dani , thanks so much for working on this! I seem to be having this exact issue when running pytorch on my M1, but the build you suggested is failing, same as above. Any chacne you could have a second to look at it?

Thank you so much for doing this :)

Kind regards,
Gideon

kulinseth · 2022-11-18T05:26:40Z

aten/src/ATen/native/mps/operations/UnaryOps.mm

@@ -231,5 +231,24 @@ void unary_op(const Tensor& self, const Tensor& output, std::string op_name, Una
                });
 }

+
+TORCH_IMPL_FUNC(remainder_out_mps) (const Tensor& self, const Tensor& output) {


This function definition is not correct. Infact it should be in BinaryOps.mm.
Please take a look at the formula to implement:
https://pytorch.org/docs/stable/generated/torch.remainder.html

kulinseth · 2022-11-18T07:14:07Z

I have attached the patch , which should work.
patch.txt

kulinseth · 2022-11-18T07:17:53Z

@yash-dani , can you rebase and apply the patch.

gpomeranz · 2022-11-22T15:10:12Z

Hi @kulinseth not sure if it would help but maybe I can take this over? I tried to apply the patch to a git copy of the current pytorch repository but was not able to compile on my M1, but happy to fork my own pytorch and apply this patch?

Kind regards,
Gideon

gpomeranz · 2022-11-22T15:32:24Z

I've also send a pull request to @yash-dani where I rebased his repo and added the patch.

yash-dani · 2022-11-23T14:01:51Z

I can rebase and merge!

yash-dani · 2022-11-29T04:12:44Z

Hey @kulinseth @gpomeranz your patch gives me the following compile error:

error: patch failed: aten/src/ATen/native/mps/operations/BinaryOps.mm:330
error: aten/src/ATen/native/mps/operations/BinaryOps.mm: patch does not apply
error: patch failed: aten/src/ATen/native/native_functions.yaml:8802
error: aten/src/ATen/native/native_functions.yaml: patch does not apply
error: patch failed: test/test_mps.py:8143
error: test/test_mps.py: patch does not apply

What would be the best way to resolve? Thanks!

kulinseth · 2022-11-30T08:18:14Z

@yash-dani , seems like the patch and your branch has diverged since I had created that patch.
Can you try manually just applying the changes I suggested in the patch?

--- a/aten/src/ATen/native/mps/operations/BinaryOps.mm
+++ b/aten/src/ATen/native/mps/operations/BinaryOps.mm
@@ -330,5 +330,29 @@ Tensor floor_divide_mps(const Tensor& self, const Tensor& other) {
   mps::binaryOpTensor(self, other, Scalar(1.0), output, "logaddexp2_out_mps", logaddexp2_op_block);
 }
 
+TORCH_IMPL_FUNC(remainder_out_mps) (const Tensor& self, const Tensor& other, const Tensor& output) {
+  // torch.remainder(a, b) == a - a.div(b, rounding_mode="floor") * b
+  mps::BinaryOpBlock remainder_op_block = ^BinaryOpFn(cachedGraph, primaryCastTensor, secondaryCastTensor) {
+    MPSGraph* mpsGraph = cachedGraph->graph();
+    // Rounding is a no-op for integral types, and also a reasonable workaround
+    // For MPSGraph bug on Apple Silicon, that throws `Function floorOp_i64 was not found in the library`
+    // See https://github.com/pytorch/pytorch/issues/84995
+    auto divTensor =  [mpsGraph divisionWithPrimaryTensor:primaryCastTensor
+                                          secondaryTensor:secondaryCastTensor
+                                                     name:nil];
+    //if (([divTensor dataType] & MPSDataTypeFloatBit) != 0) {
+      divTensor =  [mpsGraph floorWithTensor:divTensor name:nil];
+    //}
+    auto mulTensor = [mpsGraph multiplicationWithPrimaryTensor:divTensor
+                                               secondaryTensor:secondaryCastTensor
+                                                          name:nil];
+    return [mpsGraph subtractionWithPrimaryTensor:primaryCastTensor
+                                       secondaryTensor:mulTensor
+                                           name: nil];
+    };
+  mps::binaryOpTensor(self, other, Scalar(1.0), output, "remainder_out_mps", remainder_op_block);
+}
+
+

Basically it need to be added to the BinaryOps.mm

kulinseth · 2022-11-30T08:18:56Z

Thanks @gpomeranz for giving it a try? What issue did you run into ?

gpomeranz · 2022-11-30T15:03:22Z

@kulinseth I tried to manually apply the patches in the latest branch of pytorch, as I've never applied a patch to a github repo before. I unfortunately was not able to compile pytorch, so it stopped for me there. I don't remember what the error was but I believe it has something to do with my clang compiler (even though I use the latest brew one). I will try to redownload pytorch and apply the patch and will post here what my error was.

I think if @yash-dani, who has more experience, manually adds the patch and tries to compile it will succeed.

gpomeranz · 2022-11-30T20:18:47Z

This is the error that I get when manually adding the patch and compiling pytorch:

/opt/homebrew/opt/llvm/bin/../include/c++/v1/__algorithm/iterator_operations.h:131:12: error: calling a private constructor of class 'c10::impl::ListElementReference<at::Tensor, std::__wrap_iter<c10::IValue *>>'
return *std::forward<_Iter>(__i);

jinsu35 · 2023-01-13T07:18:29Z

@yash-dani , seems like the patch and your branch has diverged since I had created that patch. Can you try manually just applying the changes I suggested in the patch?

--- a/aten/src/ATen/native/mps/operations/BinaryOps.mm
+++ b/aten/src/ATen/native/mps/operations/BinaryOps.mm
@@ -330,5 +330,29 @@ Tensor floor_divide_mps(const Tensor& self, const Tensor& other) {
   mps::binaryOpTensor(self, other, Scalar(1.0), output, "logaddexp2_out_mps", logaddexp2_op_block);
 }
 
+TORCH_IMPL_FUNC(remainder_out_mps) (const Tensor& self, const Tensor& other, const Tensor& output) {
+  // torch.remainder(a, b) == a - a.div(b, rounding_mode="floor") * b
+  mps::BinaryOpBlock remainder_op_block = ^BinaryOpFn(cachedGraph, primaryCastTensor, secondaryCastTensor) {
+    MPSGraph* mpsGraph = cachedGraph->graph();
+    // Rounding is a no-op for integral types, and also a reasonable workaround
+    // For MPSGraph bug on Apple Silicon, that throws `Function floorOp_i64 was not found in the library`
+    // See https://github.com/pytorch/pytorch/issues/84995
+    auto divTensor =  [mpsGraph divisionWithPrimaryTensor:primaryCastTensor
+                                          secondaryTensor:secondaryCastTensor
+                                                     name:nil];
+    //if (([divTensor dataType] & MPSDataTypeFloatBit) != 0) {
+      divTensor =  [mpsGraph floorWithTensor:divTensor name:nil];
+    //}
+    auto mulTensor = [mpsGraph multiplicationWithPrimaryTensor:divTensor
+                                               secondaryTensor:secondaryCastTensor
+                                                          name:nil];
+    return [mpsGraph subtractionWithPrimaryTensor:primaryCastTensor
+                                       secondaryTensor:mulTensor
+                                           name: nil];
+    };
+  mps::binaryOpTensor(self, other, Scalar(1.0), output, "remainder_out_mps", remainder_op_block);
+}
+
+

Basically it need to be added to the BinaryOps.mm

You do need this conditional block to handle a MPS bug.

if (([divTensor dataType] & MPSDataTypeFloatBit) != 0) {
    divTensor =  [mpsGraph floorWithTensor:divTensor name:nil];
}

I applied the patch to the latest of master in PR #92139

gpomeranz · 2023-01-13T12:31:51Z

I can confirm that the above change indeed fixes the build problems and I was able to build pytorch from source using this code addition to the latest release.

Looking forward to have this incorporated so that I can run my gpu accelerated code in R!

vishank97 · 2023-01-19T06:15:19Z

Hi,
Any update on the fix? I'm still getting the error and I don't if there's a fix available yet. @yash-dani @gpomeranz @kulinseth

kulinseth · 2023-03-09T18:44:17Z

We have pulled in the remainder PR.

kulinseth · 2023-03-09T18:44:41Z

@yash-dani , please re-open if you think we missed something.

yash-dani requested a review from kulinseth as a code owner October 24, 2022 01:55

pytorch-bot bot added ciflow/mps Run MPS tests (subset of trunk) release notes: mps Release notes category labels Oct 24, 2022

pytorchbot added the open source label Oct 24, 2022

bdhirsh added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Oct 27, 2022

kulinseth mentioned this pull request Nov 17, 2022

Use MPS device when available DLR-RM/stable-baselines3#951

Open

14 tasks

kulinseth requested changes Nov 18, 2022

View reviewed changes

initial setup

5219485

yash-dani force-pushed the master branch from b6a8f3b to 5219485 Compare November 29, 2022 04:03

kulinseth closed this Mar 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for aten::remainder.Tensor_out for MPS backend #87582

Add support for aten::remainder.Tensor_out for MPS backend #87582

yash-dani commented Oct 24, 2022

pytorch-bot bot commented Oct 24, 2022 •

edited

Loading

linux-foundation-easycla bot commented Oct 24, 2022 •

edited

Loading

gpomeranz commented Nov 6, 2022

kulinseth Nov 18, 2022

kulinseth commented Nov 18, 2022

kulinseth commented Nov 18, 2022

gpomeranz commented Nov 22, 2022

gpomeranz commented Nov 22, 2022

yash-dani commented Nov 23, 2022

yash-dani commented Nov 29, 2022

kulinseth commented Nov 30, 2022

kulinseth commented Nov 30, 2022

gpomeranz commented Nov 30, 2022

gpomeranz commented Nov 30, 2022

jinsu35 commented Jan 13, 2023

gpomeranz commented Jan 13, 2023

vishank97 commented Jan 19, 2023

kulinseth commented Mar 9, 2023

kulinseth commented Mar 9, 2023

Add support for aten::remainder.Tensor_out for MPS backend #87582

Add support for aten::remainder.Tensor_out for MPS backend #87582

Conversation

yash-dani commented Oct 24, 2022

pytorch-bot bot commented Oct 24, 2022 • edited Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/87582

❌ 2 Failures

linux-foundation-easycla bot commented Oct 24, 2022 • edited Loading

gpomeranz commented Nov 6, 2022

kulinseth Nov 18, 2022

Choose a reason for hiding this comment

kulinseth commented Nov 18, 2022

kulinseth commented Nov 18, 2022

gpomeranz commented Nov 22, 2022

gpomeranz commented Nov 22, 2022

yash-dani commented Nov 23, 2022

yash-dani commented Nov 29, 2022

kulinseth commented Nov 30, 2022

kulinseth commented Nov 30, 2022

gpomeranz commented Nov 30, 2022

gpomeranz commented Nov 30, 2022

jinsu35 commented Jan 13, 2023

gpomeranz commented Jan 13, 2023

vishank97 commented Jan 19, 2023

kulinseth commented Mar 9, 2023

kulinseth commented Mar 9, 2023

pytorch-bot bot commented Oct 24, 2022 •

edited

Loading

linux-foundation-easycla bot commented Oct 24, 2022 •

edited

Loading