[Tosa] Add legalization of BroadcastTo #60692

Tai78641 · 2023-05-24T15:49:39Z

For tf/tfl boradcast-to operator with input and shape where:

shape is compile time constant, and
shape's rank is greater than or equal to input's rank, and
input element type is not complex, and
input element type is not integer whose bitwidth is greater than 32

will convert to tosa operators as follows:

if input element type is floating point, add input with constant -0.f of the broadcast shape
if input element type is i1, logical-or input with constant 'false' of the broadcast shape
if input element type is i32, add input with constant 0 (i32) of the broadcast shape
otherwise, cast input to i32, add with constant 0 (i32) of the broadcast shape, and cast back to original element type

added tf/tfl lit tests

Change-Id: I12302adcf1c791d452a5b5a928e63e5ffcd523bc

gbaned · 2023-06-12T06:09:52Z

Hi @Tai78641 Can you please take a look on below internal errors. Thank you!

TensorFlow crashed, please file a bug on https://github.com/tensorflow/tensorflow/issues with the trace below.
#0 0x0000562cb79c0f6e llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) ()
#1 0x0000562cb79bee29 llvm::sys::RunSignalHandlers()
#29 0x00007f49cbc1d69f clone (/usr/grte/v5/lib64/libc.so.6+0x13969f)
FileCheck error: '' is empty.
FileCheck command line: FileCheck /tensorflow/compiler/mlir/tosa/tests/tf-to-tosa-pipeline.mlir

Tai78641 · 2023-06-12T22:09:12Z

@gbaned I rebased and double checked. tf-to-tosa-pipeline.mlir passes for me.

Tai78641 · 2023-06-21T02:42:13Z

@jpienaar I cannot figure out why the Py+CPP Test Suite failures.

Tai78641 · 2023-06-30T19:41:00Z

rebased to use dyn_cast(...) style

jpienaar · 2023-07-10T18:40:59Z

tensorflow/compiler/mlir/tosa/transforms/legalize_common.cc

+  int32_t num_elements = 1;
+  SmallVector<int64_t> new_shape;
+  for (int i = 0; i < shape_rank; i++) {
+    auto shape_dim = shape_elems.getValues<IntegerAttr>()[i].getInt();


Has TOSA been moved to properties for shape?

not sure I understand the question, which probably means the answer is no?

I think Jacques is referring to using shape_elems. Generally we try to use the getType() value as its "assumed to be true" value. Obviously this is TFLite so either the shape_elems or the getType() could be under defined. I would recommend pulling both and consolidating the correct value.

I forgot this was TFL, I was thinking about requiring IntegerAttr contruction here which could be avoided by properties.

rsuderman · 2023-07-10T21:22:36Z

tensorflow/compiler/mlir/tosa/transforms/legalize_common.cc

+
+  if (element_type.isa<FloatType>()) {
+    // F32: legalize to broadcastable Add with (-0.f)
+    std::vector<float> values(num_elements, -0.f);


For consistency you should change over to llvm::SmallVector>

rsuderman · 2023-07-10T21:24:13Z

tensorflow/compiler/mlir/tosa/transforms/legalize_common.cc

+  int32_t num_elements = 1;
+  SmallVector<int64_t> new_shape;
+  for (int i = 0; i < shape_rank; i++) {
+    auto shape_dim = shape_elems.getValues<IntegerAttr>()[i].getInt();


I think Jacques is referring to using shape_elems. Generally we try to use the getType() value as its "assumed to be true" value. Obviously this is TFLite so either the shape_elems or the getType() could be under defined. I would recommend pulling both and consolidating the correct value.

For tf/tfl broadcast-to operator with input and shape where: - shape is compile time constant, and - shape's rank is greater than or equal to input's rank, and - input element type is not complex, and - input element type is not integer whose bitwidth is greater than 32 will convert to tosa operators as follows: 1. if input element type is floating point, add input with constant -0.f of the broadcast shape 2. if input element type is i1, logical-or input with constant 'false' of the broadcast shape 3. if input element type is i32, add input with constant 0 (i32) of the broadcast shape 4. otherwise, cast input to i32, add with constant 0 (i32) of the broadcast shape, and cast back to original element type added tf/tfl lit tests Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I12302adcf1c791d452a5b5a928e63e5ffcd523bc

Tai78641 · 2023-07-11T20:52:25Z

tensorflow/compiler/mlir/tosa/transforms/legalize_common.cc

+            op,
+            "shape's constant value has different elements than its static "
+            "dimension");
+        return std::nullopt;


added this check that shape_elems (derived from compile time constant for input "shape") is consistent with the getType() for input "shape".

Should this be part of the op verification? (I mean I think this is TFL side, so you could add a TODO here and I could ping folks)

Tai78641 · 2023-07-18T15:59:13Z

@rsuderman please have a look at this again. thanks

jpienaar · 2023-07-18T16:38:40Z

tensorflow/compiler/mlir/tosa/transforms/legalize_common.cc

+            op,
+            "shape's constant value has different elements than its static "
+            "dimension");
+        return std::nullopt;


Should this be part of the op verification? (I mean I think this is TFL side, so you could add a TODO here and I could ping folks)

jpienaar · 2023-07-18T16:40:34Z

tensorflow/compiler/mlir/tosa/transforms/legalize_common.cc

+    // reshape input to shape_rank
+    SmallVector<int64_t> reshaped_shape((shape_rank - input_rank), 1);
+    for (auto dim : input_type.getShape()) {
+      reshaped_shape.push_back(dim);


You could also use insert method on SmallVector. (append_range helper in LLVM could also be useful)

gbaned · 2023-07-20T09:31:15Z

Hi @Tai78641 Can you please check @jpienaar's comments and keep us posted ? Thank you!

rsuderman · 2023-07-24T22:21:20Z

I had to make some modifications to land. I believe it should still be good but you may need to validate that things are working as intended.

google-ml-butler bot added the size:L CL Change Size: Large label May 24, 2023

google-ml-butler bot assigned gbaned May 24, 2023

github-actions bot added the kokoro:force-run Tests on submitted change label May 24, 2023

kokoro-team removed the kokoro:force-run Tests on submitted change label May 24, 2023

gbaned added this to Assigned Reviewer in PR Queue via automation May 25, 2023

gbaned requested a review from jpienaar May 25, 2023 11:59

google-ml-butler bot added the awaiting review Pull request awaiting review label May 25, 2023

Tai78641 force-pushed the pr_broadcast_to branch from fff6902 to 8da0835 Compare June 1, 2023 01:26

github-actions bot added the kokoro:force-run Tests on submitted change label Jun 1, 2023

kokoro-team removed the kokoro:force-run Tests on submitted change label Jun 1, 2023

jpienaar approved these changes Jun 1, 2023

View reviewed changes

PR Queue automation moved this from Assigned Reviewer to Approved by Reviewer Jun 1, 2023

google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Jun 1, 2023

kokoro-team removed the kokoro:force-run Tests on submitted change label Jun 1, 2023

Tai78641 force-pushed the pr_broadcast_to branch from 8da0835 to 3ed47c9 Compare June 12, 2023 17:13

google-ml-butler bot removed the ready to pull PR ready for merge process label Jun 12, 2023

github-actions bot added the kokoro:force-run Tests on submitted change label Jun 12, 2023

kokoro-team removed the kokoro:force-run Tests on submitted change label Jun 12, 2023

gbaned requested a review from jpienaar June 16, 2023 15:40

Tai78641 force-pushed the pr_broadcast_to branch from 3ed47c9 to 0f18bab Compare June 30, 2023 19:39

github-actions bot added the kokoro:force-run Tests on submitted change label Jun 30, 2023

kokoro-team removed the kokoro:force-run Tests on submitted change label Jun 30, 2023

jpienaar reviewed Jul 10, 2023

View reviewed changes

Tai78641 force-pushed the pr_broadcast_to branch from 0f18bab to c650975 Compare July 10, 2023 20:19

github-actions bot added the kokoro:force-run Tests on submitted change label Jul 10, 2023

kokoro-team removed kokoro:force-run Tests on submitted change labels Jul 10, 2023

rsuderman suggested changes Jul 10, 2023

View reviewed changes

PR Queue automation moved this from Approved by Reviewer to Reviewer Requested Changes Jul 10, 2023

Tai78641 force-pushed the pr_broadcast_to branch from c650975 to eb9e49a Compare July 11, 2023 20:48

github-actions bot added the kokoro:force-run Tests on submitted change label Jul 11, 2023

kokoro-team removed the kokoro:force-run Tests on submitted change label Jul 11, 2023

Tai78641 commented Jul 11, 2023

View reviewed changes

Tai78641 requested a review from rsuderman July 11, 2023 20:53

jpienaar approved these changes Jul 18, 2023

View reviewed changes

google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Jul 18, 2023

kokoro-team removed the kokoro:force-run Tests on submitted change label Jul 18, 2023

copybara-service bot merged commit aec0be7 into tensorflow:master Jul 24, 2023
10 of 14 checks passed

PR Queue automation moved this from Reviewer Requested Changes to Merged Jul 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Tosa] Add legalization of BroadcastTo #60692

[Tosa] Add legalization of BroadcastTo #60692

Tai78641 commented May 24, 2023 •

edited

gbaned commented Jun 12, 2023

Tai78641 commented Jun 12, 2023

Tai78641 commented Jun 21, 2023

Tai78641 commented Jun 30, 2023

jpienaar Jul 10, 2023

Tai78641 Jul 10, 2023

rsuderman Jul 10, 2023

jpienaar Jul 11, 2023

rsuderman Jul 10, 2023

Tai78641 Jul 11, 2023

rsuderman Jul 10, 2023

Tai78641 Jul 11, 2023

jpienaar Jul 18, 2023

Tai78641 commented Jul 18, 2023

jpienaar Jul 18, 2023

jpienaar Jul 18, 2023

gbaned commented Jul 20, 2023

rsuderman commented Jul 24, 2023

[Tosa] Add legalization of BroadcastTo #60692

[Tosa] Add legalization of BroadcastTo #60692

Conversation

Tai78641 commented May 24, 2023 • edited

gbaned commented Jun 12, 2023

Tai78641 commented Jun 12, 2023

Tai78641 commented Jun 21, 2023

Tai78641 commented Jun 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Tai78641 commented Jul 18, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gbaned commented Jul 20, 2023

rsuderman commented Jul 24, 2023

Tai78641 commented May 24, 2023 •

edited