[microNPU][ETHOSU] Channel pad offloaded to NPU #14765

sergio-grovety · 2023-05-04T11:28:57Z

A separate channel-dimension nn.pad relay operator is rewritten as Relay concatenate operation.

tvm-bot · 2023-05-04T11:29:01Z

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

cc @Mousius, @leandron _{See #10317 for details}

_{Generated by tvm-bot}

sergio-grovety · 2023-05-04T11:31:37Z

cc @neildhickey, @ekalda, @ilyag-grovety, @Aleksei-grovety @arina-grovety

python/tvm/relay/op/contrib/ethosu.py

Aleksei-grovety · 2023-05-04T13:07:46Z

python/tvm/relay/op/contrib/ethosu.py

+            return None
+        if (
+            list(pad_width[0]) != [0, 0]
+            or list(pad_width[1]) != [0, 0]


Are there networks that have paddings in height, width and channels? If there are such, then it would be possible to remove width and height restrictions and add width and height padding processing to the legalization using depthwise convolution as it is done for pad2d.

Yes, you are right, spatial and channel pad can of course occur in neural networks. This is a separate task, we discussed that it will be useful in the future. We plan to solve it when we have time or a network with such pad appears.

sergio-grovety · 2023-05-06T07:19:48Z

@tvm-bot rerun

ekalda

Thanks @sergio-grovety! Left some comments...

ekalda · 2023-05-16T14:10:42Z

python/tvm/relay/backend/contrib/ethosu/legalize.py

@@ -1447,6 +1447,84 @@ def callback(
        )


+class ChannelPadRewriter(DFPatternCallback):
+    """Convert ethos-u.pad2d composite function to the Relay concatenate operation"""


Suggested change

"""Convert ethos-u.pad2d composite function to the Relay concatenate operation"""

"""Convert ethos-u.channel-pad composite function to the Relay concatenate operation"""

ekalda · 2023-05-16T14:16:06Z

python/tvm/relay/backend/contrib/ethosu/legalize.py

+        params.ifm.tensor = post.args[0]
+
+        concat_args = list()
+        # Activations requiring LUT is currently not supported, so setting it to an empty list


I know every operator here has this copy pasted legacy comment, but let's remove it... Firstly LUT based activations are supported and secondly it could leave an impression that implementing something like fused pad + sigmoid is a TODO.

Got it, thank you.

ekalda · 2023-05-16T14:22:42Z

python/tvm/relay/backend/contrib/ethosu/legalize.py

+        axis = 3
+        return relay.op.concatenate(relay.Tuple(concat_args), axis=axis)


Since it is not used elsewhere, maybe just

Suggested change

axis = 3

return relay.op.concatenate(relay.Tuple(concat_args), axis=axis)

return relay.op.concatenate(relay.Tuple(concat_args), axis=3)

ekalda · 2023-05-16T14:25:46Z

tests/python/contrib/test_ethosu/test_legalize.py

+
+        def are_pad_on_graph(self, subgraph) -> bool:
+            """
+            This function recursively visits the graph and checks if 'nn.pad' op is ongraph


Nit:

Suggested change

This function recursively visits the graph and checks if 'nn.pad' op is ongraph

This function recursively visits the graph and checks if 'nn.pad' op is on graph

ekalda · 2023-05-16T15:08:01Z

tests/python/contrib/test_ethosu/test_legalize.py

+    mod = partition_ethosu_by_table(mod, conv2d_pattern_table)
+
+    mod["tvmgen_default_ethos_u_main_0"] = dataflow_pattern.rewrite(
+        legalize.Conv2DRewriter(), mod["tvmgen_default_ethos_u_main_0"]
+    )
+
+    verify(mod["tvmgen_default_ethos_u_main_0"])


I'm a bit confused about what that test does... It creates a TFLite graph with channel pad and conv2d, then partitions them for microNPU, then legalizes the conv2d into ethosu.conv2d and then checks that the Relay pad is still there?

Hi @ekalda, thank you for the review!
Yes, it is. Here we check that the pad by channel does not merge with conv2d, as it happens with the spatial pad.

ekalda · 2023-05-16T15:15:01Z

tests/python/contrib/test_ethosu/test_legalize.py

+@pytest.mark.parametrize("ifm_shape", [(1, 55, 55, 3), (1, 23, 32, 7)])
+@pytest.mark.parametrize("channel_padding", [(0, 1), (1, 1), (5, 2)])
+@pytest.mark.parametrize("const_value", [0, 5, 125, -5])
+def test_tflite_separate_channel_padding_legalize(ifm_shape, channel_padding, const_value):


Shouldn't this test be using ChannelPadRewriter and then check in verify that the concatenates got created?

Thanks for the comment, the test really did not correspond to the tested functionality. I corrected the test.

…ixes.

sergio-grovety · 2023-05-19T07:36:11Z

@tvm-bot rerun

ekalda

LGTM 🚀

Aleksei-grovety

LGTM!

ekalda · 2023-05-19T15:21:08Z

Thanks @arina-grovety, @sergio-grovety and @Aleksei-grovety, this is now merged!

A separate channel-dimension nn.pad relay operator is rewritten as Relay concatenate operation. --------- Co-authored-by: Sergey Smirnov <89378719+sergey-grovety@users.noreply.github.com> Co-authored-by: arina.naumova <naumova@grovety.com>

[microNPU][ETHOSU] Channel pad offloaded to NPU

741f00b

Aleksei-grovety reviewed May 4, 2023

View reviewed changes

Corrected inaccuracy in comments

ed58b59

sergio-grovety requested a review from Aleksei-grovety May 16, 2023 13:44

ekalda requested changes May 16, 2023

View reviewed changes

arina-grovety added 2 commits May 17, 2023 18:06

Rewrite test test_tflite_separate_channel_padding_legalize(). Minor f…

ff98133

…ixes.

Test test_tflite_separate_channel_padding_legalize() extended

6870f7f

ekalda approved these changes May 19, 2023

View reviewed changes

Aleksei-grovety approved these changes May 19, 2023

View reviewed changes

ekalda merged commit bbfe481 into apache:main May 19, 2023

sergio-grovety deleted the ethosu-channel-pad branch May 20, 2023 07:10

ysh329 mentioned this pull request Jul 12, 2023

[Release] v0.13.0 Release Candidate Notes #15295

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[microNPU][ETHOSU] Channel pad offloaded to NPU #14765

[microNPU][ETHOSU] Channel pad offloaded to NPU #14765

sergio-grovety commented May 4, 2023

tvm-bot commented May 4, 2023 •

edited

Loading

sergio-grovety commented May 4, 2023 •

edited

Loading

Aleksei-grovety May 4, 2023

arina-grovety May 5, 2023 •

edited

Loading

sergio-grovety commented May 6, 2023

ekalda left a comment

ekalda May 16, 2023

ekalda May 16, 2023

arina-grovety May 17, 2023

ekalda May 16, 2023

ekalda May 16, 2023

ekalda May 16, 2023

arina-grovety May 17, 2023

ekalda May 16, 2023

arina-grovety May 18, 2023

sergio-grovety commented May 19, 2023

ekalda left a comment

Aleksei-grovety left a comment

ekalda commented May 19, 2023

	"""Convert ethos-u.pad2d composite function to the Relay concatenate operation"""
	"""Convert ethos-u.channel-pad composite function to the Relay concatenate operation"""

		axis = 3
		return relay.op.concatenate(relay.Tuple(concat_args), axis=axis)

	This function recursively visits the graph and checks if 'nn.pad' op is ongraph
	This function recursively visits the graph and checks if 'nn.pad' op is on graph

[microNPU][ETHOSU] Channel pad offloaded to NPU #14765

[microNPU][ETHOSU] Channel pad offloaded to NPU #14765

Conversation

sergio-grovety commented May 4, 2023

tvm-bot commented May 4, 2023 • edited Loading

sergio-grovety commented May 4, 2023 • edited Loading

Choose a reason for hiding this comment

arina-grovety May 5, 2023 • edited Loading

Choose a reason for hiding this comment

sergio-grovety commented May 6, 2023

ekalda left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sergio-grovety commented May 19, 2023

ekalda left a comment

Choose a reason for hiding this comment

Aleksei-grovety left a comment

Choose a reason for hiding this comment

ekalda commented May 19, 2023

tvm-bot commented May 4, 2023 •

edited

Loading

sergio-grovety commented May 4, 2023 •

edited

Loading

arina-grovety May 5, 2023 •

edited

Loading