[microNPU] Fix typo in depthwise to allow int8 weights #9299

lhutton1 · 2021-10-15T13:53:15Z

Small typo means that ifm datatype is checked when it should be weight.

Mousius · 2021-10-15T14:25:25Z

hi @lhutton1, thanks for fixing this. Is it possible to wrap this in a test case?

ekalda · 2021-10-15T15:23:06Z

Thanks @lhutton1 for fixing this, well spotted! As of a test case, I think all of our NPU tests exercise it since all of our weights in the tests are int8 (before that fix this check didn't actually check the weights dtype correctly)

Mousius · 2021-10-15T16:08:44Z

If there was a test case for this then the checks would've failed in main wouldn't they?

lhutton1 · 2021-10-15T17:07:10Z

The reason I ran into this was a use case where Depthwise had uint8 ifm/ofm but int8 weights. I'll look into adding this case in test_replace_depthwise_conv2d.py. Adding this case to other places will be more tricky since we rely on TFLite generated models as input.

ekalda · 2021-10-18T08:33:16Z

Hi @lhutton1, for making a test, you can also use make_ethosu_conv2d from infra.py , that will create the NPU op directly and you can intitialise it with "wrong" weight dtype. There are some examples of using it in test_type_inference.py

Mousius · 2021-10-19T14:25:41Z

Hi @lhutton1,

I see you've updated the test cases, and apologies if I'm not seeing it, but they're all int8 or uint8 so there's no test of the actual error case here?

lhutton1 · 2021-10-19T15:18:48Z

@Mousius see https://github.com/apache/tvm/pull/9299/files#diff-78a348fdab3d613a6644dc72ea3e697a8965c92a90117d345aa905c5027311f8R34-R35 :)

The reason answer at the end refers to dtype twice is because the input and output data types are being referenced, not the data type of the weights.

Mousius · 2021-10-20T22:23:04Z

So that's the happy path, are there tests that check the error is fired. Could I remove this ICHECK without effecting any tests?

lhutton1 · 2021-10-21T13:37:59Z

Hi @Mousius, yes, currently you can remove ICHECK without affecting the tests.

However, I'm not too sure I follow. I understand ICHECK is an assertion. A test could be added to check that the assertion was fired for incorrect datatypes but I don't think that's good practice, since assertions should only exist as a programmer aid. ~~They're also only triggered in a debug build, so the test would fail if this is not the case~~ (I was incorrect about this point). Furthermore, interaction from a user perspective should mean this assertion is never encountered since we only pattern match and offload the operations (and compatible attributes) that are supported by the microNPU. Incompatible operations should never reach this level, and so the assertion shouldn't get triggered unless there is a programmer error.

Mousius · 2021-10-23T16:32:22Z

I would suggest that if we've added the check and we want to ensure its checking the right things we should have a test.

ekalda · 2021-10-25T09:14:50Z

In general I agree that the code behaviour should be tested, but in this case I agree with Luke. It is not a hugely consequential check and it is not enabled on all the builds, so it is quite hard to test is consistently. Luke is right that these checks are developer facing, IIRC all the other Relay operators also have checks like these that are not tested.

Mousius · 2021-10-25T11:18:40Z

Going back to the TVM Code Review Guidelines it can be seen in F2 that we aim to ensure code runs correctly in all possible settings. This PR was raised as the error was triggering in the incorrect setting, if we've considered this check as valuable to keep in the codebase and therefore want it to be robust to code changes and refactoring there should be associated testing or otherwise it should be removed.

You can see similar testing added for other logging features here: #9350

ekalda · 2021-10-25T14:00:12Z

I think testing this ICHECK is not trivial since the test would need to have different behaviours based on the build type, if it is a choice between spending time to come up with a test for it or removing it, I'd go for removing it. I'd be interested in what @mbaret and @manupa-arm think about it though

Mousius · 2021-10-25T15:51:08Z

Hi @ekalda, the current strategy appears to be testing with checks enabled as these tests have been introduced in other PRs; we just need a test which has a weight of neither int8 nor uint8 and to catch it using pytest.raises - apologies if I've missed something here.

ekalda · 2021-10-26T08:07:36Z

Hi @Mousius, I think you are actually right that we could test the ICHECK if we wanted to without too much trouble, so apologies for the confusion there. However, for what it's worth, I don't think it is a good idea since I don't think we should test everything just because we can test it. Tests are code to maintain, so we should think carefully what is worth testing and what isn't. There are currently 5184 ICHECKs in the codebase and I don't think many of them are tested. So if we were to test every developer facing assert, we would end up having to maintain lots of not particularly useful tests.

Small typo means that ifm datatype is checked when it should be weight. Change-Id: I975e87c0bc91049ddee650f485f3a94841c55aba

Change-Id: I19fd934cda105187c86191313acf0fe88c75d244

Change-Id: Ibaf202287e2173bee638f9db8e553350ce1dff31

lhutton1 · 2021-10-28T11:20:49Z

Thanks for the discussion @Mousius @ekalda, I've added a test for incompatible weight data types to get this patch in. However, I do wonder whether this implies every assertion added should also be tested?

Mousius

Thanks for adding the test @lhutton1, I've just added one last point on the test itself.

However, I do wonder whether this implies every assertion added should also be tested?

All of the software teams I've worked in have tested for error cases, but we didn't add assertions in unreachable places as we had test coverage to ensure we never reached those lines. Happy to have a chat about this 😸

tests/python/contrib/test_ethosu/test_replace_depthwise_conv2d.py

…a more sensible location. Change-Id: I11f707ed4d81454765d7b3b597fa0c51e6909e3f

Mousius

LGTM! Thanks for your patience @lhutton1 and @ekalda 😸

manupak · 2021-10-28T19:36:17Z

@Mousius @ekalda @lhutton1 ,
A relevant past discussion : https://discuss.tvm.apache.org/t/rfc-icheck-and-check-conventions-and-error-messages/8709

I think this topic need a wider discussion on the forum, especially if we are deciding to enforce tests on ICHECKs (as opposed to CHECKs).

I would suggest we continue that discussion a bit in the forum as there might be both pros and cons.

cc : @tqchen @tkonolige

manupak

LGTM

tkonolige · 2021-10-28T20:05:09Z

src/relay/op/contrib/ethosu/depthwise.cc

@@ -126,7 +126,7 @@ bool EthosuDepthwiseConv2DRel(const Array<Type>& types, int num_inputs, const At
  ICHECK(ifm->dtype == DataType::UInt(8) || ifm->dtype == DataType::Int(8))
      << "Expected ethosu_depthwise_conv2d type(uint8) or type(int8) for ifm but was "
      << ifm->dtype;
-  ICHECK(weight->dtype == DataType::UInt(8) || ifm->dtype == DataType::Int(8))
+  ICHECK(weight->dtype == DataType::UInt(8) || weight->dtype == DataType::Int(8))


Please use the diagnostic context to report these errors. Here is an example: https://github.com/apache/tvm/blob/main/src/relay/op/nn/nn.cc#L65-L68.

tkonolige · 2021-10-28T20:05:34Z

The diagnostic context is the correct way to report these errors. Here is an example: https://github.com/apache/tvm/blob/main/src/relay/op/nn/nn.cc#L65-L68.

Convolution and depthwise convolution use the ICHECK format of error checking during type inference. This PR updates these checks to use the diagnostic context. This supersedes apache#9299. Change-Id: I67a06181440f84e9c38e02b9bf27218a6d6dd9d4

lhutton1 · 2021-11-08T13:22:16Z

Thanks @tkonolige, we have rewritten these checks using the diagnostic context in #9470. Closing this PR as #9470 supersedes this.

lhutton1 requested review from anijain2305, jroesch, junrushao, jwfromm, MarisaKirisame, mbrookhart, slyubomirsky, vinx13, wweic, yzhliu, zhiics and ZihengJiang as code owners October 15, 2021 13:53

lhutton1 requested review from areusch, comaniac, merrymercy and tqchen as code owners October 18, 2021 09:13

lhutton1 added 3 commits October 28, 2021 09:52

[microNPU] Fix typo in depthwise to allow int8 weights

a7218c6

Small typo means that ifm datatype is checked when it should be weight. Change-Id: I975e87c0bc91049ddee650f485f3a94841c55aba

Add testcase

ebc2319

Change-Id: I19fd934cda105187c86191313acf0fe88c75d244

Add test for incorrect weight data type

6ca9535

Change-Id: Ibaf202287e2173bee638f9db8e553350ce1dff31

lhutton1 force-pushed the fix-depthwise-typo branch from 544b834 to 6ca9535 Compare October 28, 2021 11:20

Mousius requested changes Oct 28, 2021

View reviewed changes

tests/python/contrib/test_ethosu/test_replace_depthwise_conv2d.py Outdated Show resolved Hide resolved

Flatten error check, also move test to type inference tests which is …

7938edc

…a more sensible location. Change-Id: I11f707ed4d81454765d7b3b597fa0c51e6909e3f

Mousius approved these changes Oct 28, 2021

View reviewed changes

manupak approved these changes Oct 28, 2021

View reviewed changes

tkonolige reviewed Oct 28, 2021

View reviewed changes

lhutton1 mentioned this pull request Nov 8, 2021

[microNPU] Replace ICHECK with diagnostic context in type inference #9470

Merged

lhutton1 closed this Nov 8, 2021

lhutton1 deleted the fix-depthwise-typo branch November 16, 2021 10:33

Mousius mentioned this pull request Apr 14, 2022

[AutoTVM] Use new target object #10963

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[microNPU] Fix typo in depthwise to allow int8 weights #9299

[microNPU] Fix typo in depthwise to allow int8 weights #9299

lhutton1 commented Oct 15, 2021

Mousius commented Oct 15, 2021

ekalda commented Oct 15, 2021

Mousius commented Oct 15, 2021

lhutton1 commented Oct 15, 2021

ekalda commented Oct 18, 2021

Mousius commented Oct 19, 2021

lhutton1 commented Oct 19, 2021

Mousius commented Oct 20, 2021

lhutton1 commented Oct 21, 2021 •

edited

Mousius commented Oct 23, 2021

ekalda commented Oct 25, 2021

Mousius commented Oct 25, 2021

ekalda commented Oct 25, 2021

Mousius commented Oct 25, 2021

ekalda commented Oct 26, 2021

lhutton1 commented Oct 28, 2021 •

edited

Mousius left a comment

Mousius left a comment

manupak commented Oct 28, 2021

manupak left a comment

tkonolige Oct 28, 2021

tkonolige commented Oct 28, 2021

lhutton1 commented Nov 8, 2021

[microNPU] Fix typo in depthwise to allow int8 weights #9299

[microNPU] Fix typo in depthwise to allow int8 weights #9299

Conversation

lhutton1 commented Oct 15, 2021

Mousius commented Oct 15, 2021

ekalda commented Oct 15, 2021

Mousius commented Oct 15, 2021

lhutton1 commented Oct 15, 2021

ekalda commented Oct 18, 2021

Mousius commented Oct 19, 2021

lhutton1 commented Oct 19, 2021

Mousius commented Oct 20, 2021

lhutton1 commented Oct 21, 2021 • edited

Mousius commented Oct 23, 2021

ekalda commented Oct 25, 2021

Mousius commented Oct 25, 2021

ekalda commented Oct 25, 2021

Mousius commented Oct 25, 2021

ekalda commented Oct 26, 2021

lhutton1 commented Oct 28, 2021 • edited

Mousius left a comment

Choose a reason for hiding this comment

Mousius left a comment

Choose a reason for hiding this comment

manupak commented Oct 28, 2021

manupak left a comment

Choose a reason for hiding this comment

tkonolige Oct 28, 2021

Choose a reason for hiding this comment

tkonolige commented Oct 28, 2021

lhutton1 commented Nov 8, 2021

lhutton1 commented Oct 21, 2021 •

edited

lhutton1 commented Oct 28, 2021 •

edited