Add CMSIS-NN SVDF kernel.

[jenselofsson]

This pull request adds the optimized version of the (int8) SVDF kernel in CMSIS-NN to TensorFlow Lite Micro.

The pull request to CMSIS-NN is still pending, and the CMSIS version in third_party_downloads.inc will need to be updated once that is merged.

Fixes #42650

[njeffrie]

This looks great - just one nitpick:
Since this is the first time we are duplicating the SVDF float code (xtensa_hifimini does not implement a floating point SVDF), do you think we can extract the contents of EvalFLoatSVDF into tensorflow/lite/kernels/internal/reference/svdf.h and share the implementation between the CMSIS-NN kernel and the reference kernel?

Let me know if this sounds reasonable to you.

[jenselofsson]

This looks great - just one nitpick:
Since this is the first time we are duplicating the SVDF float code (xtensa_hifimini does not implement a floating point SVDF), do you think we can extract the contents of EvalFLoatSVDF into tensorflow/lite/kernels/internal/reference/svdf.h and share the implementation between the CMSIS-NN kernel and the reference kernel?

Let me know if this sounds reasonable to you.

Sounds good, I'll update the pull request with that change.

[jenselofsson]

This looks great - just one nitpick:
Since this is the first time we are duplicating the SVDF float code (xtensa_hifimini does not implement a floating point SVDF), do you think we can extract the contents of EvalFLoatSVDF into tensorflow/lite/kernels/internal/reference/svdf.h and share the implementation between the CMSIS-NN kernel and the reference kernel?

Let me know if this sounds reasonable to you.

I had a bit of a look at it, and there seems to already be a EvalFloatSVDF() in tensorflow/kernels/inernal/reference/svdf.h. The difference between that EvalFloatSVDF() and the one found in micro is (among other things) that the micro-version have TfLiteEvalTensor as input. Should I still copy over the EvalFloatSVDF() function from micro to that file?

EDIT: On second, that won't work due to the fact that tensorflow/lite/kernels/reference/svdf.h depends on tensorflow/lite/kernels/reference/tensor_utils.h, which in turn depends on Eigen. @njeffrie do you have a preferred solution for this?

[njeffrie]

It looks like we need to do some internal clean-up for the shared SVDF code before it will work for this. Let's land this first then we can internally manage extracting the float version into a common kernel in a future change.

[jenselofsson]

It looks like we need to do some internal clean-up for the shared SVDF code before it will work for this. Let's land this first then we can internally manage extracting the float version into a common kernel in a future change.

Sounds good! Let me know if you need anything from my end.

[Abhipray]

Hi @jenselofsson, I am curious how you use this kernel? Is there a python implementation of svdf that converts to this tf lite op?

[gbaned]

@jenselofsson Can you please resolve conflicts? Thanks!

[jenselofsson]

@gbaned Done!