pkg/tensorflow-lite: add support to RIOT

[aabadie]

Contribution description

This PR adds support for TensorFlow Lite for microcontrollers to RIOT.

It based on 2 PRs

• pkg: add support for FlatBuffers serialization library #12846: add support for FlatBuffers, which is required for TensorFlow Lite
• Makefile.base: add variables for customizing C++ builds #12844: to be able to build c++ files with the .cc extension and to exclude a list of c++ files (because not all should be built in a given tensorflow directory).

The integration into the build system can certainly be improved, so comments are welcome, of course.

The provided test application is just a plain copy (with .cc extensions changed into .cpp) of the hello-world sample, but there are more and it shouldn't be too difficult to adapt them.

cc'ing @adjih and @emmanuelsearch, who might be interested in testing this.

Testing procedure

Just build and run the test application:

make BOARD=<board of your choice> -C tests/pkg_tensorflow-lite flash term

It takes a bit of time to build but it's worth waiting :)

Issues/PRs references

Based on #12846 and #12844

[aabadie]

I pushed some code that adapts the magic_wand example but couldn't make it work yet: the interpreter remains stuck when calling it Invoke method (I'm still investigating).
The accelerometer handler uses the lsm6dsl sensor instead of the lis2dh12 (SparkFun) and LSM9DS1 (Arduino) used by the upstream examples.

[aabadie]

the interpreter remains stuck when calling it Invoke method (I'm still investigating).

I found the problem: the stack size of the main thread was not large enough. Doubling it and the workflow now works. Unfortunately, I can't get a valid gesture detection.
I'll try to update to a more recent commit of the package since it seems there were some updates recently.

[emmanuelsearch]

@aabadie could it be that the gesture model is (highly) dependent on the actual accelerometer type?

[emmanuelsearch]

And the new URL for the examples upstream are https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/micro/examples

[aabadie]

could it be that the gesture model is (highly) dependent on the actual accelerometer type?

It could be several things. In the examples, they are using different accelerometers (all ST though) and use the FIFO mode, which RIOT doesn't provide. Maybe the range of data is not the same.

There was no model training example for the magic wand in the TensorFlow version used by this PR. It was added recently so it may help training a model that best fit the RIOT accelerometer data.

[aabadie]

I updated the package version and adapted to the new structure (was quite easy) but still no luck with the magic wand. Maybe it's time to train some model on the sensor data I'm using ? ;)

[aabadie]

I managed to get valid prediction with the magic wand model:

2019-12-20 09:07:07,931 # main(): This is RIOT! (Version: 2020.01-devel-1453-g4c8384-pr/pkg/tensorflow-lite)
2019-12-20 09:07:07,954 # Magic starts!
2019-12-20 09:07:21,134 # WING:
2019-12-20 09:07:21,136 # *         *         *
2019-12-20 09:07:21,138 #  *       * *       *
2019-12-20 09:07:21,140 #   *     *   *     *
2019-12-20 09:07:21,142 #    *   *     *   *
2019-12-20 09:07:21,144 #     * *       * *
2019-12-20 09:07:21,146 #      *         *
2019-12-20 09:07:21,146 # 

It's still very inaccurate and requires changes in the lsm6dsl driver. I think having the possibility to use the FIFO mode of the driver would also help.

[aabadie]

Finally, I replaced the magic_wand example with a new one based on the MNIST hand-written digit. The new example comes with Python scripts to train the model and to generate the input digit used by the application. This new example replicates the application of the utensor package.

Regarding the magic wand, I'll provide it as a follow-up because it requires other changes in RIOT ST accelerrometer drivers (read data as float, require FIFO mode support). So no need to block this PR because of this.

[adjih]

Note that the magic wand example is not about drawing a letter with the hand. It is about doing a gesture with an Harry Potter-like wand, and was inspired by J. Wang's example .

Indeed FIFO support can be skipped: if one is really interested into this application (for next Halloween, etc.), one can train with a custom dataset, and using provided CNN/LTSM training ).

[aabadie]

Tensorflow released the 2.1 version this week. I'll check if this PR can be based on it, instead of using a random commit in master.

[aabadie]

So I gave a try to the 2.1 version and it seems to not contain the latest changes that are used in this PR.

[aabadie]

@fjmolinas, rebased now that #12846 is merged. Maybe I should squash since there was no review until now ?

[aabadie]

I got a green light from @fjmolinas IRL, so squashed :)

[fjmolinas]

Some style comments.

[fjmolinas]

Tested generating a new image (digit 1) and code correctly detected it, the test script doesn't work though since the digit detection is hardcoded:

main(): This is RIOT! (Version: 2020.01-devel-1738-g1b61e-pr-12847)
Digit prediction: 1
Timeout in expect script at "child.expect_exact("Digit prediction: 7")" (tests/pkg_tensorflow-lite/tests/01-run.py:8)

[aabadie]

@fjmolinas, I think I addressed all your comments. Let me know if it's ok for you now!

[fjmolinas]

@aabadie one last comment please squash after amending.

Wrong button, ups

[fjmolinas]

ACK!