[µTVM] Enable AutoTVM for ARM STM32F746XX Boards #4274

weberlo · 2019-11-07T21:00:34Z

This PR adds support for autotuning via MicroTVM. To test this infrastructure on a physical board, I have added support for ARM STM32F746XX boards, featuring Cortex-M7 CPUs. As a followup to this PR, I will write a tutorial for tuning conv2d.

Here are the most notable changes:

All components in the µTVM infra are now parameterized by the word size of the target device.
Device configuration has been expanded to include the memory layout, word size, and thumb mode indicator of the device.
There is now a micro.device Python namespace featuring a global registry of all supported devices. The registry is indexed by device ID (e.g., host, riscv_spike, or arm.stm32f746xx). and maps to dictionaries containing two functions: create_micro_lib (for creating libraries specific to that device) and default_config (for generating default device-specific config).
The µTVM runtime API has been expanded to include timing functions, where each implementation is device-specific.`
There is new a src/runtime/micro/device folder which mirrors the structure of the micro.device folder and includes device initialization and timer implementations for each device.
RPC sessions will now use the MicroTimeEvaluator when possible, to make use of cycle-accurate timings available on microcontrollers, instead of using wall clock time (which would include communication overhead).

Many thanks to @tqchen for discussing the design with me!

CC @u99127 @ajtulloch @jwfromm

tqchen

quick comments

python/tvm/contrib/binutil.py

python/tvm/micro/base.py

python/tvm/micro/device/arm/stm32f746xx.py

tqchen · 2019-11-07T22:33:04Z

src/runtime/micro/device/arm/stm32f746xx/utvm_timer.c

@@ -0,0 +1,102 @@
+#ifdef __cplusplus
+extern "C" {


.cc o be consistent with the rest part of the stack

This is meant to be compiled and loaded on the device. The #ifdef __cplusplus is just in case a C++ compiler is run over it.

python/tvm/micro/rpc_server.py

src/runtime/micro/device/arm/stm32f746xx/utvm_timer.c

src/runtime/micro/openocd_low_level_device.cc

weberlo · 2019-11-08T00:35:33Z

@tqchen It looks like the CI doesn't allow assembly---namely, utvm_init.s. That file is required to enable the Cortex-M7 FPU and stack pointer. Can you add it as an exception? We might also want an assembly file whitelist for the entire src/runtime/micro/device directory.

tqchen · 2019-11-11T18:22:41Z

please rebase against the master due to #4286 also fix the ci error

src/runtime/micro/micro_session.cc

python/tvm/contrib/binutil.py

u99127

Aren't the commented bits part of BinaryContents ?

u99127

I've spent a couple of hours this evening reviewing this and I have some initial minor corrections for what struck my eye when reading this.

It would be good to consider the following points for future direction. I am not asking for this to be fixed as part of this PR.

In the Arm architecture - there is an architecture level, there are optional features in the architecture, usually an FPU. There are multiple implementations for a particular architecture level and finally multiple devices for each of those CPU implementations. There are many differences between the multiple devices but in the context of uTVM the differences we need to start by worrying about between the devices is really the memory maps and what optional features of the ISA are implemented in that device.

Now, why is this important ? In this world there are multiple implementations with Cortex-M7 with an FP5-SP-D16 FPU init, but the memory maps might well be different between different boards from different manufacturers, thus having easy ways of describing only those differences in a first class way are useful.

regards,
Ramana

python/tvm/micro/base.py

python/tvm/micro/device/arm/stm32f746xx.py

src/runtime/micro/micro_session.h

weberlo · 2019-11-23T00:13:05Z

Now, why is this important ? In this world there are multiple implementations with Cortex-M7 with an FP5-SP-D16 FPU init, but the memory maps might well be different between different boards from different manufacturers, thus having easy ways of describing only those differences in a first class way are useful.

@u99127 This is good to know. We should evolve the design to accommodate these instances as they crop up.

next stop: Tcl-driven

actually relocating the binary now

but now floating point instructions don't work

Also, - templatize `EncoderAppend` - remove `DeviceLocation` class - add `TargetVal` union that `DevPtr` uses

weberlo · 2019-11-25T16:59:36Z

One last change incoming. Forgot to move create_micro_mod out of micro.Session into a static method.

src/runtime/micro/device/arm/stm32f746xx/utvm_timer.c

tqchen · 2019-12-01T15:37:00Z

ping @u99127 please https://docs.tvm.ai/contribute/code_review.html?highlight=approve#approve-and-request-changes-explicitly

u99127

Sorry about the time it's taken.

While this feels like a very initial integration, I think the Arm backend parts should certainly be made more modular to make board addition simpler and the overflow counting for performance counters needs to be handled in the future.

regards
Ramana

tqchen · 2019-12-02T18:38:32Z

Thanks @u99127 @weberlo !

tqchen added the status: need review label Nov 7, 2019

weberlo force-pushed the add-arm-autotvm-utvm branch from f75c516 to ebaac5d Compare November 7, 2019 22:14

tqchen requested changes Nov 7, 2019

View reviewed changes

python/tvm/micro/rpc_server.py Outdated Show resolved Hide resolved

src/runtime/micro/device/arm/stm32f746xx/utvm_timer.c Outdated Show resolved Hide resolved

src/runtime/micro/openocd_low_level_device.cc Show resolved Hide resolved

weberlo force-pushed the add-arm-autotvm-utvm branch from 1ca1cda to 2279ce9 Compare November 10, 2019 06:28

weberlo force-pushed the add-arm-autotvm-utvm branch from 5b86677 to babeb97 Compare November 12, 2019 17:49

weberlo commented Nov 12, 2019

View reviewed changes

src/runtime/micro/micro_session.cc Outdated Show resolved Hide resolved

weberlo force-pushed the add-arm-autotvm-utvm branch from cd15454 to c9629a8 Compare November 19, 2019 20:06

tqchen requested changes Nov 22, 2019

View reviewed changes

python/tvm/contrib/binutil.py Outdated Show resolved Hide resolved

tqchen approved these changes Nov 22, 2019

View reviewed changes

u99127 reviewed Nov 22, 2019

View reviewed changes

tqchen approved these changes Nov 22, 2019

View reviewed changes

u99127 suggested changes Nov 22, 2019

View reviewed changes

weberlo added 14 commits November 24, 2019 15:12

TEMP

dbbffbd

TEMP 2

5b356a7

GDB-driven execution works

93e7068

next stop: Tcl-driven

fadd works!

cbce8e0

RAM-only fadd works

c6f6f2b

Remove unnecessary include

a9f1736

TEMP

bc87f18

compiling as obj instead of shared

bbffe95

actually relocating the binary now

no longer relying on external makefile

043297d

working with dynamically loaded kernels

5e48b68

Make conv2d werk

c476648

added timing funcs

70e9b75

but now floating point instructions don't work

bring FPU back online and print execution times

6197423

Add alternative timing impl

88e34dd

weberlo added 13 commits November 24, 2019 15:12

Lint

c85b475

Lint

07a2129

Address tqchen's comment

5e38d2b

Also, - templatize `EncoderAppend` - remove `DeviceLocation` class - add `TargetVal` union that `DevPtr` uses

Lint

17715ee

Fix CI

aeb840f

Move '-device_type=micro_dev' check to ndarray.py

70d607e

Fix

3ecd4bf

Fix binutil tests

98462b9

Fix

1224eed

Fix

721ceb3

Fix

27897d6

Make quotes thicker

013795e

Address comments

9d77f82

weberlo force-pushed the add-arm-autotvm-utvm branch from 4674aa0 to 9d77f82 Compare November 24, 2019 23:14

Remove copyright lines

3d594b7

Make create_micro_mod static

afd614e

u99127 reviewed Nov 25, 2019

View reviewed changes

src/runtime/micro/device/arm/stm32f746xx/utvm_timer.c Show resolved Hide resolved

u99127 approved these changes Dec 2, 2019

View reviewed changes

tqchen merged commit 47c870a into apache:master Dec 2, 2019

tqchen added status: accepted and removed status: need review labels Dec 2, 2019

tmoreau89 pushed a commit to tmoreau89/tvm that referenced this pull request Dec 3, 2019

[µTVM] Enable AutoTVM for ARM STM32F746XX Boards (apache#4274)

2d2f1b4

zxy844288792 pushed a commit to zxy844288792/tvm that referenced this pull request Dec 13, 2019

[µTVM] Enable AutoTVM for ARM STM32F746XX Boards (apache#4274)

cbe09c2

zxy844288792 pushed a commit to neo-ai/tvm that referenced this pull request Dec 13, 2019

[µTVM] Enable AutoTVM for ARM STM32F746XX Boards (apache#4274)

cf93ef0

tmoreau89 mentioned this pull request Mar 23, 2020

[uTVM][Runtime] Deprecate uTVM Standalone Runtime #5060

Open

9 tasks

zhiics mentioned this pull request Sep 15, 2020

TVM v0.7 Release Note Candidate #6486

Closed

mehrdadh mentioned this pull request Jun 21, 2021

[microTVM] Refactor uTVM to microTVM #8283

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[µTVM] Enable AutoTVM for ARM STM32F746XX Boards #4274

[µTVM] Enable AutoTVM for ARM STM32F746XX Boards #4274

weberlo commented Nov 7, 2019

tqchen left a comment

tqchen Nov 7, 2019

weberlo Nov 7, 2019

weberlo commented Nov 8, 2019 •

edited

tqchen commented Nov 11, 2019

u99127 left a comment

u99127 left a comment

weberlo commented Nov 23, 2019

weberlo commented Nov 25, 2019

tqchen commented Dec 1, 2019

u99127 left a comment

tqchen commented Dec 2, 2019

[µTVM] Enable AutoTVM for ARM STM32F746XX Boards #4274

[µTVM] Enable AutoTVM for ARM STM32F746XX Boards #4274

Conversation

weberlo commented Nov 7, 2019

tqchen left a comment

Choose a reason for hiding this comment

tqchen Nov 7, 2019

Choose a reason for hiding this comment

weberlo Nov 7, 2019

Choose a reason for hiding this comment

weberlo commented Nov 8, 2019 • edited

tqchen commented Nov 11, 2019

u99127 left a comment

Choose a reason for hiding this comment

u99127 left a comment

Choose a reason for hiding this comment

weberlo commented Nov 23, 2019

weberlo commented Nov 25, 2019

tqchen commented Dec 1, 2019

u99127 left a comment

Choose a reason for hiding this comment

tqchen commented Dec 2, 2019

weberlo commented Nov 8, 2019 •

edited