Integrating Matter IoT with Micropython

[x4t-de]

Dear Micropython community,

I would like to make a PoC for using Micropython and Matter. I set up all prerequisites and the build chain for Micropython and ESP-MATTER and I am able to compile the "light" - example from the esp-matter example collection. I can also build Micropython with custom user-C-modules.

First question at this stage: are there any sources I could read for integrating Matter into Micropython? Has somebody already tried it? Is it even possible, or are there any things that would speak against such an endeavour?

Now I wanted to see how I need to adapt the CMakeFiles in Micropython so that the esp-matter SDK can be used. I added matter.c to the other custom source files, where I just have the following lines:

// Includes from the "light device" example.
#include <esp_matter.h>
#include <esp_matter_cluster.h>
#include <esp_matter_console.h>
#include <esp_matter_ota.h>

// Matter is C++ based, so functions need to be wrapped into extern "C"
extern "C" void matter_build_test()
{
    // Use something from Matter SDK to check build chain.
    node::config_t node_config;
}

One obstacle was that according to espressif/esp-matter#604 the esp-matter SDK expects a "main" folder in my project. So I made a copy of the main_esp32 folder named "main".

Now the compilation goes a bit further, but stops with:

CMake Error:
  Error evaluating generator expression:

    $<TARGET_FILE:__idf_mbedtls>

  Target "__idf_mbedtls" is not an executable or library.

Does somebody have an idea about where to look for? I searched for occurences of __idf_mbedtls, but it only appears in the build dir of the light example, probably from my build test of this example project. Is this maybe related to this: espressif/esp-idf#8359? Are there maybe conflicting mbedtls libs or build definitions?

Thank you very much in advance for any tips and suggestions.

Regards

JR

[jimmo]

@x4t-de I looked into this briefly about a year ago, it didn't look easy at all.

The problem is essentially you need to combine three things: the Matter SDK, MicroPython, and the vendor SDK (e.g. ESP-IDF), and all three of those things expect to be "in charge".

Espressif is probably going to be one of the easiest routes to Matter support though. I would be interested to hear if anyone else has looked into this.

[x4t-de]

Thanks for your reply. I assumed that this was not going to be an easy task, so I am not too much surprised about being stuck so early 😬 I already started a thread in the ESP32 forum: https://www.esp32.com/viewtopic.php?f=13&t=39085

In the meantime I'll try to work my way through the CMakeLists. Since I hope this is not based on magic, there must be a way to fix this.

[igrr]

Hi @x4t-de,

I'm attaching a patch to micropython which will hopefully help you continue integrating Matter into Micropython.

0001-demo-of-adding-esp-matter-to-Micropython-build.patch.txt

The patch adds esp-mater as a managed component from the IDF component registry: https://components.espressif.com/components/espressif/esp_matter. I've added everything into the new main component, as esp-matter currently expects that component to exist.

Depending on the version of the IDF component manager you have installed locally and your OS, you might also need to set one environment variable for the build to succeed:

export IDF_COMPONENT_OVERWRITE_MANAGED_COMPONENTS=1

[x4t-de]

This is awesome!!! The error is solved with it. There are some warnings (e.g. aboput unused variables). But I'll check if I have updated sources for all parts first.

Unfortunately, I get an error during linking, see below. I set the env variable as suggested,l but that didn't seem to have to do something wih the linker error. Will report if it still persists after checking everything is up to date.

/mnt/dev_data/espressif_env/esp_v5.1.2/bin/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: esp-idf/esp_system/libesp_system.a(panic_handler.c.obj):(.literal.panic_handler+0x44): undefined reference to __wrap_esp_panic_handler' /mnt/dev_data/espressif_env/esp_v5.1.2/bin/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: esp-idf/esp_system/libesp_system.a(panic_handler.c.obj): in function frame_to_panic_info':
/mnt/dev_data/espressif_env/esp_v5.1.2/esp-idf/components/esp_system/port/panic_handler.c:107: undefined reference to `__wrap_esp_panic_handler'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

[x4t-de]

OK, after the micropython update (also of all sub libraries), building works. Fantastic help @igrr, thank you so much!

[x4t-de]

I think that solves the building problem for now. Though I'd ike to ask: in the patch you provided, the matter.cpp that includes the source code for the matter part of the application, resides in the micropython folder. Do you see a chance to use matter in the custom user sources? I tried to add esp_matter.h to a user source file, but the esp_matter.h header was not recognized. I assume because the location of matter headers is introduced later than adding user code to the build. Do you see a possibility to make matter available in user modules?

[igrr]

@x4t-de I am sorry, I am not yet familiar with user modules. I will need to take a look at the way they are included into the build.
I'll try to give you a solution next week.

[beyonlo]

Integrating Matter IoT with Micropython will be amazing for IoT low power solutions!

Thanks all!