Add a port which outputs a set of .c files for embedding into a project

[dpgeorge]

This is another way to embed MicroPython into a custom project with a custom build system.

It has two steps:

1. Build the embed port, which outputs a self-contained set of source files for building MicroPython. They can exist out-of-tree.
2. Build the project. This just requires compiling all .c files from the above step.

This is an alternative to #5964.

TODO:

• decide on name for this new port
• write a README for the port

[jimmo]

Excellent!! I have been looking forward to this, and it's an overall better and more flexible approach than the libmicropython.a port.

As with #5964 I think we should replace the existing embedding example. It isn't particularly useful and adds more confusion because it's so tied into the unix port.

But like the existing embedding example, it would be good to make the example also includes things like defining modules and QSTRs etc. (i.e. it needs to be an input to the genhdr files).

[dpgeorge]

But like the existing embedding example, it would be good to make the example also includes things like defining modules and QSTRs etc. (i.e. it needs to be an input to the genhdr files).

Yes, I've done this on the project this was based on. But IMO it's worth having an "absolute minimum" example. And then we can add another example with more complicated use cases.

[dpgeorge]

I have replaced the existing embedding example with this new one.

[github-actions]

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 

[codecov-commenter]

Codecov Report

Merging #9529 (5c6e581) into master (3b5cafc) will not change coverage.
The diff coverage is n/a.

❗ Current head 5c6e581 differs from pull request most recent head e8d57fd. Consider uploading reports for the commit e8d57fd to get more accurate results

@@           Coverage Diff           @@
##           master    #9529   +/-   ##
=======================================
  Coverage   98.50%   98.50%           
=======================================
  Files         155      155           
  Lines       20537    20537           
=======================================
  Hits        20229    20229           
  Misses        308      308           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[SKZ81]

Hello, I just ran into a little problem using embed port.
It really works well on my desktop, and now, I want to integrate it into an esp-idf component.

NOTE: The python script will be used to customize the esp32 firmware behaviour, which will be mostly written in C/C++.
Also, I want to be able to manage the functionality subset I include in the component, so it seems a good way to go.

Here is my "workflow":

1. I cloned micropython as a submodule in a "third-party source" directory in my project,
2. My project has a components/micropython_embed directory, which contains a CMakeList.txt for idf.py to build it,
3. The project 'root' CMakeList.txt has a custom command/rule to call make -f .../embed.mk and generate a micropython_embed/timestamp empty file (used for deps mgmt & regen). After this step the component sources are populated.
4. idf.py builds the entire project and its components

Things start to go bad when I copied the mphalport.* from the actual esp32 port into my component, to replace the basic ones provided in the example/embedding directory of micropython.

The problem happens while generating the components/micropython_embed source tree: as esp32's mphalport uses esp-idf specific files (for instance FreeRTOS headers), and, since the embed.mk makefile does not declare any include dir related to esp-idf (or any other platform-specifcs), it fails with a "freertos/FreeRTOS.h" not found.

I tried to workaround this by passing a (dirty and) extra -I to the compiler, pointing to esp-idf's freertos include dir...
But it get even worse as it now requires FreeRTOSConfig.h which is generated, but not exposed (at least I did not find it) by esp-idf build system (even in project's build/ dir).

The workaround I now have locally, is to rather pass a -DBUILDING_MICROPYTHON_EMBDED_SOURCE_TREE to the compiler, and garding the platform specific code by

#ifndef BUILDING_MICROPYTHON_EMBDED_SOURCE_TREE

...

#endif

I'm not sure if this is a good idea or not ?

esp-idf is deprecating makefile support, maybe a cleaner and more integrated solution would imply to rewrite the Makefile to a CMakelist ??

I'm not very fluent in cmake, and just enough in makefile, so i'm no expert at all here, please share any though or advice !

Thanks in advance,

[projectgus]

@SKZ81 Unfortunately I think what you're trying to do here is a bit tricky, I'd recommend opening a Discussion to talk about rather than discussing here in this merged PR.

The description of what you've done here is really clear, but if you're making a discussion please clarify exactly what outcome you want in the post: the embed port is intended to embed a custom scripting language in another program, with some custom bindings between the Python code and the host program. If your goal is to do this then I think it's possible with some translation from the generated embed port Makefile and an ESP-IDF component's CMakeLists.txt.

OTOH if your goal is to embed the entire esp32 port in an unrelated ESP-IDF project (i.e. including machine module drivers and things like this), it's probably going to be very hard and you might look at going the other direction (forking the MicroPython esp32 port and linking your other C code into it).