unix: Add example of embedding interpreter in a standalone app.

[pfalcon]

No description provided.

[pfalcon]

Alternative to #1279. AS #1279 argues, if we go further, we may even cross some paths with it. What's interesting though is that here we start from the start, not from the middle or some other side.

[pfalcon]

@dpgeorge : So what about merging this?

[dpgeorge]

I just see this as another variation on the minimal/ "port" and adds more noise than anything else. Already people have too much choice to start from (bare-arm, minimal, unix, stmhal) and this makes it more confusing, and more stuff for us to maintain (how/when does this get built to check it works, etc?).

[pfalcon]

This is not a port, this is example of embedding interpreter in a user program. Actually, very first step towards that, but already working example. It's supposed to be elaborated further, but without first step, we can't go further. It's supposed to be used in following way: people ask if uPy supports embedding in a user app, and instead of answering "no", we answer "yes, build 'embed' target".

[pfalcon]

Well, to elaborate on what I might assume: Ideally, embedding example should be in a separate dir (perhaps subdir of examples/). We can't do that yet (libmicropython.a is on critical path to that), but can reuse existing unix port makefile - again, with aim to answer "yes" instead of "no" to the question above (and sooner, not waiting till embedding will be supported "perfectly" - it never will unless people cna play with imperfect version).

[dpgeorge]

People can already play enough with minimal and unix ports, and adapt to their needs. I think this example is misleading because there is too much hidden stuff (eg stdio bindings, gc) and the Makefile obscures and hides a lot of what you need to know to embed (there's a lot of magic with the build system that's not easy to extract to put in your own existing code).

For embedding, people want to know the answer to: "what's the minimum I need to do to get uPy running within my existing program". Telling them to build "make embed" will just lead to more questions.

[pfalcon]

People can already play enough with minimal and unix ports, and adapt to their needs.

Again, this is not "port", so bringing in "ports" here is what's confusing.

I think this example is misleading because there is too much hidden stuff (eg stdio bindings, gc) and the Makefile obscures

It doesn't obscure anything - the source is there. Yes, there should be documentation on embedding, but again, it's just 1st step in the direction of supporting at all.

Note that 1st messages about people building static lib out of uPy were on the forum ~ a year ago. They didn't end up in submissions back to the project, but people keep asking about embedding uPy (well, I suggest people to embed uPy, and they ask "how" in return). Proposed is the minimal patch which crosses border between "not supporting embedding" and "supporting embedding".

(Why it gets so hard to have a further progress on uPy development - we've accumulated technical debt over last 1.5 years, I'm trying to resolve it, but there appears to be resistance to do anything non-trivial ;-) ).

[dpgeorge]

Again, this is not "port", so bringing in "ports" here is what's confusing.

But an embedded port needs to be just that: a port to your own application / runtime, hooking into existing stdio etc.

Each port that we already have are examples of "embedding" uPy into an existing runtime framework; eg stmhal provides an OS for STM chips, and embeds uPy within that.

I see this PR as just another "port" to unix, a simple one that compiles and executes a string.

I'm trying to resolve it, but there appears to be resistance to do anything non-trivial ;-)

I feel it too! I would be more receptive to something more radical, like a new "port" called "embed/" which tried to do something like build an amalgamation of py/ source files, provide sensible qstrdefs.generated.h and mpversion.h files, and a trivial Makefile that didn't need the entire uPy source tree to work. That's something people can really use, and something we can build on for the long term.

[pfalcon]

Each port that we already have are examples of "embedding" uPy into an existing runtime framework;

Yes, those are may words, which I told yet at the forum topic. But saying them doesn't help to people, there should be an example.

I would be more receptive to something more radical, like a new "port" called "embed/"

If you suggest moving this code to "embed/" to merge it, ok. But then it should examples/embed/, because this is not a port, but an example, how people should approach embedding. But then Makefile there will be just duplication of unix' Makefile, and that's what hard to maintain.

which tried to do something like build an amalgamation of py/ source files, provide sensible qstrdefs.generated.h

Well, that's too radical. How we did it so far is sustainable gradual changes, and I suggest we keep that way. So, embedding library would likely be based on unix port, so early library-less example just lives there to avoid duplication. Standard way to embed one code in another is to make that one a library, and that would be next step, but there're lot of small questions for this step, so it's not hasted. Amalgamation? I consider that completely niche feature, nice to do when the base is done, when somebody feels like doing it, and for the purpose of showing "MicroPython can do that too", not because it's particularly useful (again, the standard way to embed code is by using library).

[dpgeorge]

But saying them doesn't help to people, there should be an example.

See minimal/.

But then it should examples/embed/, because this is not a port, but an example

But it should compile, and so should be a port.

But then Makefile there will be just duplication of unix' Makefil

No, it won't have all the extra complexity.

Well, that's too radical. How we did it so far is sustainable gradual changes, and I suggest we keep that way.

For something like this we need to have a big change. Evolution sometimes is not enough, sometimes you need new steps. And this is exactly what you are arguing for saying that it's hard to get new non-trivial features in uPy.

Standard way to embed one code in another is to make that one a library

Well, standard way is too restrictive. There are real-life cases where a library doesn't work, eg how is the library compiled for a given arch? How to embed using a web IDE that has control over the compiler and linking stages?

[pfalcon]

Ok, where do we go from here? As I mentioned initially, I didn't consider this patch ideal for merging, exactly because it may happen that it may move completely to another dir or just rewritten completely. The reason I'd prefer it merged is exactly because I don't plan to work in that direction for the foreseeable future and would like to leave a reference for other people to look at/continue from - to break anti-pattern of effort being spent, but not leading to overall progress in the upstream (all those year-ago lib builds, etc.)

[dpgeorge]

It might be more useful to actually write a (short) piece of documentation about simple porting/embedding. And then this embed.c can go in that documentation. At least we know that that's something we eventually need, and can be built upon.

[pfalcon]

It might be more useful to actually write a (short) piece of documentation about simple porting/embedding.

Yes, it would be useful, but usually it happens that (clean, easy enough) code is a documentation, and actual docs come much, much later (e.g. asm syntax reference). And if I's work on docs, that would be main library reference, so this would have to wait a long time.

Ok, closing this, I'll keep this branch in my repo.