[Discussion]: introducing %lua{ } expansions?

[Delapouite]

Hi.

The recent release of neovim 0.2 made a big splash partially because it lets you replace viml scripts by lua ones. (the API is currently a bit cumbersome, but it should be better with version 0.3).
Clearly, people love this new addition.

So this has me thinking: would there be any benefits to have some kind of similar mechanism to also write native lua script in kakoune?
Something like %lua{} string expansions alongside the already existing %sh{} (or something else).
Note that I'm a JavaScript guy and know nothing about the feasibility, the difficulty, the memory cost or the weight cost of such en embedding in the kakoune C++ codebase.

I know that mawww vision (shared by many other people, myself included) is to keep the project small and this feature may clearly be out of scope, and I totally understand.

Current kak scripts already work quite well, and so far they have let us do pretty much everything we needed.
But, I personally find working with shell scripting quite hard, the whole POSIX compatibility, the string escaping madness, and I have the feeling that using the power of higher level language such as lua would really simplify the creation of bigger and more complex plugins.

Do some of you feel the same way? Is this proposal completely stupid or following this path may eventually result in something good?

[danr]

Just a heads up: there is a little bit of discussion in doc/manpages/faq.asciidoc about this:

Why aren't there other scopes similar to %sh{} e.g. python ?

Supporting custom scopes would add hard dependencies to the project, which
is too much of a drawback when balanced against the low cost of using
an interpreter in a regular shell scope (e.g. %sh{ python -c "..." }).
The shell scope allows users to spawn any interpreter they want, for a minimal
cost in terms of performance, it is therefore the reason why it's the only
one available by default.

[lenormf]

This question is addressed in the FAQ.

Adding more custom scopes creates coupling between the project and third-party interpreters that might not be available on the target system, whereas we're guaranteed to have a shell if kakoune is to run at all. Another issue could be the version of the language targeted, e.g. how do we handle python2 and python3 (%py2{}? %py3{}? only support one or the other in %py{}?).

I used to think adding custom scopes would be nice as well, but now after deploying software to completely different systems, I've learned that the less dependencies (even soft), the better.

[Delapouite]

Thanks for these precisions.

I explicitely mentionned lua (and not python, ruby or another language), because from what I understand about all the work that has been done on the neovim side (and I may be totally wrong), lua seems specifically designed to be embedded in the host project. So it means that it's supposed to be a lightweight intrepreter merged into the final kak binary and avoid the issue of relying on an external interpreter.

[lenormf]

There are several drawbacks to this approach that neovim has apparently taken:

• you have to monitor the upstream development of the library (the lua interpreter) you embed constantly, to patch it when security issues occur
• users will consistently complain that some feature implemented recently upstream are not available in the library embedded by the editor
• all users have to re compile the project when any of the above occurs, as opposed to installing the package released for the separate interpreter
• users who don't like lua will shaun the project for forcing them to have the interpreter on their system, even if it's embedded as static code (it takes more space, and adds to the attack surface of the system security-wise)

Kakoune is very UNIX-centric, and I don't think the above approach fits very well with this paradigm: users who want to use lua can do so with the method described in the FAQ, to everyone's benefit, and no additional burden is put on the maintainer's shoulders in the process (c.f. above list).

[Delapouite]

I'm not good enough in C++ to attempt to write a proof of concept. I also don't want to put this burden on mawww. So closing for now.

[Delapouite]

As food for thoughts for future attempts, here's a similar prototype by @doppioandante embedding a scheme manager: doppioandante@0451d28

http://synthcode.com/scheme/chibi/

[tadeokondrak]

If anyone found this issue looking for something similar for personal use, I've implemented %js{} statements with MuJS here in a small patch: https://gist.github.com/tadeokondrak/da159465c43587d9afb1936a9ecb6b3e

The MuJS API is very similar to the Lua API, so one for Lua could be done in the same way.

[Delapouite]

Thanks! This integration looks great.

https://mujs.com/introduction.html

MuJS implements ES5.

Unfortunately, going back to prehistoric era will feel a bit too harsh for me.

[occivink]

There's that js interpreter by Fabrice Bellard, QuickJS which might be a good candidate too.