Script-based plugins(effects, controllers, etc.)

[JohannesLorenz]

I had the following idea: we have many plugins, but if you want to write your own for just one song, there is a long way to go:

• Write the plugin in C/C++
• Register the plugin
• Wait for the next LMMS version ;)
• Hope that it will also compile on Windows etc ;)

Anyways, I have once written a C interpreter. It builds a so called "abstract syntax tree" from a string input once, and this tree can then be executed multiple times, like a compiled program. We could predefine controllers and effects that use such equations.

Example: Amplifier:

Predefined: An effect with stereo in and out, 16 knobs, and one string input.
What you do: Type "for(int i = 0; i < buf_size; ++i) { out_l[i] = in_l[i] * k[0]; out_r[i] = in_r[i] * k[0]; }" into the string input.
Meaning: out_l, out_r, in_l, in_r are predefined variables for the in/out buffers. buf_size is predefined for the buffer size. k[0] is knob 0.

Other examples: Controllers, e.g. that delay the controller signal, apply inertia to it or whatever.

What do you think about such an interpreter?

Note: I know scripts are usually slower. But once a script proves as usable, we could just move the script code into the lmms sources, since they have the same languages.

[JohannesLorenz]

A few technical details about the equation solver:

• it has only a few 100 lines of code
• it requires g++ >= 4.8 or clang >= 3.3
• it needs a lot of memory/cpu power while compiling, but it runs fast once it is compiled

[zonkmachine]

It sounds cool and I don't think a slower app is a problem as that is a natural part of the deal with scripted stuff, but wouldn't support for one of the bigger languages csound or pd be a more natural path?

[JohannesLorenz]

Maybe, but I think they might be more complicated to code into LMMS, and for many C developers, they might be more work to learn. Also, you can not easily port them into LADSPA or similar effects, as they are not really C.

[eagles051387]

I think in addition to what you are saying here Johannes what would be
helpful im sure is a web based version of the doxygen commented code for
ease of reference correct?

On Tue, Apr 1, 2014 at 3:23 PM, JohannesLorenz notifications@github.comwrote:

I had the following idea: we have many plugins, but if you want to write
your own for just one song, there is a long way to go:

• Write the plugin in C/C++
• Register the plugin
• Wait for the next LMMS version ;)
• Hope that it will also compile on Windows etc ;)

Anyways, I have once written a C interpreter. It builds a so called
"abstract syntax tree" from a string input once, and this tree can then be
executed multiple times, like a compiled program. We could predefine
controllers and effects that use such equations.

Example: Amplifier:

Predefined: An effect with stereo in and out, 16 knobs, and one string
input.
What you do: Type "for(int i = 0; i < buf_size; ++i) { out_l[i] =
in_l[i] * k[0]; out_r[i] = in_r[i] * k[0]; }" into the string input.
Meaning: out_l, out_r, in_l, in_r are predefined variables for the
in/out buffers. buf_size is predefined for the buffer size. k[0] is knob 0.

Other examples: Controllers, e.g. that delay the controller signal, apply
inertia to it or whatever.

What do you think about such an interpreter?

Note: I know scripts are usually slower. But once a script proves as
usable, we could just move the script code into the lmms sources, since
they have the same languages.

Reply to this email directly or view it on GitHubhttps://github.com//issues/551
.

Jonathan Aquilina

[JohannesLorenz]

Am Dienstag, 1. April 2014, 07:31:36 schrieb eagles051387:

I think in addition to what you are saying here Johannes what would be
helpful im sure is a web based version of the doxygen commented code for
ease of reference correct?

You mean so you could copy parts of existing C code into the script editor?
Probably, I think.

Independent from that, web based versions of code/Doxygen are actually never a
bad idea, imo.

[eagles051387]

That would eliminate the need to compile them when you compile lmms.

@toby Could we host doxygen generated documentation on SF?

On Tue, Apr 1, 2014 at 4:38 PM, JohannesLorenz notifications@github.comwrote:

Am Dienstag, 1. April 2014, 07:31:36 schrieb eagles051387:

I think in addition to what you are saying here Johannes what would be
helpful im sure is a web based version of the doxygen commented code for
ease of reference correct?

You mean so you could copy parts of existing C code into the script
editor?
Probably, I think.

Independent from that, web based versions of code/Doxygen are actually
never a
bad idea, imo.

Reply to this email directly or view it on GitHubhttps://github.com//issues/551#issuecomment-39212830
.

Jonathan Aquilina

[diizy]

As it happens, I was just thinking about something similar just the other day.

There's some considerations:

1. Will the scripting engine (or any scripting engine) be fast enough for realtime audio processing? Remember that we have to be able to process audio in realtime, with a reasonably low latency, and reasonably low CPU consumption. Any scripting engine is going to be extra overhead there. If it's only for very simple effects and equations, if it's used sparingly, it might work. But this would again limit the functionality... if the engine takes too much CPU, no one is going to use it.
2. In order to be useful from a user's perspective, the chosen language should be easy enough to master for a non-programmer. C is often considered a bit clunky. From this perspective, something like Python would be ideal.

So of course, these two goals are in direct contradiction with each other. C syntax is harder, whereas Python is too slow...

Ideally, we should develop our own scripting language, which is optimized for audio work, and has a syntax suitable for non-programmers (like python). But then, do we really have time/resources to do that?

Anyway, this could all be implemented as an effect plugin with an embedded interpreter, so there's really nothing stopping you from doing this within the existing plugin framework of LMMS.

[softrabbit]

IMO, using an existing language would be wise, one with a user base, tutorials, years of development and so on. Before starting on an in-house solution, it'd be good to do some serious evaluation of the existing alternatives, on points like:

• language (computer scientists like lisp-y things for some reason, that might possibly be even worse than C?)
• embeddability (obviously)
• flexibility, e.g. is it audio-only or can it do other things, too?
• performance

There are 7 BSD|(L)GPL alternatives in the list over there, maybe one of them fits:
http://en.wikipedia.org/wiki/Comparison_of_audio_synthesis_environments

[diizy]

On 04/02/2014 10:33 AM, Raine M. Ekman wrote:

IMO, using an existing language would be wise, one with a user base,
tutorials, years of development and so on. Before starting on an
in-house solution, it'd be good to do some serious evaluation of the
existing alternatives, on points like:

• language (computer scientists like lisp-y things for some reason,
  that might possibly be even worse than C?)
• embeddability (obviously)
• flexibility, e.g. is it audio-only or can it do other things, too?
• performance

There are 7 BSD|(L)GPL alternatives in the list over there, maybe one
of them fits:
http://en.wikipedia.org/wiki/Comparison_of_audio_synthesis_environments

I've heard some good things about ChucK, and it seems like an
interesting concept... the entire language is concurrent, and makes it
very easy to work with timing.

http://en.wikipedia.org/wiki/ChucK

[JohannesLorenz]

Just btw: Why not let the plugin compile C code? There are only two cases:

• The user has the needed version of g++ and all the libs (easy to check)
  -> it can be compiled.
• Otherwise, it can not be compiled, the user gets a warning on loading the
  project file.

This will cause some (very few) people to not be able to load some songs, but
if you'd write your plugin the old style and develop it with the song, they
could not load your songs as well.

So what's so bad about compiling?

[tobydox]

...99% of Windows and OS X users won't be able to use that feature. Besides that, compiling of course would be the best option for optimal performance.

[JohannesLorenz]

Am Mittwoch, 2. April 2014, 03:40:01 schrieb Tobias Doerffel:

...99% of Windows and OS X users won't be able to use that feature.

Agreed, but the same argument would count if I give a Windows user my song
together with a plugin that the song needs. So this is no reason agains
compiling imo.

That's why I'd suggest trying to write code that does this.

[diizy]

On 04/02/2014 01:40 PM, Tobias Doerffel wrote:

...99% of Windows and OS X users won't be able to use that feature.
Besides that, compiling of course would be the best option for optimal
performance.

Well... Windows/OS X users won't be able to use LinuxVST at all, OS X
users won't be able to use Windows VST's, and Linux users... well, Linux
users can use most Windows VST's, but there are definitely some that
don't work via Wine even though they work on the Windows version of LMMS.

This is just my opinion, but personally, I don't see it as much of a
problem if LMMS has some advantages on Linux vs. other platforms. Linux
is free, and 99% of our core features are still cross-platform. Maybe
it'll encourage some people into trying Linux - not that that should be
a goal of LMMS in itself, but if it happens as a side effect, I'm fine
with it.

[softrabbit]

Here's a volume control plugin in FAUST (minus some metadata and a library import):

smooth(c)   = *(1-c) : +~*(c);
gain        = vslider("[1]", 0, -70, +4, 0.1) : db2linear : smooth(0.999);
process     = *(gain);

I think something in this direction is what's needed if we talk compiled code:

• No explicit looping over the buffer
• The slider value is just a variable (that might or might not be re-checked for every sample, it doesn't matter for the algorithm)

IOW, hide all the stuff that isn't strictly needed and let the plugin writer focus on the DSP code. Maybe restrict him/her to what's in math.h and no more. Or maybe just use FAUST, if the syntax is good enough.

Maybe this actually is something that could be in a separate program, with suitable hooks to make it work smoothly with LMMS but also usable as a stand-alone code generator?

[unfa]

Could a compiled script's binary code be kept in MMP file so the plugin can
run even on a system that doesn't have libs and compiler to make the code?

That however could be a major security risk.
On 2 Apr 2014 12:00, "JohannesLorenz" notifications@github.com wrote:

Just btw: Why not let the plugin compile C code? There are only two cases:

• The user has the needed version of g++ and all the libs (easy to check)
  -> it can be compiled.
• Otherwise, it can not be compiled, the user gets a warning on loading
  the
  project file.

This will cause some (very few) people to not be able to load some songs,
but
if you'd write your plugin the old style and develop it with the song,
they
could not load your songs as well.

So what's so bad about compiling?

Reply to this email directly or view it on GitHubhttps://github.com//issues/551#issuecomment-39311522
.

[diizy]

On 04/03/2014 02:38 AM, unfa wrote:

Could a compiled script's binary code be kept in MMP file so the
plugin can
run even on a system that doesn't have libs and compiler to make the code?

That however could be a major security risk.

Yeah let's not do that.

[eagles051387]

Shouldnt the presets effects etc have their own extension so we can
decipher between projects lmms templates presets etc?

On Thu, Apr 3, 2014 at 1:38 AM, unfa notifications@github.com wrote:

Could a compiled script's binary code be kept in MMP file so the plugin can
run even on a system that doesn't have libs and compiler to make the code?

That however could be a major security risk.
On 2 Apr 2014 12:00, "JohannesLorenz" notifications@github.com wrote:

Just btw: Why not let the plugin compile C code? There are only two
cases:

• The user has the needed version of g++ and all the libs (easy to check)
  -> it can be compiled.
• Otherwise, it can not be compiled, the user gets a warning on loading
  the
  project file.

This will cause some (very few) people to not be able to load some songs,
but
if you'd write your plugin the old style and develop it with the song,
they
could not load your songs as well.

So what's so bad about compiling?

Reply to this email directly or view it on GitHub<
https://github.com/LMMS/lmms/issues/551#issuecomment-39311522>
.

Reply to this email directly or view it on GitHubhttps://github.com//issues/551#issuecomment-39397444
.

Jonathan Aquilina

[JohannesLorenz]

Am Mittwoch, 2. April 2014, 16:38:37 schrieb unfa:

Could a compiled script's binary code be kept in MMP file so the plugin can
run even on a system that doesn't have libs and compiler to make the code?

That however could be a major security risk.

Putting in binary code is an extremely bad idea imo:

• We get (large) binary code in our XML files. Thus, you can not put them
  under revision control (you can, but...).
• We start getting closed source. Only one related problem: If a newer
  version of the plugin is needed, you need to contact the author for
  recompiling. However, the author might not respond (I've seen multiple
  projects dying because of this). So you can throw away all project files that
  used this plugin.
• Security is even the smallest problem I'd worry about here.

LMMS is open source and should never encourage people to develop closed source
plugins (my opinion). I'd suggest to put script/program source code into the
XML files.

[JohannesLorenz]

This is just my opinion, but personally, I don't see it as much of a
problem if LMMS has some advantages on Linux vs. other platforms. Linux
is free, and 99% of our core features are still cross-platform. Maybe
it'll encourage some people into trying Linux - not that that should be
a goal of LMMS in itself, but if it happens as a side effect, I'm fine
with it.

I agree here.

Also note that popular plugins can later still be compiled into LMMS, so
Windows users would just have to wait a bit longer.

Toby, will you accept code that allows C++ source code in XML project files?

[eagles051387]

If you want a language that is good for concurrency check out google's
golang

from what I have seen the syntax is very easy to work with too.

On Wed, Apr 2, 2014 at 10:48 AM, Vesa V notifications@github.com wrote:

On 04/02/2014 10:33 AM, Raine M. Ekman wrote:

IMO, using an existing language would be wise, one with a user base,
tutorials, years of development and so on. Before starting on an
in-house solution, it'd be good to do some serious evaluation of the
existing alternatives, on points like:

• language (computer scientists like lisp-y things for some reason,
  that might possibly be even worse than C?)
• embeddability (obviously)
• flexibility, e.g. is it audio-only or can it do other things, too?
• performance

There are 7 BSD|(L)GPL alternatives in the list over there, maybe one
of them fits:
http://en.wikipedia.org/wiki/Comparison_of_audio_synthesis_environments

I've heard some good things about ChucK, and it seems like an
interesting concept... the entire language is concurrent, and makes it
very easy to work with timing.

http://en.wikipedia.org/wiki/ChucK

Reply to this email directly or view it on GitHubhttps://github.com//issues/551#issuecomment-39303433
.

Jonathan Aquilina

[pgiblock]

I for one wouldn't support that. It is way too difficult, in my opinion, to sanitize C++ code so you can trust the resulting binary to not be malicious. I could share project files that infect users computers, leak personal data, or destroy the machine. I would prefer to see a restricted dialect, perhaps not even C-derived. This could be compiled to machine language or source-to-source compiled: instrument-script => C => assembly.

[diizy]

On 04/03/2014 10:06 AM, Paul Giblock wrote:

I for one wouldn't support that. It is way too difficult, in my
opinion, to sanitize C++ code so you can trust the resulting binary to
not be malicious. I could share project files that infect users
computers, leak personal data, or destroy the machine. I would prefer
to see a restricted dialect, perhaps not even C-derived. This could be
compiled to machine language or source-to-source compiled:
instrument-script => C => assembly.

Going to have to agree here...

[JohannesLorenz]

Thanks Paul, I've overseen this danger. However:

What if we'd disallow any #includes and #defines? That way, the programmer
could not do anything malicious I'd guess?

On 04/03/2014 10:06 AM, Paul Giblock wrote:

I for one wouldn't support that. It is way too difficult, in my
opinion, to sanitize C++ code so you can trust the resulting binary to
not be malicious. I could share project files that infect users
computers, leak personal data, or destroy the machine. I would prefer
to see a restricted dialect, perhaps not even C-derived. This could be
compiled to machine language or source-to-source compiled:
instrument-script => C => assembly.

Going to have to agree here...

[diizy]

On 04/03/2014 11:29 AM, JohannesLorenz wrote:

Thanks Paul, I've overseen this danger. However:

What if we'd disallow any #includes and #defines? That way, the
programmer
could not do anything malicious I'd guess?

The list of things we'd have to disallow is a bit longer I'm afraid...

#includes just bring code in from headers or installed libs, disallowing
them doesn't matter at all when the user can just enter that code into
the script manually.

Same with #defines, defines are just macros that expand to code, and
disallowing them does nothing as the user can just enter that code manually.

We'd basically have to disallow huge swaths of code... and there'd still
be potential for exploitation. We'd be bound to miss something, someone
would figure out a way around the restrictions (have you seen all the
ways C/C++ code can be obfuscated? trying to account for all the ways
malicious code can be written would be a huge, neverending task).

I don't really want .mmp files to be able to execute arbitrary binary
code on my computer. That's the kind of thing that's bitten Microsoft in
the ass more times than we can count... if this were implemented, I
could never open any .mmp file downloaded from the internet. The sharing
platform would become a huge target for people looking for members for
their botnets.

[eagles051387]

I think the only language that we could really work with which is platform
neutral would be java :( but then again its not really a good way to take
audio, but im impressed how well things perform for this virtual organ
platform I know of called jorgan

On Thu, Apr 3, 2014 at 10:40 AM, Vesa V notifications@github.com wrote:

On 04/03/2014 11:29 AM, JohannesLorenz wrote:

Thanks Paul, I've overseen this danger. However:

What if we'd disallow any #includes and #defines? That way, the
programmer
could not do anything malicious I'd guess?

The list of things we'd have to disallow is a bit longer I'm afraid...

#includes just bring code in from headers or installed libs, disallowing
them doesn't matter at all when the user can just enter that code into
the script manually.

Same with #defines, defines are just macros that expand to code, and
disallowing them does nothing as the user can just enter that code
manually.

We'd basically have to disallow huge swaths of code... and there'd still
be potential for exploitation. We'd be bound to miss something, someone
would figure out a way around the restrictions (have you seen all the
ways C/C++ code can be obfuscated? trying to account for all the ways
malicious code can be written would be a huge, neverending task).

I don't really want .mmp files to be able to execute arbitrary binary
code on my computer. That's the kind of thing that's bitten Microsoft in
the ass more times than we can count... if this were implemented, I
could never open any .mmp file downloaded from the internet. The sharing
platform would become a huge target for people looking for members for
their botnets.

Reply to this email directly or view it on GitHubhttps://github.com//issues/551#issuecomment-39426850
.

Jonathan Aquilina

[softrabbit]

Don't look at what to disallow, start from nothing and then allow things until you're happy. The C parser mentioned at the start would actually be pretty useful for sanitizing the code, if the compiler input is then built from the parse tree it's possible to only let safe things pass.

[diizy]

On 04/03/2014 12:31 PM, Raine M. Ekman wrote:

Don't look at what to disallow, start from nothing and then allow
things until you're happy. The C parser mentioned at the start would
actually be pretty useful for sanitizing the code, if the compiler
input is then built from the parse tree it's possible to only let safe
things pass.

I think we'd only need to allow a x number of variables and ways to
manipulate them with mathematic functions, equations etc. Everything
else is superfluous.

Some predefined variables: input, output, sample rate, control values
from knobs. Add to that some user-definable variables: a limited amount
of variables/arrays which the user can define and use.

Some predefined functions: sin, cos, square root, addition,
multiplication, power, etc. Control structures: if clauses, for/while loops.

Upside would be that the syntax could be made very simple this way.

[eagles051387]

When you say Linux Vst's cant you still compile them and they should run on
windows or OSX ?

On Wed, Apr 2, 2014 at 1:03 PM, Vesa V notifications@github.com wrote:

On 04/02/2014 01:40 PM, Tobias Doerffel wrote:

...99% of Windows and OS X users won't be able to use that feature.
Besides that, compiling of course would be the best option for optimal
performance.

Well... Windows/OS X users won't be able to use LinuxVST at all, OS X
users won't be able to use Windows VST's, and Linux users... well, Linux
users can use most Windows VST's, but there are definitely some that
don't work via Wine even though they work on the Windows version of LMMS.

This is just my opinion, but personally, I don't see it as much of a
problem if LMMS has some advantages on Linux vs. other platforms. Linux
is free, and 99% of our core features are still cross-platform. Maybe
it'll encourage some people into trying Linux - not that that should be
a goal of LMMS in itself, but if it happens as a side effect, I'm fine
with it.

Reply to this email directly or view it on GitHubhttps://github.com//issues/551#issuecomment-39317297
.

Jonathan Aquilina

[diizy]

On 04/06/2014 02:19 PM, eagles051387 wrote:

When you say Linux Vst's cant you still compile them and they should
run on
windows or OSX ?

Well, that depends of course on whether those LinuxVST's are also
WindowsVST's or MacVST's. Has the author made them to be compilable
cross-platform. And of course, whether they're open source. Not all
LinuxVST's are open source.

Also, even when we implement LinuxVST support, that doesn't necessarily
automatically make MacVST support work.

[JohannesLorenz]

One different idea:

1. You write C code
2. the C parser reads this C code it and generates (equivalent) C code from
   it
3. a compiler finally compiles the generated C code

This should prevent anything malicious (I guess, since the things we allow are
restricted by the C parser) and would be as fast a compiled in library.

Any comments about this?

[QrchackOfficial]

Have you guys considered LUA? It's designed for scripting, very lightweight and the only dependency is standard C library. Also 100% cross-platform.

[musikBear]

performance and security... nough said ..except, it would create a humongus confusion in explaining
'What Is LMMS And What Does It Do'.

imho no

[Sti2nd]

It's pretty cool, and obviously for advanced users 🙏 We don't always need to target newbies, you know @musikBear