Any offline data processing?

[sadko4u]

The LV2 standard provides the LV2_Worker_Interface extension that allows to run non-RT-safe offline jobs like loading audio samples etc.

In CLAP, I found two extensions:

1. thread-pool
2. timer-support

Both of them don't match well because:

1. The line #26 of the documentation to the thread-pool extension mismatches the request_exec definition
2. As I understand, the request_exec performs synchronously while LV2_Worker_Interface does the asynchronous work.
3. For the thread-pool extension, there is no way to transfer some supplementary data in to the request_exec that will be passed to the exec function.
4. It could be possible to simulate the case with the timer-support extension but the timer-support provides only periodic timer facility and there is no possibility to define one-shot timers or timers with manual restart.

In VST 2.4 I was required to introduce my own thread that executes the jobs asynchronously.
Should I go the same way with CLAP or is there some extension that will fullfill my expectations?

[baconpaul]

Timer support allows idle scheduling into the main thread. The thread pool extension is to stop contention among audio processing plugins running in a daw which has a scheduled strategy. Neither do what you want

in surge when we have to do something which is neither ui thread nor real-time (like loading a patch) we just use std::thread and atomics. To the best of my knowledge we have not wrapped that in an extension in clap nor have I heard folks planning to.

for the specific case of gathering file resources there is an extension but that’s about persistence and portability not about threading

if there was such an extension I’m not sure it would be useful since compensating for the host not supporting the extension would require you to write the std::thread code anyway, most likely.

curious what my colleagues think also. And happy holidays!

[sadko4u]

in surge when we have to do something which is neither ui thread nor real-time (like loading a patch) we just use std::thread and atomics.

It's not always trivial solution when we want to keep the number of threads as less as possible. In a very simple implementation there should be one additional thread created per each plugin instance, which is not good. In a non-trivial implementation there will be one thread per shared object, which is a pretty good compromise but still not well if you provide an individual shared object for each kind of plugin.

if there was such an extension I’m not sure it would be useful since compensating for the host not supporting the extension would require you to write the std::thread code anyway, most likely.

Also we should consider the maximum number of threads/processes being limited for the user with pretty small values like 1024 on several Linux distributions by default. Having huge amount of plugins in the project will cause serious problems with missing enough amount of resources for threading which is the bottleneck.

When the host controls the asynchronous task execution, we should not worry about the thread management, especially the number of threads. But the lack of such extension can be considered to the fallback case I've described above.

[baconpaul]

Yeah these are exactky the reasons the real-time audio thread thread pool exists. Avoiding that contention at processing time is a real performance boost. Plus real-time scheduling on asymmetric chips like the apple soc is really hard and the daws do it well.

We haven’t considered it for non audio threads though as far as I know.

[tim-janik]

in surge when we have to do something which is neither ui thread nor real-time (like loading a patch) we just use std::thread and atomics.

It's not always trivial solution when we want to keep the number of threads as less as possible. In a very simple implementation there should be one additional thread created per each plugin instance, which is not good. In a non-trivial implementation there will be one thread per shared object, which is a pretty good compromise but still not well if you provide an individual shared object for each kind of plugin.

I know @swesterfeld has been running into the same problem while implementing CLAP for https://github.com/swesterfeld/spectmorph, maybe its time for someone to draft a CLAP extension modeled after the LV2 worker extension?

[Bremmers]

I'm one of the very few developers from the mac/win/vst/au world who implemented the LV2 worker extension. At first I got it wrong and an LV2 guru had to explain it to me.

I think the main point of the extension is to make it easy for plugin developers. All other plugin formats can do without this, so things like thread counts aren't really a problem I think?

Commercial host developers probably wouldn't do more than simply create a new thread for each plugin that wants one (because it's just for one plugin format). Or they ignore it completely, which would go unnoticed because none of the existing VST/AU plugins that are converted to CLAP would use it.

The net result of such an extension for CLAP would probably be that plugins coming from the linux/LV2 world don't work right in commercial hosts.

[sadko4u]

The net result of such an extension for CLAP would probably be that plugins coming from the linux/LV2 world don't work right in commercial hosts.

That's not true. When developing the LV2 plugin, I'm pretty ready that the host can not support the worker extension and mark it as an optional in LV2 TTL requirements. If the host doesn't support the worker extension, I just do the suff like simply create a new thread for each plugin that wants one. It's not too hard to support.
But if we can avoid an extra thread per plugin, it is better to do it. I have had a very bad experience when kdenlive ran out of number of possible user threads/processes. And that was VERY sick because the overall system behaviour became unstable because almost all of the programs are designed so they don't expect that they can not fork()/vfork() or create yet another lightweight process called thread.

[baconpaul]

So basically the extension would be “run async(job handle)” on the host and “process callback(job handle)” on the plugin? The guarantee would be that process callback happens on a non audio thread and ideally a non main thread? The naive implementation in a plugin of missing host support would be std thread of process callback running detached? But the host could if it wanted spin up some small n of waiting threads to process this.

I don’t think there’s anything more to it than that is there?

[sadko4u]

Actually, yes. We need a lock-free function to submit some callback function pointer with some payload to the host's queue which will run this callback and pass the payload data to the callback function in a non-RT-thread (the host is responsible for what it should be: a single thread, or a small thread pool).
Also note that worker's queue can be out of free space or the lock-free algorithm can not acquire atomic lock at this moment, so the submit function should notify the plugin about the scheduling error so it could retry the submit on the next process call.

[baconpaul]

The host certainly needs that but why does the api?

it seems to me the api is really just bool submit (void*) on the host and void process(void *) on the plug. Plus a comment that submit can be called from any thread and is lock free in a robust implementation, right?

[sadko4u]

it seems to me the api is really just bool submit (void*) on the host and void process(void *) on the plug.

Oh yes, since void * can be anything, also containing the pointer to the function to execute, it's OK.

Plus a comment that submit can be called from any thread and is lock free in a robust implementation, right?

Seems to be pretty well

[baconpaul]

Yeah the other callback apis all call to a single function exactly. They use an id of some form but a void * works just as well in this case,

let’s see what my colleagues think but I think we now understand the proposal at hand pretty well!

[abique]

Why don't you simply create a background thread yourself?

1. new thread
2. do the job there
3. set a "flag" that your job is finished
4. host->request_callback()
5. handle job completion on main thread

It is the wrong approach to rely on the host for doing this because:

1. it is easy to implement
2. we can implement it in clap-helper and it will work in every host
3. if you rely on the host to provide this utility then what do you do if the host doesn't implement it?

[abique]

I'm not sure what would be the advantage of having the host doing this job for the plugin?
Even for the plugin it increases the testing surface with all the different hosts.

[sadko4u]

Why don't you simply create a background thread yourself?

I already have explained why this solution is not pretty good here

[abique]

Why don't you simply create a background thread yourself?

I already have explained why this solution is not pretty good here

Still, I'm not convinced.
The plugin would not keep a thread alive forever, it'd just be for doing one job and then it'd terminate.

Also even if we add an extension for this task, the plugin can't simply assume that every host will implement it and will need fallback code anyway. So I don't see a benefit.

[abique]

I close this issue now.
If anyone want to pursue this idea, then he'd have to make a PR with the host extension, as well as a robust design for canceling tasks from either host side or plugin side, with full thread specification, fallback code for the clap-helpers when the host impl isn't available etc...