Expose all racks parameters with OSCquery

[avilleret]

Hi,

thanks a lot for making this great tool open source !
One thing I might need in the near future, is to control VCVRack with a sequencer.
I could do it with MIDI. But nowadays I'm more confortable with OSCQuery protocol.

A good, crossplatform and opensource library for that is libossia.
I started thinking about making a plugin that exposes all rack parameters with OSCQuery.
But after a quick look into sources, it seems that rack::modules vector isn't accessible from plugins API without some rewriting, isn't it ?
On the other side, if putting OSCQuery into VCVRack needs some code refactoring, what about including it directly into the core engine ?

Would it be something @AndrewBelt will support ? (I mean give some hint and accept some pull-request)

[AndrewBelt]

I've never used OSC, so I'd have to learn how to use it, get some hardware, and test it. The easy part would be writing the code, which I might as well do to avoid inconsistencies with such an easy task. We can't use libossia because I believe the license restricts it from use when a proprietary plugin is loaded. Is that correct? Is there an alternative? Is it possible for you to make an alternative library?

And no, this shouldn't be a Rack plugin because plugins should only add modules which don't interact with other modules.

[avilleret]

concerning the licence, @jcelerier might have an idea, I already asked him, but it would be annoying if a commercial software can't link to libossia.
And about the hardware, you don't need anything specific.
You can experiment locally with Score for example.

[jcelerier]

We can't use libossia because I believe the license restricts it from use when a proprietary plugin is loaded

Hi, libossia is under the LGPL which means that it can be used even by proprietary software (and AFAIK the GPL / LGPL licenses only concern redistribution, not execution... else no one would be able to build for instance LGPL-licensed externals for Max/MSP or LGPL VST / AU plug-ins).

For instance Max/MSP uses FFMPEG which is under the LGPL too : https://cycling74.com/downloads

[jcelerier]

I've never used OSC, so I'd have to learn how to use it, get some hardware, and test it.

I think that most OSC implementations are software. If you have a tablet or something like this there's TouchOSC which allows quick testing : https://hexler.net/software/touchosc ; OpenStageControl is also a nice free app for this : http://osc.ammd.net/ . Also, most "creative" environments (Max, Pd, OpenFrameworks, etc...) do support it in some way so there's a lot of choice :p

[AndrewBelt]

The license appears to be CeCILL-C, so it is odd that you say it is LGPL since you wrote most of it. But if you say so, I won't argue with that.

I estimate I will get around to this in ~6 months, if it commercially beneficial, i.e. if the number of VCV plugins sold will increase by a significant amount if this feature exists, because the complexity of the build system and licensing would be increased and incur continuous development costs.

[jcelerier]

The license appears to be CeCILL-C

Yes, it's basically a french version of LGPL, but which has been made with a goal of explicit compatibility with it (https://en.wikipedia.org/wiki/CeCILL : 'CeCILL-C, for "component" software, which is fully compatible with the FSF's LGPL license.').

Though if it causes problems, relicensing explicitely to LGPL can be discussed with the contributors (and their respective employers which is, I suppose, the most problematic step !).

@avilleret maybe (when there's time) we could provide a pull request ?

Thanks for the interest in any case !

[AndrewBelt]

Perhaps maintaining your own fork is a better solution if you want OSC support. When I have time and I evaluate the value/cost of having this large dependency in Rack, I may merge it.

[avilleret]

@avilleret maybe (when there's time) we could provide a pull request ?

I'll try to make a draft asap and then improve it if my project needs it

[avilleret]

hi !
I've spent some time on implementing libossia into Rack.
Here it is : https://github.com/avilleret/Rack/tree/ossia
It is quite straightforward.
All parameters are automatically exposed to OSCQuery with no changes in modules' code.
By default, parameters are called param.1, param.2 etc... which is not very user friendly.
That's why I added an argument to createParam function to specify a custom param name (see https://github.com/avilleret/Rack/blob/ossia/include/rack.hpp#L49).
I've ported all the Fundamental modules here : https://github.com/avilleret/Fundamental/tree/ossia
Yes, the diff is not huge at all...
The OSSIA parameter is accessible from the plugins API, so man can redefine it as he want (with type, unit, clipping, tags, description...).

There is still some work to do :

1. on our (I mean libossia team) side, we have to provide a just enough library to expose parameter with oscquery, then we can update the Rack build system to use it easily
2. since there are some switches and selectors in the Rack's GUI, it would be nice to use boolean and string (with enum) parameter
3. Rack's parameter don't have unit, OSSIA parameter does and unit conversion is handled by the lib itself (for example from Hz to ms) so maybe we can add unit to Rack but this require more work and some refactoring.

I'll wait for 1. to be done before submitting a PR.
Feedback are welcome in the meanwhile :-)

[petervanhaaften]

I would just like to add that it would be incredibly useful to implement OSCquery in Rack!

[AndrewBelt]

"+1 posts are not allowed. You may vote on feature requests by using the Thumbs Up/Down reaction on the first post."

[lazzarello]

@AndrewBelt You expressed interest higher up but were unfamiliar with OSC. Here's the rundown

• Open Sound Control is a data format to express an address and arguments. Addresses have a namespace, arguments have types.
• OSCQuery appears (I was not familiar with it until this thread) to be a device schema which describes all the address => parameter mappings of any device and a query interface for other programs to extract that schema (and values?)
• OSC is typically transmitted over a UDP socket, though the protocol is flexible. TCP is typically supported but I think that is a bit silly for realtime controls. There is also a direct serial implementation which can run on a microcontroller maintained by CNMAT at Berkeley.
• OSC is not bound to a single protocol, which means the resolution of each parameter is only constrained by the bandwidth of your transport protocol. This is a huge improvement over 7bit MIDI CCs, which are useless to control tuning beyond a couple octaves.

I'm excited for @avilleret 's fork. I found this thread searching for prior work before I began to do exactly the same thing.

Of note, I own a DIY controller with a 4x4 banana jack patch matrix which can forward data between arbitrary OSC addresses. it's quite cool and would be perfect for a modular setup like VCVRack.

[AndrewBelt]

As expressed in #575 (comment), I'm reconsidering my earlier quote

And no, this shouldn't be a Rack plugin because plugins should only add modules which don't interact with other modules.

since the grossness would be confined to a single global std::vector<Module*> gModules defined in engine.hpp. Then you could do whatever you want in your plugin, with whatever libraries you need. I'd highly discourage using gModules for any other purpose, but exposing it for an OSC plugin would make sense.

As for parameter names, I like the idea, but not right now.

[WIZARDISHUNGRY]

I'm considering the utility of a (Rack-internal) OSC bus to allow Soundhack-derived modules to broadcast/receive saved spectra.

[AndrewBelt]

#664 is relevant to this. Note that the OSC module will need to access a list of ModuleWidgets, not Modules, since you actually want to control the knob position, not the Param itself. Because of this, you're actually to get started now instead of when Rack 0.6 releases, since you can get a list of modules with gRackWidget->moduleContainer with appropriate dynamic_casting.

[bafonso]

What an interesting discussion. I am wondering if going for a fuller implementation of algorithmic interaction won't be a better use of time. OSC then is simply just one of the protocols you can use to interact with rack. To be more clear, this api would allow in real time:
1 - query for all modules available
2 - load and connect modules
3 - query and manipulate module's parameters

I'd love to see an OSC implementation of this but having this possibility alone would be quite interesting.

[petervanhaaften]

OSCQuery implementation could also be relevant to #78.

A 'preset module' could grab current state of all parameters via OSCQuery & save those as a plain text file. A good example of this in practice is the 'Cue Manager' module for the Jamoma Library in Max/MSP. In fact, by first implementing OSCQuery, the Jamoma cue manager could be used in the interim to handle VCVRack patch 'presets' or 'states'.

[JTriggerFish]

I would also like to request that the API offers the modules a way to register string names for their parameters.
I would like to add extensions to Rack to use Ableton Push2 to control the module parameters and this can only be done properly if parameter names are known.
Of course this is not specific to Push2 but a general thing for external controllers especially if they have a screen.

[AndrewBelt]

The engine and UI have been mostly separated in Rack v1 and moreso in Rack v2, so it makes sense to think about this feature request at some point. I've never used the OSC protocol and haven't read the standard yet, but I'll make the following guesses.

• Rack would need to launch a TCP server upon launch.
• There's a feature in the OSC protocol to ask for a list of parameter IDs and names. Rack's OSC server could respond with some tuple (module ID, param ID), and the name "VCV VCO-1 Frequency". Or does OSC work with integer IDs only?
• Is that it, just listing, viewing, and changing parameter values?

[jcelerier]

Hello, that is great to hear :)

There are two different things :

• OSC which is a low-level, message-based protocol which generally uses UDP (though that is not mandated - it is also sometimes used directly over serial port, over TCP, etc...).
  An OSC message would generally look like :

<address>     <comma> <list of types>   <list of arguments>
/foo/bar      ,if                       123      435.345

the arguments are written in their binary form in big endian, and zero-separated / zero-padded to a boundary of 4 bytes.
That is all for OSC, there are only a few types (int, float, string, timestamp ...) and a bundle feature to allow for sending multiple messages at once.

• OSCQuery which adds reflection information and allows other apps to query what are the available addresses of an OSC app, what are its types, bounds, etc etc. The exchange format is JSON over WebSockets (so TCP).

The spec for that is here : https://github.com/Vidvox/OSCQueryProposal and we have a LGPL implementation in libossia.

Is that it, just listing, viewing, and changing parameter values?

that would already be great as it allows anyone to build remote controls :)

[AndrewBelt]

UDP is fine. I'd need to either write my own OSC implementation or find a decent MIT-like one.

Is there an interface for a client to query a list of parameters? How are parameters identified? By integers or strings?

[jcelerier]

• a decent MIT-like one.

OSCPack is a BSD-licensed OSC implementation : http://www.rossbencina.com/code/oscpack which is used in a lot of software.

• Is there an interface for a client to query a list of parameters?
  OSC does not really have a way to "pull" a parameter, only send messages.
  e.g. with raw OSC there is no proper way to have a knob with a parameter, for which a client can query a value ; instead, the client would set the value it wants to the knob.

That is what OSCQuery adds on top of OSC : a notion of parameter that can be queried, etc. (in OSCQuery it is possible to query multiple parameters at once by querying a parent node in the tree, e.g. if you want /foo/lfo.1/frequency and /foo/lfo.1/amplitude you query /foo/lfo.1)

• How are parameters identified? By integers or strings?
  By string (URL-like addresses - /foo/bar, /my-rack/lfo.1/frequency, etc etc)

[AndrewBelt]

Oh, so OSC is client-to-server only? Yeah, I'd need to support bidirectional communication, which is what I assume OSCQuery does.
Is OSCQuery widely supported on iPad apps, controllers, etc, or is it a fairly new proposal?

By string (URL-like addresses

Awesome, that works nicely for Rack, where modules (groups of parameters) are dynamic (can be added or removed while running).

[jcelerier]

Yeah, I'd need to support bidirectional communication, which is what I assume OSCQuery does.

it is indeed

Is OSCQuery widely supported on iPad apps, controllers, etc, or is it a fairly new proposal?

A few apps support it in the media art scene (mostly on desktop software such as VDMX, Millumin, Vezér, ossia score) so I would not be able to say you how much it is used on iApps frankly. There is an Objective-C implementation I think.

[avilleret]

one big advantage of using libossia is that it allows to expose the application namespace through different protocol (OSCQuery and OSC among others) at the same time (or not) without any effort.

[AndrewBelt]

libossia looks really nice, and I'd ask for licensing if that's even possible (if the contributors can be enumerated). Otherwise I'll have to derive OSCPack to add OSCQuery support, or ask for help from the community to do so.

[jcelerier]

hm, libossia cannot be relicensed easily, as part of work on it was done under contract with a french university and multiple companies. I think that in that case it may make more sense to make a clean room reimplementation

[AndrewBelt]

Makes sense, that's what I thought. Should be easy enough to make a lightweight implementation.

[AndrewBelt]

I can't see a reason that this feature needs to be built into Rack itself. If it was, it would be delivered as a Core module anyway, and there's nothing Core can do that a third-party plugin can't.

In order to get around licensing issues with libossia (or any other OSC implementation), someone can make a third-party plugin that is GPL or LGPL. This would be a better option than including this into Rack.

[jcelerier]

Hm, I've looked around the current API headers a bit but could not find the main thing necessary for this to work : how can a plug-in enumerate (and, even better, get notified of changes) of every parameter / knob / module ... in a given rack session ? I've grepped a bit for global instances and things like that but could not find any.

[AndrewBelt]

You should use Engine methods. For example,

Module* module = APP->engine->getModule(moduleId);
float value = module->paramQuantities[paramId]->getValue();

It looks like there's one step missing: enumerating all the modules.
Just so I don't forget, open an issue for me to add an Engine::getModuleIds() method to Rack v2.
You can already enumerate paramIds with module->paramQuantities.size().

You cannot be notified for changes. You will need to periodically query what you need, perhaps every 512 samples or so to detect new Modules.

Manipulating Modules is safe as long as you're in the process() method, because they are only added/removed when no threads are processing.
If you want to hold onto a parameter over multiple process() calls, you need to use ParamHandles. This also displays a neat handle icon next to each mapped parameter. Users are able to remove these handles through the parameter context menu if they want, so you should definitely use them to offer this functionality.

[jcelerier]

Thanks, that APP thing is what I was looking for. Should be all good then !