update plugins to work as AudioPluginService.V2 plugins

[atsushieno]

On main branch, aapinstrumentsample is implemented to work as MIDI 1 synth. Other plugins are either not updated or cause runtime crashes. They have to be either fixed or rewritten to work as V2 plugin.

Though, it kinda requires updating to the new scheme for parameter changes i.e. interpret MIDI2 inputs in realtime manner. That brings in unified event processing model with MIDI1 based inputs, and not everything is ready for that yet.

There are couple of sub-tasks:

• implement MIDI2 processing part in PluginPreview (and thus samples-host-engine and aaphostsample) to work with the new scheme if (1) there is a MIDI2 in port and (2) there are "parameters" (it was only partially implemented).
• make changes to aapinstrumentsample to also take parameter changes via UMPs.
• Once PluginPreview gets fixed and starts working fine, then it should also support (send) unified MIDI2 inputs to the instantiated plugin.
• midi-device-service should work regardless of whether the port is midi1 or midi2 (needs to insert UMP translator).
• Once it gets working, then aap-lv2 needs to be rewritten to migrate to the new parameter changes scheme.
  • get aap-ayumi working for both midi1 and midi2 modes (with conditional build)
  • update aap-import-lv2-metadata to support midi2 and parameters
  • get midi-device-service working
  • formalize port mappings (which became inconsistent in the new design and causing run-time crash)
  • get aap-lv2-mda effects working
  • get aap-lv2-mda instruments working
  • get aap-lv2-fluidsynth working
  • get aap-lv2-sfizz working as MidiDeviceService
  • get aap-lv2-string-machine MIDI output correct synthesis results
  • get LV2 patch input working
  • implement Atom output reader (and push to AAP midi2 output port)
• The same has to go in aap-juce.
  • get cmake support working
  • get Projucer support working
  • get plugins working
  • get hosting working

[atsushieno]

It is a more fine-grained task list compared to #80.

[atsushieno]

Here is a draft documentation for: AAP V2 Protocol changes

Rationale

AAP is going to support decent parameter changes with unified MIDI2 input events. It is a paradigm shift from traditional LV2 port to LV2 Atom or CLAP unified events.

There are handful of reasons to make this change, but namely:

• The existing parameter changes were supported via ports, and a port can come up with unfixed length of buffers. It is not quite effective. It was okay when parameters were like 10..20, but now plugin parameters can be hundreds.
• To resolve that issue, most of the plugins support parameters via unified parameter change event port. LV2 has Atom. CLAP defines its own as well as MIDI1/MIDI2 events. VST3 does not define event queues, but it defines callable parameter change events by its own.
• The traditional AAP implementation only dealt with MIDI 1.0 compatibility inputs, and it has (kind of) achieved sample-accurate input events based on SMF-like time designation (7-bit encoded length). But audio plugin parameters are usually 32-bit float (even 64-bit double sometimes). They do not fit into traditional MIDI 1.0 7-bit land. It is also quite complicated to "unify" multiple sequences of MIDI1 input events when there is a DAW sequencer and also MIDI keyboard inputs at the same time.

AAP uses MIDI 2.0 UMPs, it reuses the standard technology, compared to other technology like LV2 Atom or CLAP events. Parameter changs are sent as MIDI 2.0 Assignable Controllers.

Breaking Changes

This change leads to the new V2 protocol because it breaks plugin compatibility. AAPs that implement V1 is fundamentally incompatible with V2, and therefore they are not returns for V2 plugin queries (and vice versa).

The details:

• "midi" 1.0 ports will not be supported. There will be only "midi2" ports. They only process UMPs. V1 protocol supported both MIDI1 and MIDI2, and they could be switeched to each other using MIDI-CI Set New Protocol. It will be gone.
• Multiple MIDI2 in ports are not supported. Multiple MIDI2 out ports are not supported either.
• Plugins that implemented MIDI1 protocol have to rewrite MIDI event processing part based on UMPs. UMPs can be translated to MIDI1 stream and @atsushieno offers that functionality as "cmidi2" library, so it should not be a big deal.
• aap-lv2 plugin developers should not have to deal with the changes. aap-lv2 developers (@atsushieno) have to deal with it. That is going to be ABI breakage.

AAP-LV2 implementation

• aap-lv2 plugins only support MIDI1 events via Atom Sequence, so aap-lv2 has to rewrite its process function part to translate UMPs to MIDI1 events (already done).

aap_metadata mappings

• In aap-lv2-metadata-importer LV2 control ports are not directly mapped to AAP <port>s anymore. They are instead mapped to <parameter>s.
• Port mappings between AAP ports and LV2 ports: they used to be simple (just one by one, same index), but now that not all LV2 ports have a corresponding LV2 port and the mapping are not obvious, there should be some explicit mapping information. Though there is no explicit way to indicate the LV2 port index in AAP port (they are not indexed anymore).
  • An audio port is mapped to a <port> element. Control port is mapped to a <parameter> element.
  • Regardless of what .ttl defines, aap-import-lv2-metadata will ALWAYS generate midi2 <port> elements, one for input, another for output.
  • An atom port that accepts either LV2 midi or LV2 patch will receive inputs from midi2 input port, and those ports that send midi or patch Atom messages will redirect the outputs from midi2 output port.
  • If there are more than one LV2 MIDI Atom input ports, they will be assigned different MIDI2 "group" number. Same goes for Atom output ports. Group number is incremented as those ports appear, ordered by port index (e.g. if those MIDI in Atom ports appear at 3 and 7, then group 0 maps to port 3 and group 1 maps to port 7). Those LV2 MIDI ports are supported only up to 15 (due to the group limitation in UMPs).
  • aap-import-lv2-metadata iterates all ports in the order by index, then occurrence order in the .ttl, and generates those XML elements described above. There will be no implicit port that are not declared in AAP-LV2 world. androidaudioplugin-lv2 populates ports instances.

ports and events mappings on instance

• Those LV2 port that accepts MIDI inputs will be assigned an internal buffer.
• Those LV2 port that accepts Patch inputs will be assigned an internal buffer, if different from above.
• For each AAP MIDI2 input,
  • MIDI2 Assignable Controller is a parameter change, so it is either (1) translated to an Lv2 patch event if an atom input port that accepts it exists, or (2) the input to those control ports (already done, but unverified).
  • Other MIDI2 message is translated to an LV2 midi event if an atom input port that accept it exists. Otherwise, ignored.

AAP-JUCE implementation

• aap-juce plugin developers should not have to deal with the changes. aap-juce developers (@atsushieno) have to deal with it.
• aap-juce plugins only support MIDI1 events so far (its AudioProcessor may extend support to UMPs in the future), so it has to translate UMPs to MIDI1 events.
• JUCE parameters are not mapped to AAP ports anymore. One and only MIDI2 in port handles the parameter changes.
• JUCE plugin clients have to translate MIDI1 inputs to UMPs, then pass to the AAP midi2 port.

[atsushieno]

aap-lv2 depends on #118 (need to retrieve ports).

[atsushieno]

A fundamental problem was raised - Assignable Controllers are used for parameter changes, but when we convert MIDI1 bytes to UMPs, they are also used to achieve NRPN (and DTE) replacement, and it is kind of mandatory as the UMP specification states those MIDI1 messages SHOULD be translated to ACs. And mixing MIDI1 channel voice messages and MIDI2 channel voice messages is not MIDI2 compliant.

The alternative method is to use universal sysex8 for those parameter changes. It is actually what atsushieno/augene-ng does for audio plugin parameter changes within UMPs.

[atsushieno]

Some issues I found during migration from ACC to Sysex8:

• Unlike ACC, Sysex8 does not have the channel information. We will have to prepare another field for that.
• There is per-note ACC, which I thought would be good for this per-note era. For example, CLAP clap_event_param_value_t provides key (and noteId) which fulfills this feature. Sysex8 of course does not have the corresponding field, so we will need another field.

[atsushieno]

It seems aap-lv2 had regressed in main branch, aap-ayumi is not generating any output.

[atsushieno]

^ is fixed. aap-fluidsynth works now. But aap-sfizz MidiDevice crashes. aap-string-machine plays well as a plugin, but MidiDevice generates super simple tone, which likely means that it does not set correct parameters.

[atsushieno]

The latest status: aap-juce-obxd and aap-juce-dexed seem to work with aap-juce-plugin-host. aap-juce-hera seems broken, even on aaphostsample.

[atsushieno]

• aap-juce-dexed: generates empty output for aaphostsample. Works for MIDI and AudioPluginHost.
• aap-juce-hera: MIDI still does not work

[atsushieno]

• fix aap-juce effct plugins, they are not working because we only check acceptsMidi() to determine whether we process MIDI inputs, but now we send parameter changes over there too.
• fix buffer assignments for processReplacing(): the existing code does not seem to work with it, most likely due to buffer pointer assignments.

[atsushieno]

We have aap-juce-frequalizer working. aap-juce-simple-reverb still does not reflect parameter changes on AudioPluginHost, maybe neither frequalizer (APH too crashy to connect multiple plugins and test outputs and parameter changes) though. We can investigate after the final 0.7.4 release work.

[atsushieno]

With the latest 0.7.4 release I'm finally closing this issue. Those remaining tasks are logged as ^.