Fix command line rendering with VSTs

[DomClark]

Fixes #3449.

Change summary:

• Move some UI code out of non-UI classes vestigeInstrument and VstEffectControls into manageVestigeInstrumentView and manageVSTEffectView. There's still some more to go but this much fixes the problem.
• Stop trying to show the VST loading message if we're rendering from the command line.
• Added a new (only used from code) VST embedding mode "headless" that starts the VST without creating the UI.

Window class registration in RemoteVstPlugin I've moved to main() - the class is also used for the message window that receives timer events and messages to be processed on the main thread, so it must be created regardless of whether a UI exists or not.

[PhysSong]

Just FYI - instead of isRenderOnly, you may use if (gui). Here gui is a macro equivalent to GuiApplication::instance() which is NULL when render from command line.

[zonkmachine]

Conflicting files
plugins/vst_base/RemoteVstPlugin.cpp

@DomClark Fix conflicts and then merge?

[DomClark]

Fix conflicts

Done, although I've never done this before so let's hope I haven't messed anything up...

and then merge?

I'm happy, but has anyone else tested this? I'm often worried I'm not rigorous enough with these things, and don't currently have the time to test further.

[tresf]

has anyone else tested this? I'm often worried I'm not rigorous enough with these things, and don't currently have the time to test further.

Agreed. I'll put a request out on Discord.

[PhysSong]

Per request on Discord #support, here's the Windows 64-bit installer:

https://transfer.sh/BG7VH/lmms-1.2.0-rc5.17-win64.exe (link expired)
https://drive.google.com/open?id=1VIYRKVUq1rNY-i4xUgnC7VqZ--3f9-kc

[karmux]

Command-line rendering with VSTs works for me nicely with this PR.

[tresf]

Here's some baseline VST testing kindly provided from JD on Discord. I asked the community to test out the build to make sure it did not introduce any regressions.

tres [, I'm] testing now.
Avast really doesn't like rc5.17, I've actually had to disable it for a tad.
...
Serum ran better than how it has for me in my use of 1.1.3, the visuals not only worked, but didn't lag down the CPU all that much and was able to load my most complicated preset in my library. Saved and loaded the project file for that one without any issues.

Massive did exactly the same, with the GUI responding visually (unlike in 1.1.3) where I tweaked in window. Saved and loaded project files without issues.

Sylenth1 behaves exactly as it does in 1.1.3, you can't load presets in any way, and some parts of the GUI; however those parts are not vital to making the individual instruments. Saved and loaded project files without issues.

Kontakt 5 (using Acoustic Sessions 2 which, as aforementioned is my most troublesome Kontakt instrument) actually is the one that surprised me. 1.1.3, I can load the VST no problem, its reloading closed project files that would make project files unusable. In the test of rc5.17, not only did Kontakt work better overall while in an active project, but I could save, close, then reopen the project file without the file becoming unstable or not opening. Granted, it took a while to reload (48secs) but it did reload with the preset I had loaded into it. It still was a bit of a CPU hog though.
(in short, Kontakt 5 saved and loaded project files without major issues aside from time taken)

edit: Serum running in 1.1.3 has a major CPU issue when on the oscilator page, the visuals cause a massive CPU hike when notes are played regardless of complexity of the instrument or if there are any animations. rc5.17 as was sent to me does not have this issue, but still has a CPU spike of a lesser degree.(edited)
Deseiver that's what I was writing.
...
tresf I have some more notes:

I loaded an old project file which I have not been able to work on due to 1.1.3 deciding it was a corrupted file. It loaded successfully.

The instrument causing the file not to open was Serum, which now works flawlessly without me having to move it away from the Oscillator page.

It also loaded an effect VST not previously tested in my first test-Replika. Loads from the project file without any issues, loads and saves presets without any issues. Loaded into an effect chain directly, VeSTige was not used to load this plugin.
...
tresf [, the] last note before I'm done with making more notes about this version, I tested another 1.1.3 file which was previously abandoned due to load issues on account of Kontakt.

Loaded 3 separate VeSTige tracks each with Kontakt 5 (The Gentlemen as the Kontakt this time for all three VeSTige tracks). Loaded flawlessly and ran with minimal CPU load compared to my use in 1.1.3. Massive was also in the project file and behaved as expected.

[DomClark]

I'd say this is good to merge then.