Don't let plugins make the main window transparent

[DomClark]

Some plugins, e.g. Blue Cat's VSTs, have an opacity control which, in LMMS, makes the main window transparent. This PR blocks this behaviour under Windows by preventing the WS_EX_LAYERED extended window style from being set.

[tresf]

@DomClark thanks. This looks pretty safe to merge. Does this prevent all transparencies or just those invoked from remote plugins?

[DomClark]

It prevents all transparencies achieved with the Windows API which require setting an extended window style, for all windows whose messages are processed inside QCoreApplication::exec(). This includes all remote-plugin-invoked transparencies and excludes all Qt ones, so should be safe to include. Windows doesn't, however, provide a way to find out which process or window is making the modifications, so this could well also prevent third-party applications designed to make windows transparent from doing so to LMMS. I'm not sure if you would consider that a problem or not.

Also, I've made a slight mistake where windows that already have the WS_EX_LAYERED style will lose it if their extended style is modified, I'll fix that up now.

[tresf]

I'm not sure if you would consider that a problem or not.

Not at all. 👍

[Umcaruje]

Ok so I tested this on Linux Qt4, and turning the transparency knob has the same behaviour with or without this PR:

The window basically becomes uninitialized and copies whatever is underneath it, and can't be reset without removing and adding the effect again. So this PR fixes the problem on windows, but not so much on linux.

[zonkmachine]

Ok so I tested this on Linux Qt4...

This fix is #ifdef LMMS_BUILD_WIN32
@DomClark Is it possible to fix this for Linux/Wine too?

Merge?

[PhysSong]

FYI #2826 seems to resolve this issue on linux, both on Qt4 and Qt5(if #3786 is merged).

Edit: It just ignores opacity settings in X11embed method, which is used in Qt4 and going to be default in Qt5. QWindow approach always work correctly regardless of #2826. Couldn't test with QWidget::createWindowContainer() method.

[zonkmachine]

FYI #2826 seems to resolve this issue on linux, both on Qt4 and Qt5(if #3786 is merged).

OK. So Linux is fixed elsewhere. Merge this?

[DomClark]

Turns out this only works for Qt4. In Qt5 native event filtering is achieved via QCoreApplication::installNativeEventFilter. I'll implement this once I've got a Qt5 build environment working, unless anyone else wants to do so first.