Change mangohud detection and fix mangohud loading

[Jan200101]

Closes #91
Closes #165

In order of commits:

• Re-adds setting LD_LIBARY_PATH to all upstream supported locations
• replaces dlopen method of mangohud detection with a $PATH lookup for the wrapper

getExecutable should probably be offloaded to some sort of utilities class, but I hadn't found a proper location for it.

[getchoo]

can solve #91 and #165

[Scrumplex]

can solve #91 and #165

I think the latter issue also has a very interesting alternative to just hardcoding usr/local/$LIB/mangohud, as the path defined in the Vulkan layer manifest of MangoHud will probably be the more right one

[getchoo]

yeah that would probably be best since that file is generated at build time and /usr isn't a guaranteed prefix

[Jan200101]

Well, mangohud installs the vulkan layers into $PREFIX/share/vulkan/implicit_layer.d.
How can we detect the system prefix?

Here are my suggestions:

• detect mangohud wrapper and check relatively to it (e.g. ../$LIB/mangohud/
• extract LD_LIBRARY_PATH from wrapper
• use mangohud wrapper

[Scrumplex]

How can we detect the system prefix?

As described in this comment we can rely on the layer being present in XDG_DATA_DIRS. You probably wouldn't be able to use the MangoHud Vulkan layer if this wasn't the case, so it's as good as it gets. You could use QStandardPaths::locateAll with type QStandardPaths::GenericDataLocation to locate the file we are looking for (implicit_layer.d/MangoHud.json).
For systems where XDG_DATA_DIRS is not used, we can add some workarounds (like adding the predictable path on Flatpak to XDG_DATA_DIRS in our wrapper script)

[Jan200101]

How can we detect the system prefix?

As described in this comment we can rely on the layer being present in XDG_DATA_DIRS.

Alright, so it appears that the spec gives us a set of predefined paths to look for, but it doesn't appear to be using XDG_DATA_DIRS.

Is using XDG_DATA_DIRS as an alternative to $HOME/.local/share supported by implementations of this?

You could use QStandardPaths::locateAll with type QStandardPaths::GenericDataLocation to locate the file we are looking for (implicit_layer.d/MangoHud.json).

This should work fine, but I worry that this wouldn't work in flatpak.

[Scrumplex]

Qt should respect XDG_DATA_DIRS for QStandardPaths::GenericDataLocation. We could just set that env var in our Flatpak if Flatpak doesn't do it by itself.

[Jan200101]

Is using XDG_DATA_DIRS as an alternative to $HOME/.local/share supported by implementations of this?

vulkan-loader uses XDG_DATA_DIRS for it and not $HOME/.local/share/, so the approach would work

We could just set that env var in our Flatpak if Flatpak doesn't do it by itself.

We could but I worry that this will have sideffects on Prismlauncher itself or other wrappers that may be used with it.

Worst case we can add our own env var along the lines of VULKAN_PREFIX and then resolve $VULKAN_PREFIX/share/vulkan/explicit_layer.d

EDIT: $VK_LAYER_PATH exists for this case.
while QStandardPaths::locateAll is neat, it will not work for this use case, because it cannot cover all paths supported by vulkan-loader.

[Jan200101]

Its now detecting the library path by looking for and reading out vulkan layer files.

[Scrumplex]

Wrong button. Muscle memory be like

[flowln]

lgtm, thanks!

[Scrumplex]

LGTM. Thanks!