Steam runtime does not support OpenVR manifest translation

[SpookySkeletons]

With the OpenXR support well under way now is a good time to also bring this up as promised,

The OpenVR manifest may for instance point to SteamVR installed in the user's Steam common or it may point to an OpenVR translation layer installed on the system root as an alternative provider of the OpenVR API and it may be necessary to traverse symlinks that account for proper 64 and 32 bit architecture installed to the system and reform it into the expected OpenVR directory structure elsewhere which may include a system packaged OpenVR manifest.

Without this manifest translation OpenVR libs must be installed directly to /opt or somewhere in the user's home folder to bypass pressure vessel. This is largely preventing distribution packaging of these OpenVR translation layers.

An implementation should account for following symlinks to the libraries themselves and support a root installed OpenVR manifest which is symlinked in the user's home directory for easy enablement of these translation layers.

Quick example of a bare bones packaged OpenVR manifest https://gitlab.com/znixian/OpenOVR/-/blob/openxr/openvrpaths.vrpath.in?ref_type=heads

[SpookySkeletons]

@smcv Thank you for all your hard work it's making thing substantially better for other XR runtimes. Could I direct your attention to this last issue of ours, with the completion of both runtime and XR layer support into the steam sandbox, we are unable to package our own OpenVR runtime implementation as mandated by the Fedora packaging guidelines.

Right now, OpenComposite, and eventually XRizer will both be available in the Fedora repos and without this path traversal we can't construct the proper OpenVR runtime structure, so effectively our distribution is useless in Fedora.

Currently only Arch Linux with looser AUR packaging guidelines allows us to insert the runtime translation layer into /opt to work around the pressure vessel limits here.

[smcv]

The OpenVR manifest may for instance point to SteamVR installed in the user's Steam common or it may point to an OpenVR translation layer installed on the system root as an alternative provider of the OpenVR API and it may be necessary to traverse symlinks that account for proper 64 and 32 bit architecture installed to the system and reform it into the expected OpenVR directory structure elsewhere which may include a system packaged OpenVR manifest.

Please assume that I don't know anything about OpenVR, SteamVR, or your third-party runtime. How is your runtime enabled? What files have to be edited, or what environment variables have to be set, to make it be used?

At the moment I have no context for what you are asking for, and for whether it's something that Valve/Steam intends to support.

[xytovl]

It's a bit harder to document compared to OpenXR, as I don't know if there is a specification for OpenVR.

The source of truth is in https://github.com/ValveSoftware/openvr/blob/master/src/vrcore/vrpathregistry_public.cpp
An OpenVR runtime is specificed by a single directory, which an amd64 will look like this:

.
└── bin
    ├── linux64
    │   └── vrclient.so
    ├── version.txt
    └── vrclient.so

https://github.com/ValveSoftware/openvr/blob/master/src/vrcore/pathtools_public.h contains the subdirectories names.

The OpenVR loader will first search for the runtime using the VR_OVERRIDE environment variable which must point to the directory. Note that this variable may be ignored by some games (VRChat for instance).

If the override variable is unset, VR_CONFIG_PATH file is read instead, with fallback to ~/.config/openvr/openvrpaths.vrpath (I am not sure if XDG vars are used).

This configuration file is a json file:

{
	"config" : 
	[
		"/home/xytovl/.local/share/Steam/config"
	],
	"external_drivers" : null,
	"jsonid" : "vrpathreg",
	"log" : 
	[
		"/home/xytovl/.local/share/Steam/logs"
	],
	"runtime" : 
	[
		"/opt/xrizer"
	],
	"version" : 1
}

The only relevant entry is the first element of the "runtime" array, which is the path to the runtime. Other elements in the array are ignored by the loader.
I do not know the use for other entries in the manifest, nor where they are documented.

[damaestro]

From a distro packaging perspective, Fedora has strict guidance on filesystem layout.

• Don't install into $HOME (🥇 duh), so we don't do that
• Strict allocation of /opt path with only one granted exception, so we don't do that
• Install DSOs according to FHS, which is %{_libdir} (/usr/lib64 or /usr/lib)
• Strict guidance on loader visibility, ABI compatibility for a given version, and properly versioned SONAMEs (opencomposite is unversioned SONAME vrclient.so)
• Unversioned DSO requirements have strict guidance, with exceptions for Vulkan and OpenXR. If we need to use the linker pivot, I will need to get additional exceptions approved.

The result of these policies is that the opencomposite DSO is located at /usr/lib64/opencomposite/runtime/bin/linux64/vrclient.so and the runtime path is /usr/lib64/opencomposite/runtime. The DSO is not visible to the linker.

This results in the host vrclient.so not being visible to the pressure vessel.

[SpookySkeletons]

@smcv I hope all three of us can supplement understanding in the absence of an OpenVR spec.
Current flow for WiVRn operation,
The package manager automatically installs the correct env vars into the user's root env so all applications inherit PRESSURE_VESSEL_IMPORT_OPENXR_1_RUNTIMES without user intervention as part of WiVRn, where this is not possible like flatpak we pop up a copytext box informing the user to insert this as a variable on the game itself so they may proceed.

Due to an intersection of issues, Proton must currently load a test OpenVR client and then a test OpenXR client to check the Vulkan extensions available before clients load, there is not a good way to switch this over to pure OpenXR in the case of OpenXR applications from what I understand.

So this demands we have an OpenVR translation layer present or a user will not be able to run any steam games. Providing distribution packaging of these OpenVR->OpenXR layers so we get an OpenVR API for Proton presents the above problems from damaestro and Xytovl, there's currently no way to suitably provide our OpenVR translation layers in distribution packaging because we cannot have pressure vessel traverse and reconstruct an OpenVR 'runtime' path json key through symlinks so the libraries would be placed appropriately on the system root without breaking modern distribution guidelines.

We bypass this, only on arch linux user repo, by dumping the OpenComposite/ Xrizer translation layer structure into /opt where pressure vessel is allowed to go but we can't do this for other distributions and getting exceptions for these libraries might not be possible.

/opt/xrizer/
├── bin
│   ├── linux64
│   │   └── vrclient.so
│   ├── version.txt
│   └── vrclient.so
└── openvrpaths.vrpath

And ideally with that path construction the runtime key could be set to something specific and pressure vessel can automatically construct the necessary OpenVR runtime paths from the host installed vrclient.so libs for the container to access. And as damaestro says current arrangement makes it impossible for distro maintainers to ship this structure, so the only reasonable place where we see this may be solved is at the pressure vessel level.

[SpookySkeletons]

We had a small breakthrough thanks to GloriousEggroll we hope can hit main proton when it's ready

ValveSoftware/Proton#8256 (comment)

OpenXR games now no longer hard depend on OpenVR with this being present, but we could still use something of this sort for the games using the OpenVR API so they can utilize the host installed OpenVR translation library.

[smcv]

Taking a few steps back from the details of how this happens to be implemented right now: what is the high-level goal that you are aiming to achieve, how are you trying to achieve it, and what components are you modifying or replacing in order to do that?

I will definitely not be able to justify working on this to the people who choose my priorities if I can't describe what you want and why. You're mentioning the names of components (or possibly groups or people, I'm not sure) as though I will definitely know what you mean, but I'm not clear on what (or who?) OpenOVR, znixian, opencomposite, WiVRn or Xrizer are, how they relate to each other, and which of them are alternatives to / replacements for a component that Valve supports; and even if I did, other readers wouldn't have that context.

Perhaps a description of your use-case would be something along these lines:

• I have a VR game, such as (what?), which uses x, y and z libraries, and is a (native Linux game? Windows game running under Proton?)
• Normally if I install that VR game onto a Linux system via Steam in the most obvious or most-supported way, it will use (some version of Proton?) and (Steam's SteamVR modules?) to communicate with (the SteamVR compositor?)
• I want to replace (component) with (component) because (reasons).
• If I install (component) below $HOME or /opt, I can already achieve this by:
  • (steps)
• Now I want to package (component) in distros. To do that I have to install (component) below /usr.
• But when I tried to do that, it didn't work because (what file? in what location? under whose control?) describes the path to (component) on the host system; but inside a Steam Linux Runtime container, which has a different /usr, paths below the host's /usr don't mean the same thing.

At the moment I'm seeing a lot of descriptions of those last couple of bullet points, but that doesn't give me the context I need to be able to either understand what is wanted, or justify it to my manager.

The closer your use-case is to something that Valve actively supports, the more likely it is that I will be allowed to spend time on it, so it would be best if you can describe use-cases that replace a minimum number of Valve-supported components. For example, a use-case involving the SteamVR compositor will be easier to justify than a use-case involving a third-party compositor, or a use-case involving official Proton will be easier to justify than one involving a third-party/custom Proton build like Proton-GE (and if something has to be prototyped in a third-party/custom Proton build first, it will be best if there is a plausible route to it being integrated in official Proton, even if it's something that official Proton can't do right now).

[xytovl]

For the use case, on one end we have games, and on the other one VR hardware. The high level goal is to be able to run VR games using a runtime provided by the Linux distribution.

Games can be win32 or Linux native, and use either OpenVR or OpenXR as their API.

Hardware is used through a runtime like SteamVR, Monado or WiVRn.
Of those runtimes, SteamVR supports both OpenVR and OpenXR APIs, while Monado and WiVRn only provide OpenXR.

Compatibility libraries exist to provide an OpenVR API on top of OpenXR: OpenComposite (formerly OpenOVR), xrizer and VapoR.

I am not aware of any native OpenVR runtime other than SteamVR, but we can consider compatibility libraries as runtimes on their own, as OpenXR support is already working through pressure vessel.

Let's see one problematic use case: a Win32 OpenVR game on a system with Monado + xrizer.
xrizer will provide an OpenVR runtime as a set of files, under Fedora packaging rules, this will be in a directory like /usr/lib64/xrizer/runtime. This path will be read by the OpenVR loader inside of pressure vessel, and will fail as it is not visible.

Due to ValveSoftware/Proton#8256, Win32 OpenXR games will also be affected by the same issue.

Native Linux OpenVR games such as Half Life: Alyx hit the issue too.

Affected scenarios are: all VR games (except Linux native OpenXR) with OpenVR runtime packaged under /usr.

[SpookySkeletons]

One scenario which may help afford this, and by extension, our usecase:

An OpenVR runtime currently cannot be packaged and managed by the host update cycle, say for instance, SteamOS.
The SteamVR runtime, if configured & installed in the root of the system partition, so SteamOS may manage it in the same way mesa versioning of the host is managed, would be totally inaccessible by current Steam client games.

I'm hoping we would both benefit by having the OpenVR runtime managed in much the same way as the OpenXR runtime currently was just configured, so host libraries and package/ atomic update system management of the runtime, in full, applies.

[smcv]

Compatibility libraries exist to provide an OpenVR API on top of OpenXR: OpenComposite (formerly OpenOVR), xrizer and VapoR.

So am I getting this right?

Steam would normally install SteamVR into the Steam library (steamapps/common) and use it to provide the OpenVR and OpenXR runtimes for any VR game, and the compositor; so each OpenVR game (e.g. Half-Life: Alyx) loads SteamVR's OpenVR runtime, and each OpenXR game (do we have an example?) loads SteamVR's OpenXR runtime, both talking to SteamVR's compositor, which manages the actual VR hardware.

Instead of using SteamVR for that purpose, you want to use one of these compatibility libraries that implements the OpenVR API as a wrapper around OpenXR? So that each OpenVR game like Half-Life: Alyx uses (OpenComposite|xrizer|VapoR) as its OpenVR runtime, acting as a wrapper around the (Monado|WiVRn) OpenXR runtime, to talk to the (Monado|WiVRn) compositor, which manages the hardware? And meanwhile OpenXR games use the (Monado|WiVRn) OpenXR runtime directly, and, again, talk to the (Monado|WiVRn) compositor?

And, the fact that OpenVR manifest things are not currently handled exacerbates a Proton limitation, ValveSoftware/Proton#8256, which means that each Windows VR game running under Proton needs working OpenVR and OpenXR, both talking to the same compositor (SteamVR|Modado|WiVRn), even if the game itself would only need OpenVR or OpenXR?

[xytovl]

That is all correct.

For an example of OpenXR game, there is Beat Saber. I do not know of native Linux OpenXR games distributed through Steam, but there may be some.