Implement generalized XR action map system for use outside OpenXR

[dsnopek]

Describe the project you are working on

The WebXR module in Godot, as well as numerous VR prototypes and small experiences

Describe the problem or limitation you are having in your project

OpenXR uses the "OpenXR action map system" for handling input, whereas WebXR (and any other XR SDKs added via GDExtension) will use their own way of handling input. (And none of them rely on Godot's normal "Input Map" system, since input usually comes through different APIs when using XR.)

This means developers need to check which XR platform is being used, and change how they are handling input accordingly. In fact, currently, this is the main thing developers need to account for when making an XR app that works on both OpenXR and WebXR.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Add a new "XR action map system", which provides a unified interface for setting up action maps, but that allows the XRInterface to provide it's own backend implementation. This way, OpenXR can continue using OpenXR action maps, but other XR platforms that have different systems can emulate it.

In fact, most (including WebXR) would be able to use a "fallback" system that works similarly to Godot's normal "Input Map", allowing developers to map from the actions that come from the XR SDK to developer defined actions.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Proposed implementation:

• Make new XRAction, XRActionMap, XRInteractionProfile, XRIPBinding, and XRActionSet resource classes that have the same public methods as the OpenXR* versions
• Change the OpenXR* versions of the above classes to extend the general XR* ones for backward compatibility (although, maybe we could convert these when loading a Godot 4.0 project instead?)
• Replace the OpenXRActionEditor with an XRActionEditor class that edits the new XR* resources instead of their original OpenXR* versions
• Add a new XRActionMapBackend class which has methods for get/set_action_map(), get_action_sets(), is_action_set_active() and set_action_set_active().
• Add an OpenXRActionMapBackend which provides the OpenXR implementation
• Add a FallbackXRActionMapBackend which provides a developer configurable fallback implementation similar to Godot's normal "Input Map".
• Add a virtual get_action_map_backend() method to XRInterface, which returns a new FallbackXRActionMapBackend by default, and the OpenXR version on OpenXRInterface
• Replace the existing action map related methods on OpenXRInterface with ones that just call into the OpenXRActionMapBackend for backwards compatibility

If this enhancement will not be used often, can it be worked around with a few lines of script?

No. In order for this to be generalized across all current and future XR SDKs this needs to be full system, including extensive editor UIs.

Is there a reason why this should be core and not an add-on in the asset library?

Since both OpenXR and WebXR are part of the Godot engine, and OpenXR already has its OpenXR action maps system as part of the core engine, this makes the most sense as a part of the engine as well.

[BastiaanOlij]

Small addition to the above, this now also applies to TiltFive for which support is implemented as an GDExtension. It also features hardcoded actions same as with webXR and would benefit from a generic Action map system in Godot to make portability between platforms easier.

One important element worth discussing here in the long term is that if the Action map becomes a more fully features input map system and the question will undoubtly be raised why we have two seemingly competing systems. This is a worthy question worth further discussion, ideally there should only be one system. There are two key points here that convinced us to implement the OpenXR action map system separately originally:

• The action map system in OpenXR is static. You configure it while designing your game, once your game runs it immutable. Godots input map system is fully dynamic and rebindings can be done on the fly
• The action map system offers far more variations on the inputs (digital buttons, analogue buttons, joysticks, spatial poses, etc) making the action map system less easy to use than the input map system. A lot of the features required in XR were specifically not added to the input map system to keep things easy to use.
• Actions can't be mixed, as OpenXR performs the mapping, we can't create an action "Jump" and map it to both XR controller inputs and normal controller inputs. It is one or the other, mixing them in one system causes boundary issues, not unsolvable but very nasty...