Define a registry for defining how UAs expose input devices

[ddorwin]

While the WebXR spec (after immersive-web/webxr#553 is merged) provides some guidelines around exposing controllers as Gamepads, there is still room for interpretation that could lead to different behavior among implementations. In addition, future controller/input mechanisms (i.e., hands as exposed by a specific technology) may not map well to "xr-standard" or otherwise have obvious mappings.

To encourage consistency and interoperability, details about how specific controllers should be exposed by user agent implementations should be defined in a registry. The registry would be separate from the spec, allowing out-of-band updates (per an established process). It would both clearly define how implementations should expose specific controllers and thus how applications and libraries (e.g., https://github.com/immersive-web/xr-gamepad-mappings) can expect them to be exposed.

The registry would map some well understood definition of a physical controller to:

• id string
• GamepadMappingType
• The physical button that represents the primary action
• A clear definition of how the physical buttons are exposed via the Gamepad interface
  • This would include which buttons are not to be exposed, such as those reserved by the system and/or user agents.

Ideally, the entry would also include some concrete information, such as USB vendor and product IDs and/or identifier exposed by native runtime(s).

A single entry could cover multiple devices (i.e., revisions of a controller) as long as the physical appearance (due to the intent for applications to use the id to choose a controller model) and button mapping are the same. If there are multiple versions of hardware with the same product name, they may be identified by version, year, or similar.

Such a registry would also solve a couple open issues:

• Format of Gamepad identifiers (#550): While the registry could have guidelines, there wouldn’t be a need to define a perfect rule covering all hardware because the registry would specify the identifier.
• As such, identically-/similarly-named controllers (e.g., “Oculus Touch” per Format of Gamepad identifiers webxr#550 (comment)) can be clearly and easily differentiated.

One additional possibility is that [XR-specific] GamepadMappingTypes also be specified in a registry. These would be added much more conservatively, but it might be preferable to be able to define new ones without going through the full spec process. This would also allow slimming the core WebXR spec a bit.

For reference, there is an open issue for the W3C Process Document (w3c/w3process#168) to define a process for handling registries. This shouldn’t block us from proceeding, and other specs have successfully used registries in the past.

[jacobcdewitt]

A concrete example of some ambiguity in the mapping is the Oculus Touch controller.

Buttons 1-4 are determined by the xr-standard mapping in the spec. For Oculus Touch controllers, I don't think there's much ambiguity for those slots because the spec says the button in slot 3 should be the button/trigger associated with the grip (as opposed to something else like the index finger). If there were 2 or more joysticks/touchpads, then things would get a bit more confusing because it might be ambiguous which joystick is primary vs secondary.

For slots 5 and later, the order depends on a subjective determination of button importance. When implementing WebXR Gamepad integration for these controllers, I determined the order to be the following, but other developers could easily come up with a different ordering (and justification) for these buttons:

• right hand: A > B > thumbrest touch sensor
• left hand: X > Y > thumbrest touch sensor

[cwilso]

We need to keep working on this in parallel, but this isn't a requirement to hit VR-Complete.

[ddorwin]

We still need something before implementations ship, though, right? Otherwise, there could be interop problems in the initial implementations. Is my understanding that VR-Complete is intended to be the milestone on which implementations could ship enabled by default correct?

[NellWaliczek]

It is proposed to use this repo as the registry for the information described above. The assets, code library, tests, and tools also in this repository are not required to use the profile.json files for each motion controller.

Unless there are objections to this approach within the next week, I'm inclined to close this issue while we continue to iterate directly on this repository.

[ddorwin]

Note that there are only five people watching the webxr-input-profiles repository, so most of the community will not have seen the notice above. I suggest moving this back to the webxr repository for discussion and because it is really a question of what the core spec will refer to.

As for the proposal, it seems odd to use code (or a repo primarily containing code), especially code intended to be used by applications, to define user agent behavior. Also, specifications probably shouldn't be tied to or give preference to a single library.

There are also more practical concerns, such as that different levels of scrutiny should be applied to library code vs. what is effectively a specification and that the reviewers are likely to be different. Having both in a single repo makes it hard to ensure the right things are happening without overwhelming the appropriate people.

It would be helpful to understand which parts of this repo constitute the "registry" for user agents. Is it just profiles/? Perhaps that could be in a separate repository and programmatically pulled in by this library or any other that wishes to use it.

[NellWaliczek]

Interestingly, over the weekend I'd started to ponder a separate release cadence and versioning of the schema and profiles/assets. I'm not sure a list of strings warrants a separate repo, but it's definitely an option to consider.

For what it's worth, the original intention of this repo wasn't primarily to provide a code library. The profiles are the relevant bit; the provided library is simply a convenience on how to consume them. And really roughly half of the code in the repository is tests to validate the schema is authored/modified correctly and to validate the supplied profiles are conformant with the schema.

I do agree that it's easy to misunderstand this based on how the README.md is set up. And I'd already been thinking of separating the "how to use this library" and the "understanding the schema" into two separate readmes.

So perhaps the solution is as simple changing the main readme to have a small amount of text explaining the purpose of the registry, links to relevant spec text, and the formatted list of supported profile IDs. Underneath all that, we would put links to the other two readme's I mentioned above. And then the repository, would publish the library and profiles/assets separately.

Or were you looking to publish the profiles lists for consumption inside the actual browser source code?

In any event, I will be leaving this issue where it is for now. The move should not have been a surprise since doing so was discussed on a call as well as sent out in an email the public mailing list. I'd be happy to send a follow up email to the group reminding them that it moved if that would be useful.

[NellWaliczek]

Threw a quick sketch of an idea to get the conversation going #76

[ddorwin]

I'm not sure a list of strings warrants a separate repo, but it's definitely an option to consider.

The registry is much more than a list of strings. Please see the original description of this issue.

Among other things, the registry would:

1. Map a physical device to a string.
   • The key names in customProfiles in Potential registry format #76 don't really address this.
2. Provide an unambiguous specification of how user agents implementations should expose all the controls.
   • See Define a registry for defining how UAs expose input devices #71 (comment).
   • Using that comment as an example, the registry might say slot 5 is the A button, slot 6 is the B button, and slot 7 is the thumbrest touch sensor.

[ddorwin]

As we work on the definitions, it's worth considering the impact of the introduction of a profiles list (replacing the single id). Specifically, the user agents expose exactly one set of mapping, axes, and buttons per physical device, so each of the values in profiles must be a subset of the previous value. In other words, a controller with four buttons cannot pretend to be a three-button controller if the fourth button on the former is equivalent to the third button on the latter. (In practice, implementers and the registry definitions can try to avoid this as much as possible in the way they define the mapping, though GamepadMappingType definitions may not allow such flexibility.)

[NellWaliczek]

This should hopefully mostly be addressed by #78 now.

I'm inclined to treat the "subset" question separately, because it's not entirely clear that should be strictly a requirement. Oculus folks have expressed that they'd like Oculus Quest to fall back to Oculus Touch even though the latter has a thumbrest the former does not. I'll file that as it's own issue.

If you disagree with either of the above, please feel free to reopen.