UA applied mapping cannot be escaped

[pyalot]

It's sometimes desirable not to use the UAs mapping (because it might be wrong, hide some controls or a user implemented handling provides better utility).

The current specification cannot escape a UA provided mapping. I'd suggest adding a way to escape mapping.

[luser]

@kg and I talked about this a while ago. I don't remember if we had a concrete suggestion for the API. The simplest thing I could imagine is gamepad.mapping = ""; However, then we have to decide what happens to the Gamepad object that you've just set the mapping on. Does it need to be immediately remapped? If the spec describes Gamepad objects as snapshots (#8) does that make sense?

One quirk here is that there may be platform APIs for which this doesn't make sense, like XInput, where you don't actually get a bag of buttons + axes, the API provides you with a set of named controls. In that situation the standard mapping makes sense but unmapped doesn't really make sense.

[pyalot]

There's two kinds of mappings. One is the mapping defined by the Gamepads API. There's only one mapping currently (gamepad-likes). The problems with that is:

1. UAs use guessing to figure out when to apply the mapping. This can come out wrong, for instance (real world example), I had a steering wheel mapped as a gamepad.
2. If something does get mapped as a gamepad-like, other controls (which aren't conforming to that mapping) are hidden. This is an issue for things that get (correctly) identified as gamepads, but offer additional buttons/axes that aren't fitting in the schema (for instance the PS4s touchpad)

The bottom line is that UAs can make an educated guess. But by no means is their idea always the right one.

The other kind of mapping is when the OS API provides a list of axes and buttons, which might have some kind of mapping applied. In this context (for every API, Xinput, etc.) you always get a bag of controls, but there's additional properties attached (like if it's an axis or a button and a name).

For this kind of mapping much of the same caveats apply. It's a frequent occurence to get confusing names, axes reported as buttons, buttons reported as axes, inconsistent (between OSes) descriptors, etc. Regardless of the attached metadata, what is mostly consistent on these is that if one device is plugged into one OS, the bags will be the same. (I should mention that some OSes like Linux are particularly messy there because they can talk to devices via evdev, xinput, libusb etc. and depending trough which API you go, you get completely different bags.

There's relatively little a browser can do about the underlying API other than when no mapping is possible and/or desired, to return the bags as obtained from the OS.

[luser]

For XInput specifically this is what you get:
https://msdn.microsoft.com/en-us/library/windows/desktop/microsoft.directx_sdk.reference.xinput_gamepad%28v=vs.85%29.aspx

Since XInput only supports xbox 360 (and now xbox one) controllers, the buttons are correct as reported. This does not solve the general case, certainly, since it's limited to a single platform and type of controller, I'm just saying I'm not sure what we'd have a browser do in this case if you ask for an unmapped controller.

[luser]

For the rest of your comment, yes, I agree that we should allow content to opt-out of a browser's built-in mappings and substitute its own, for precisely the reasons you've elaborated.

[pyalot]

There's not much that can be done about messy OS-level APIs/drivers other than providing it in some fashion that's at least consistent for that OS/UA/device.

[luser]

Right, I'm just not entirely sure what the spec should say in that case, is what I mean. We could explicitly state what to do with XInput in that case, or we could attempt to write a more abstract version of "how to take an arbitrary input structure from a driver and translate it into buttons/axes".

[pyalot]

In cases where a translation to a bag of axes/buttons is necessary there's a couple of considerations:

1. how many APIs will the gamepad spec see come and go over its lifetime?
2. do you want to provide a new spec everytime a new flavor of API pops up?
3. can you specify a future proof algorithm that's guaranteed to be suitable for any kind of API flavor?

I believe the answers are:

1. A bunch
2. No
3. No

Assuming this, there are only two options:

1. Do not specify it, every UA does whatever it wants to convert it to a bag
2. Come up with a way to provide a mapping grammar for these cases that's decoupled from the spec, but tracks API flavor.

The only problem with that idea is that: It might end up overspecifying the issue for a single cumbersome API. It's also possible it's the only such API that's ever going to be around.

[kg]

If you consider the possibility of the web 'winning' on other platforms, there are already a handful of semantic game input APIs (most of them on game consoles).

The PS4 and XBox One both are tangibly different from the PS3 and XBox 360 in terms of controllers (though the XB360 -> XB1 gap is relatively small). And the PS3 controller was dramatically different from the PS2. The PS3/PS4 controllers have a ton of gyros and stuff, the PS4 has a touchpanel.

If you're running on a console, the console-specific input API is reporting all this stuff to you.

A world where web games are running on consoles seems somewhat unlikely, but on the other hand Microsoft is already shipping IE on some of their consoles and Sony has shipped a browser on some of theirs. Nintendo previously had wiimote input support in their embedded browser on the Wii, and the Wii U has a HTML5 SDK. So it's worth considering how we want gamepads to work in a world where XInput isn't the only semantic API and there are in fact a variety of control schemes out there in the wild.

[pyalot]

This is a problem of what the specification does. As is amply evident, a specification that follows after implementations leaves the web in a position where multiple UAs implement the same functionality differently, reconciliation takes considerable time, or never happens at all.

On the other hand, input API flavors primarily pop up as a need whenever one UA wants to bind against a new API, and so they are unlikely to provoke a specification change everytime a UA does that.

Which leaves the attempt to specify it in a limbo of not having a crystal ball about everything every UA might technically do in the future, and every permutation of every possible device and assemblage of input sensors that might be invented that might be serviced by any kind of API anybody might invent. Yet without that ability, the specification is condemned to be an eternally broken mess that nobody implements...

[kg]

That's more or less why I think the correct move is to hoist all the mapping responsibility out into userspace and encourage the use of a community-maintained library, instead of having it done by each individual UA. But I'm not sure it's feasible to do that at this point, since multiple UAs already shipped rigid, partial mapping support.

[pyalot]

The way the gamepads API is specified prevents solving this issue in a sensible manner. Primarily it's impossible to come up with any way to future-proof define a way to map a bag of names, to a bag of indices in an unambiguous manner without hardcoding it into the specification, which we're loathe to do for obvious reasons.

It's also difficult to provide a bag of names with the semantic that the gamepad API has chosen (object iteration is slow, and mapping names trough lookups is undesirable).

It would be far easier if devices, axes and buttons where objects that could be retained/remembered after initially figuring out if we're interested in them.

[pyalot]

At a bare minimum axes should be objects so there's at least an option to stick names onto them.

[pyalot]

The idea to express a mapping as indices into the axis/button arrays has always bothered me. It would be far easier if axes/buttons where objects that had names for their OS API assigned mapping if available, if not the name would be the index they're delivered from the OS, and they'd have a second name that's mapped by the UA, something like axis.mappedName.

[pyalot]

That would also take care of providing a way to opt out of the mapping. You could just ignore the control.mappedName.

[cvan]

How different is this issue from the issues described in #7 and #6?

[kg]

@cvan this is a general issue regarding information being discarded when a UA applies controller mappings, and also the possibility that the UA-provided mappings are incorrect.

#6 appears to cover a specific instance of this (dpad and trigger mappings), while #7 covers related issues (user-set mappings, controller specific mapping data, etc)

[cvan]

Got it - thanks, @kg - makes sense.

[marcoscaceres]

Action is to add a note to the accessibility section to allow browsers to allow remapping.