Consider expanding Android gamepad support

[divVerent]

Android needs gamepad support most as touch input isn't so great. Right now only gamepads supported by SDL_GameControllerDB are supported - however Android APIs already provide a standard layout mapping natively.

I filed hajimehoshi/ebiten#2557 for this, but it seems resources are lacking to implement this on Ebitengine side - thus, filing this bug so I'll look into it myself to see what it takes to make this work.

Looking for androidKeyToSDL it seems like we already have a 1:1 mapping from Android buttons to SDL names, which seems close to standard layout. We'd need the same for axes. As part of the SDL ID, we already provide info about axis and button existence, as well. For buttons it seems rather clean which are transmitted 1:1 to Go, for axes not so much as they're remapped in the Java code. Probably should move the remapping into Go and then we also have a trivial standard layout mapping.

[divVerent]

Starting work on this now.

Seems like it requires some minor refactoring, so did that first.

[divVerent]

Minor refactoring: divVerent/ebiten@1b617f4

If you can, @hajimehoshi, please quickly confirm whether the internal gamepad API change for this is OK (regardless whether it compiles yet or not, I'm not that far yet). If so, I will try to add Android and Linux native standard layout support using this.

[hajimehoshi]

There seems a compile error.

Do you want to rename some functions by the way?

[hajimehoshi]

Ah now I understand what you want to do. The direction seems correct.

[divVerent]

OK, got a first draft but I started for Linux as it is easier to develop there and the same problems need solving.

My android_stdlayout branch now supports my Xbox 360 gamepad mostly even without gamepaddb, but no D-pad or triggers yet.

The issue is that to support these, we need to be able to map axes vs hats vs buttons, which SDL does and gamepaddb does, but the current framework to do standard layout remapping in Ebitengine does not.

[divVerent]

So, approach will be:

• Find out if we need the same for Android.
• If yes, add this first.
• If not, the current incomplete support is still better than no support (after all, gamepaddb still overrides), so PR that and in the meantime work on Android support already.

[divVerent]

Seems like this IS needed: https://developer.android.com/develop/ui/views/touch-and-input/game-controllers/controller-input

E.g. AXIS_HAT_{X,Y} and BUTTON_DPAD_{UP,DOWN,LEFT,RIGHT} should both be mapped to the D-pad.

So I think next step is further refactoring so we return a struct indicating whether a standard button/axis is backed by an axis, button or hat. At least for Linux we'd need that anyway.

Now one more thing is irritating: addAndroidDefaultMappings() seems to exist as a different way to achieve the same, by parsing the axis and button masks from the SDLID. Then how come it doesn't actually work? Nevertheless I'd be very tempted to implement the more "proper" thing anyway, but it might actually be possible to make THAT work instead. Gotta decide for one or the other :)

[divVerent]

While at it, dumping here how axis<->button<->hat mapping works in gamepaddb:

	from Button	from Axis	from Hat
to Button	1:1	(x + 1) / 2 > (30/255 + 1) / 2	1:1
to Axis	b ? 1 : -1	1:1	h ? 1 : -1

[hajimehoshi]

Now one more thing is irritating: addAndroidDefaultMappings() seems to exist as a different way to achieve the same, by parsing the axis and button masks from the SDLID.

I'm not sure I fully understand it, but it is OK to refactor the existing functions to achieve your purpose.

[divVerent]

Yeah, me neither. The functionality I want for Android to support generic gamepads is there (but hidden inside gamepaddb code, not native gamepad mapping code like all the others).

I gotta figure out why it is failing for my gamepad then (and one of my users reported the same).

So, possibly I will solve this by fixing addAndroidDefaultMappings() - possibly I will solve this by deleting it and implementing native standard gamepad mapping support for Android. First gotta see why it fails. Kinda sad I can't really compare to SDL in practice, as I have no SDL Android app available that supports gamepads and do not know how to compile one. In particular it'd be useful to know if my gamepad gets the wrong SDLID, or if we are parsing the SDLID to standard mapping wrong. Either way, I am not convinced at all that addAndroidDefaultMappings() even uses the right approach, as it only gets a few bits of information and very likely we need more than that to fully support standard layouts on Android.

Either way, though, for full Linux support I'll have to refactor standard button/axis support to be able to map to buttons, axes and hats. This will then also enable automatic standard layout on Windows/XInput, however I probably won't be able to be the one to implement that.

The big goal here is to deprecate the need for SDL_GameControllerDB for anything but "weird gamepads" - anything Xbox or PlayStation compatible then should work out of the box. (Not sure about Nintendo there, didn't see references to them on SDL source code much)

[divVerent]

Results with my gamepad and Ebitengine 2.5:

Name:Generic X-Box pad SDLID:05000000d620000002280000ff7f3f00 AxisCount:6 ButtonCount:25
trigger left = axis 2 stdaxis 2 AXIS_Z
trigger right = axis 5 stdbutton 7 AXIS_RZ
shoulder left = button 9 stdbutton 4 L1
shoulder right = button 10 stdbutton 5 L2
left stick x = axis 0 stdaxis 0 AXIS_X
left stick y = axis 1 stdaxis 1 AXIS_Y
right stick x = axis 3 stdaxis 3 AXIS_RX
right stick y = axis 4 stdbutton 6 AXIS_RY
select = button 4 stdbutton 8 SELECT
guide = button 5 MODE
start = button 6 stdbutton 9 START
dpad = button 11,12,13,14 button 21,22,23,24 stdbutton 12,13,14,15 DPAD_*
a = button 0 stdbutton 0 BUTTON_A
b = button 1 stdbutton 1 BUTTON_B
x = button 3 stdbutton 3 BUTTON_X
y = button 2 stdbutton 2 BUTTON_Y
lstick = button 7 stdbutton 10 THUMBL
rstick = button 8 stdbutton 11 THUMBR

I'll edit this reply once I used another app that gives me the button and axis names from Java's POV.

[divVerent]

So the issues are:

• Trigger left is mapped to StandardAxisRightStickHorizontal (should be StandardButtonFrontBottomLeft)
• Right stick Y is mapped to StandardButtonFrontBottomLeft (should be StandardAxisRightStickHorizontal)
• Guide is mapped nowhere (should be StandardButtonCenterCenter)

Rest is actually correct.

This is on my Pixel 6 - on my Moto G7 Play I get no gamepad support at all.

[divVerent]

As for the Guide button, we have this:

›   ›   // TODO: If SDKVersion >= 30, add this code:                             
›   ›   //                                                                       
›   ›   //     gamepadButtonMappings[id][StandardButtonCenterCenter] = &mapping{ 
›   ›   //         Type:  mappingTypeButton,                                     
›   ›   //         Index: SDLControllerButtonGuide,                              
›   ›   //     }                                                                 

However, why NOT already add this? What does this do if SDKVersion is too low?

Other than that, this remapping code is very weird and just can't be right in general. E.g. the axis mask is just computed from the number of axes - not which are actually available.

The main problem is, whatever I do may conform to Android docs, but break some corner case the SDL code covers. But there are also cases vice versa. So I wonder if this should be introduced as a build flag or as an ebitengine API option one can set at startup.

[divVerent]

But yeah, it seems like Ebitengine 2.5 fixed some of this already - the issues were worse in 2.4. I see some changes have been done there for 2.5, e.g. for 47558d20c505edf2a43eabeebbd39cffe3158579

[divVerent]

OK... I think I'll get the Linux version of this change in, and then probably will give it a try to implement Google's guidelines in Ebitengine.

The current implementation is heavily based on SDL's, and part of it is sure required to keep the axes and buttons indexed consecutively for Ebitengine's non-standardlayout-API use - so I will probably have keep most current code as is even if I use the native gamepad support (that does not go via gamepaddb) I implemented.

[divVerent]

Hm... now that I went through it, the current implementation might be some kind of nonsense for axes (but that's only really a problem for gamepads that lack a right stick but have triggers).

For buttons it's sound though. And it at least roughly matches SDL, so I may want to keep it as is for now, modulo bugs.

So I improved debugging in AAAAXY to be able to logcat the actual mapping, and will see later what it takes to fix it (possibly locating accidental diffs with SDL).

[divVerent]

(However, doing this mapping inside gamepaddb is probably wrong as well - but not our issue. I may want to refactor this code from gamepaddb.go into gamepad_android.go, but that's indepedent from the fixing here)

[divVerent]

Tried again, this time with better debug code to remove chance of human error.

Full list of mistakes:

• Guide button is not mapped (seems like there's commented out code in Ebitengine for that - no idea why)
• Right stick X axis on device (reported as axis 3) controls right stick Y axis in standard layout
• Right stick Y axis (reported as axis 4) controls left trigger in standard layout
• Left trigger (reported as axis 2) controls right stick X axis in standard layout

So, in terms of Linux's mapping, ABS_RX is understood as ABS_RY, ABS_RY as ABS_Z, and ABS_Z as ABS_RX.

Axes 0 and 1 are ABS_X and ABS_Y as they should be. Axis 5 is the right trigger, also as it should be.

[divVerent]

Axis	Index	Index that would make things work
ABS_X	0	0
ABS_Y	1	1
ABS_Z	2	4
ABS_RX	3	2
ABS_RY	4	3
ABS_RZ	5	5

So if we made the sorting X, Y, RX, RY, Z, RZ, things would work just fine. But of course, if we go by Android's documentation, it should be X, Y, Z, RZ, RX, RY.

Wonder what SDL does...

[divVerent]

By reading source, SDL does the same mistake. But I can't try out without having an SDL based gamepad app and knowing how to compile it...

[divVerent]

If I find nothing better, THIS might actually be a starting point for adding logging of gamepad events: https://github.com/georgik/sdl2-android-example

[divVerent]

Looks like SDL2 comes with its own gamepad test, just is a bit tricky to compile - need the attached patch and then run:

for x in gamepadmap.bmp gamepadmap_back.bmp button.bmp axis.bmp; do xxd -i "$x"; done > bmps.h
export ANDROID_HOME=$HOME/Android/Sdk
export ANDROID_NDK_HOME=$HOME/Android/Sdk/ndk/25.0.8775105
../build-scripts/androidbuild.sh org.libsdl.testgamepad testgamepad.c testutils.h testutils.c bmps.h
( cd ../build/org.libsdl.testgamepad && ./gradlew installDebug )

internalbitmaps.zip

[divVerent]

With this, all is perfect in SDL. Need to figure out if I'm using a gamecontrollerdb mapping somehow though...

[divVerent]

03-04 08:04:18.696 28056 28092 I SDL/APP : Gamepad device 1 added.
03-04 08:04:18.698 28056 28092 I SDL/APP : Opened gamepad BDA Xbox ONE Core controller, /dev/bus/usb/002/002/20d6/2802/0
03-04 08:04:18.698 28056 28092 I SDL/APP : Rumble supported
03-04 08:04:18.699 28056 28092 I SDL/APP : XBox One Controller: BDA Xbox ONE Core controller, /dev/bus/usb/002/002/20d6/2802/0 (guid 0300c289d62000000228000000006800, VID 0x20d6, PID 0x2
802, player index = 0)
03-04 08:04:18.699 28056 28092 I SDL/APP : Mapping: 0300c289d62000000228000000006800,*,a:b0,b:b1,back:b4,dpdown:b12,dpleft:b13,dpright:b14,dpup:b11,guide:b5,leftshoulder:b9,leftstick:b7,
lefttrigger:a4,leftx:a0,lefty:a1,rightshoulder:b10,rightstick:b8,righttrigger:a5,rightx:a2,righty:a3,start:b6,x:b2,y:b3,crc:89c2,

[divVerent]

That mapping corresponds to sorting in the order X Y RX RY Z RZ.

[divVerent]

So, next step is adding debug code into SDL to find out where it's making the mapping work.

[divVerent]

HAHA, on SDL we're not even hitting that Java code. The joystick is opened via hidapi.

Wonder if I can switch to using that Java code instead...

[divVerent]

When using SDL's Java code, the exact same bug happens:

03-04 09:48:35.345  5655  5690 E SDL     : motion range axis 0 min -1.0 max 1.0
03-04 09:48:35.346  5655  5690 E SDL     : motion range axis 1 min -1.0 max 1.0
03-04 09:48:35.346  5655  5690 E SDL     : motion range axis 11 min -1.0 max 1.0
03-04 09:48:35.346  5655  5690 E SDL     : motion range axis 12 min -1.0 max 1.0
03-04 09:48:35.346  5655  5690 E SDL     : motion range axis 13 min -1.0 max 1.0
03-04 09:48:35.346  5655  5690 E SDL     : motion range axis 14 min -1.0 max 1.0
03-04 09:48:35.346  5655  5690 E SDL     : motion range axis 15 min -1.0 max 1.0
03-04 09:48:35.346  5655  5690 E SDL     : motion range axis 16 min -1.0 max 1.0

So on SDL my gamepad (and probably all Xbox ones) only works when using the HIDAPI implementation.

To make it actually use the Java implementation, I had to:

-#define SDL_JOYSTICK_HIDAPI     1
+// #define SDL_JOYSTICK_HIDAPI     1

I'll file a SDL bug, but as SDL has a solution by an alternate API, I do not expect them to fix it.

[divVerent]

The alternate API that SDL uses is IIRC basically Linux raw device access, which happens to work now but I expect Google to lock that down more and more in future API versions - so I'd rather not want Ebitengine to rely on that.

[divVerent]

Reported as libsdl-org/SDL#7404 and am going to provide a PR to that that will keep existing working SDL_GameControllerDB entries alive.

[divVerent]

Filed libsdl-org/SDL#7405 - and if that gets merged, I'll port the same change into Ebitengine to keep the two in line (so that they interpret SDL_GameControllerDB files the same way).

[divVerent]

Provided alternate fix option in libsdl-org/SDL#7405 (comment) - this one is probably cleaner but may impact existing gamecontrollerdb mappings. Let's have SDL team pick and mirror it in Ebitengine.

[divVerent]

Done - changes are in and my gamepad which is a no-name xbox compatible pad now works out of the box.