Mouse events simulation doesn't work on Android 11

[SDLBugzilla]

This bug report was migrated from our old Bugzilla tracker.

These attachments are available in the static archive:

• screenshot gamepad (Screenshot_20200928-194649.png, image/png, 2020-10-05 20:57:50 +0000, 340457 bytes)

Reported in version: HG 2.1
Reported for operating system, platform: Android (All), All

Comments on the original bug report:

On 2020-10-02 20:47:18 +0000, Deve wrote:

I mean the events that are sent when SDL_HINT_TOUCH_MOUSE_EVENTS is enabled. On Android 11 it doesn't work anymore. It seems that Android doesn't report touch id, because the id is always -1, so that the

if (id != SDL_MOUSE_TOUCHID)

in SDL_SendTouch() SDL_touch.c:261 is never true and mouse event is never sent.

We have bug reports from some people that SuperTuxKart doesn't work on Android 11 (we use mouse events for GUI navigation).

I also reproduced it under emulator by using "input tap x y" command from adb shell (it was working on older versions, but didn't work on Android 11).

From what I understand the SDL_MOUSE_TOUCHID is just to avoid multiple events, that eg. mouse event simulates touch event, and then touch event shouldn't simulate mouse event again (true?).

As a quick fix we changed SDL_MOUSE_TOUCHID to different value, so that it doesn't conflict with the id=-1.

I can make a patch or make some tests, but do you have some particular fixes in mind? Add additional parameter to SDL_SendTouch() function for simulated events? Or just change SDL_MOUSE_TOUCHID value?

Btw. there are also some issues with gamepad support on Android 11, but I don't have real device to test... I will open separate bug report later.

On 2020-10-03 11:46:32 +0000, Sylvain wrote:

Just the context: SDL2 is creating Mouse (resp. Touch.) 'synthetic' events when the user move the real Touch (resp. Mouse) device. They appear with an id -1.
And only if the hints are enabled.

https://hg.libsdl.org/SDL/file/1b1fae168557/include/SDL_touch.h#l60
/* Used as the device ID for mouse events simulated with touch input /
#define SDL_TOUCH_MOUSEID ((Uint32)-1)
/ Used as the SDL_TouchID for touch events simulated with mouse input */
#define SDL_MOUSE_TOUCHID ((Sint64)-1)

https://hg.libsdl.org/SDL/file/1b1fae168557/include/SDL_hints.h#l350
See the hints:
SDL_HINT_MOUSE_TOUCH_EVENTS
SDL_HINT_TOUCH_MOUSE_EVENTS

...

you can add traces in SDLActivity.java: public boolean onTouch(View v, MotionEvent event):
trace the value of "touchDevId", it should be a real value, not -1.

( https://developer.android.com/reference/android/view/MotionEvent#getDeviceId() )

But maybe the simulator reports a virtual device. And maybe you can dump the "event.getSource()" too.

(If needed, other places:
src/video/android/SDL_androidtouch.c: Android_OnTouchAndroid_OnTouch()
src/video/android/SDL_androidmouse.c: Android_OnMouse() )

Maybe there are stuff in the bug report:
"Android, some issue with InputDevice checks"
https://bugzilla.libsdl.org/show_bug.cgi?id=3958
I saw various strange checks.

On 2020-10-04 09:23:15 +0000, Sylvain wrote:

If Android just send -1 as a valid id, the easiest patch for SDL2, would be to add:

"if (touchDevId < 0) touchDevId -= 1;"

So that -1 is restrict to the synthetic events.

https://hg.libsdl.org/SDL/file/1b1fae168557/android-project/app/src/main/java/org/libsdl/app/SDLActivity.java#l1952

Also, maybe please dump stuffs from "initTouch()"

https://hg.libsdl.org/SDL/file/1b1fae168557/android-project/app/src/main/java/org/libsdl/app/SDLActivity.java#l1220

On 2020-10-05 07:59:18 +0000, Sylvain wrote:

I tried with Android 11 emulator rev8, and I also got -1 id when using adb shell.
to send mouse and touch events.

When using the real mouse or real touch screen, it worked though.

Here's a fix, can you double check:
https://hg.libsdl.org/SDL/rev/e1c4081a20db

On 2020-10-05 08:32:42 +0000, Deve wrote:

I think just make sure that touchDevId is not equal to SDL_MOUSE_TOUCHID should be enough, so your fix should work. I will test today.

Though in theory it's possible on other platforms too, so maybe better use additional parameter, something like this?

int SDL_SendTouch(SDL_TouchID id, SDL_FingerID fingerid, SDL_Window * window, SDL_bool down, float x, float y, float pressure, bool simulated = false);

And are you able to test real gamepad device? It seems that it returns -1 too...

On 2020-10-05 17:32:23 +0000, Sylvain wrote:

• We don't want to drop the -1 id coming from Android, it's valid even if it's a virtual device. we re-affect it to -2, and other n negative to n - 1. We keep -1 for internal sdl synthetic id.
  We could change the function, as well, not sure why we didn't do that.

I have no real android 11 so I used the emulator. Also I can't connect a gamepad and only use the shell commands.
id -1 seems to be the joystick named Virtual

This probably need to be changed to accept -1 :
https://hg.libsdl.org/SDL/file/6b1b46ba0a2d/android-project/app/src/main/java/org/libsdl/app/SDLControllerManager.java#l95

I see the joystick appearing, but I could get it working neither with android 11 nor old android.

The check here might be wrong
https://hg.libsdl.org/SDL/file/6b1b46ba0a2d/android-project/app/src/main/java/org/libsdl/app/SDLControllerManager.java#l95
could be:
int source = event.getSource();
if (source == InputDevice.SOURCE_JOYSTICK || source == InputDevice.SOURCE_GAMEPAD || source == InputDevice.SOURCE_DPAD) {

Just after, all MotionEvent.ACTION_* are not handle.
so, we might want to return "true" for ACTION_MOVE and false otherwise...
and/or move the code onNativePad{Up,Down} from SDLActivity.
Not really sure ...

On 2020-10-05 20:57:10 +0000, Deve wrote:

If you get Virtual -1, then it's the same what gets that guy on his Pixel 4 XL Phone with Android 11 and Logitech USB controller (see the screenshot).

And he already tested my quick fix with changing SDL_MOUSE_TOUCHID to different value and it fixed these mouse events for him. I can ask him to test your patch if you need to try it on real device.

I also tested your patch on android emulator and it works for me too.

On 2020-10-05 20:57:50 +0000, Deve wrote:

Created attachment 4474
screenshot gamepad

On 2020-10-06 06:17:54 +0000, Sylvain wrote:

Please don't use your patch (eg.
As a quick fix we changed SDL_MOUSE_TOUCHID to different value, so that it doesn't conflict with the id=-1. )

Ask the user to test:
with this first patch in https://hg.libsdl.org/SDL/rev/e1c4081a20db

Also, in addition, with this second patch (to be made) to accept -1:
https://hg.libsdl.org/SDL/file/6b1b46ba0a2d/android-project/app/src/main/java/org/libsdl/app/SDLControllerManager.java#l95

replace:
if ((device == null) || (deviceId < 0)) {
by
if (device == null) {

Please tell if all is ok with those two patches, thanks

On 2020-10-06 08:27:53 +0000, Sylvain wrote:

NB: by default, even on non Android 10, and without gamepad connected,
my phone report a device id with -1,

Virtual id=-1
device id == 769 (512 + 256 + 1)

which is CLASS={ BUTTON }
source(s): DPAD and KEYBOARD

So maybe your issue with gamepad, is totally different: HID not compiled ? gamecontrollerdb.h entry missing ?

On 2020-10-07 21:37:33 +0000, Deve wrote:

I asked that guy to test your patches, will tell you when I will get some information.

The "if (device == null)" will not work for multiple gamepads if all gamepads will have device id -1 (would be useful eg. on Android TV). But not sure how to solve it... if device id = -1 then generate id based on device name?

And I wonder if it's maybe a bug in android, because it's a bit strange behavior... I had a quick look and there is one related commit

https://android.googlesource.com/platform/frameworks/native/+/0d8ed6e8ade20652cea20f579f40b1d698ce8fc0

But it's for injected events, so I'm not sure if/how it affects real touch/gamepad events.

[gbgames]

I do not have an Android 11 device to test it myself, but two play testers have Android 11 phones.

One reports that touchpresses are not getting detected, which sounds like this issue. I confirmed that the app isn't frozen on their device. He said his OnePlus 7T Model HD1907 upgraded to Android 11 shortly before he started testing, which means I don't know how it would have worked on that device with Android 10.

The other play tester, however, says that the app functioned just fine on his Galaxy S20. The only issue is that it ran in portrait mode instead of landscape mode, despite specifying landscape mode in the Android manifest. But that is a different issue.

I am still using SDL2_2.0.12, though, and targeting Android's SDK 29 (Android 10), and while I plan to try 2.0.14 to target Android 11, I figured that old apps should still run as expected. Perhaps Android 11 made a breaking change to some behavior that SDL2 is depending on, bu as described above, it only affects certain devices?

[1bsyl]

@gbgames
This report was about touch event simulation. For instance, connecting a mouse, using the hint SDL_HINT_TOUCH_MOUSE_EVENTS, and expecting to receive SDL touch event, with Id == -1.
Is that what you're doing ? (otherwise, please open another issue )
anyway:

• yes, please update to latest head / dev SDL2 version
• update also the android sdk revision/tools/etc. (don't change the target SDK 29!)

[gbgames]

@1bsyl I apologize. I misunderstood since this issue mentioned other people reporting Super Tux Kart not working with Android 11, and I assumed they were using a device with a touchscreen.

To clarify, the issue I'm experiencing is that touch events are not registering on the actual devices, and so mouse events are not being simulated as expected.

But you're saying this issue is about a real mouse being used and touch events are not being simulated?

If so, I'll create a new issue.

Also, are you suggesting that the latest SDL2 dev version has addressed the mouse simulation issue?

[1bsyl]

@gbgames ok, create a new issue

[Benau]

For the record this issue has been fixed in latest sdl for a while 2e38c94

But more importantly the way google changed to hide the device id from api breaks gamepad too:
https://issuetracker.google.com/issues/186142510

But google doesn't seem to care, maybe if SDL2 devs complain to it it may reply..

[1bsyl]

Closing the issue since it s fixed