Multitouch on Windows 10: Events queue up under heavy load

[PrimaryFeather]

Problem Description

In Windows 10, when an AIR app is under heavy load (read: it does not reach the requested frame rate) and there are many touch events (or even mouse clicks), events will queue up and will be dispatched long after the finger left the screen (or the mouse has stopped clicking). In effect, the app will become unusable for a while (becoming worse the more events queued up).

• AIR SDK 28 and 29 beta.
• Reproducible on Windows 10.
• Not reproducible on iOS and Android.
• The topic came up originally in this Starling thread, but happens in non-Starling apps, too.
• This Adobe Tracker issue seems to concern exactly the problem described: AIR-4081695.

Steps to Reproduce

Use the code from this gist to reproduce the problem.

Experiment A

• Instantiate the provided class (TouchExperimentsFla) in an AIR project for Desktop and add it to the stage. On each click / touch, a dot will appear on the stage.
• Set the frameRate of this project to 30 fps.
• The method onEnterFrame is responsible for creating the heavy load. Adapt the value of count (line 76) so that you reach only about 20 frames per second, or less.
• Now repeatedly tap the screen with 5-10 fingers, and repeat (fast!) for about 5 seconds.
  • Alternatively, click on the mouse as often and as fast as you can.
• Result: the touch / mouse events will continue to fire long after you stopped tapping / clicking!

Experiment B

• Now change the value of count so that the app easily reaches the 30 fps.
  • Alternatively, change the target frameRate to just, say, 10.
• Again, create as many touch/click events as you can for a few seconds.
• Result: you'll find that this time, events will not queue up. This is how it's always supposed to work.

Note: if you run experiment A on iOS or Android, you'll find that there won't be any queuing there, either. The problem is really limited to Windows.

Note: Please also check the sample posted by @MadNoMad0 below!

Known Workarounds

Developers need to make sure target FPS is always easily reached by the system.

[dedosmedia]

+1 This bug needs Adobe attention.

[tadeubas]

I think this issue is very old and is not related to Win10 only or touch + mouse only. I made a game in 2012 that used Keyboard + Mouse events for player actions, and when the CPU load was intense (FPS drop), both events would delay a lot, only if I use both inputs. Only using the mouse or only using keyboard the problem would not appear, even with CPU load and FPS drop.

[PrimaryFeather]

Ah, I see! That's useful information, thanks @Tudumanu!

[MadNoMad0]

As a follow up to the discussion here - https://forum.starling-framework.org/topic/windows-multitouch-problem/page/3?replies=75#post-112611 - I've posted a rather simple code here - https://gist.github.com/MadNoMad0/106a347b3551acf0b802104e27f839ac - that helps visualize how the AIR runtime can temporarily freeze when processing even the most simple sequence of touch events. I believe this phenomenon is not only related to the issue reported by @PrimaryFeather, but it may be at the very heart of it. To experience the freeze, do the following:

1. Run the gist using any AIR version (tested up to 29) in Windows 8 or 10.
2. Touch an empty area of the stage with 1 finger.
3. Now repeat the gesture with a second finger.
4. Lift the first finger.
5. Lift the second finger.

Repeat this sequence of touches until you see the circle animation freezing for about a second. It may take several attempts, but it does happen repeatedly. It could happen in the very first attempt, and it might also happen a couple times in a row. If, during the freeze, a new touch is registered, the application immediately unfreezes.

I've tried different framerates down to 10 fps, and still managed to cause the freeze. It does not happen in Windows 7.

[PrimaryFeather]

Here's another useful diagnosis from @MadNoMad0 on the Starling Forum:
https://forum.starling-framework.org/topic/windows-multitouch-problem/page/3?replies=79#post-114899

Definitely interesting for anyone who's looking into this issue!

[PrimaryFeather]

Apparently, Adobe made an attempt of fixing this in the latest AIR 31 Beta (that's a new version succeeding the official AIR 31 build). In my first quick test, I could still reproduce it, but this could simply mean that it's fixed for just some of the affected devices. Maybe someone else can give it a try?

[MadNoMad0]

Thanks for the heads up, Daniel. Well, I've been testing this new version and I'm very happy to inform that the issue is gone. To be honest, I haven't had a chance to test on multiple devices so your initial diagnostic may me correct. But on an old PQ Labs G3 overlay it is now running butter smooth. And the performance difference is staggering.

My one half wants to congratulate Adobe and say a big "thank you" for finally squashing this serious bug. But my other half just wants to shout "what took you so long, Adobe?". Anyway, better later than never, right?

Now, it should be interesting to hear from Adobe if the issue is in fact hardware specific. Regardless, I'll be testing this on every touch sensor I can lay my hands on and I'll report back here.

[PrimaryFeather]

Ah, that's great news! Thanks a lot for trying it out!

You're right, even if it took so long, what counts right now is that it's fixed – or at least closer to a complete fix than ever before. In any case, I'll re-run my tests from yesterday, maybe it was an error on my side.

[PrimaryFeather]

Okay, I stand corrected: I just tried again, and I can't reproduce the problem any longer. I'm a little ashamed to admit, but I think I made a debug build.

So, it seems this is really fixed! W00t! Thanks to everyone involved! 😄

[PrimaryFeather]

Okay, one more comment on this (mainly so that it's documented):

With extreme settings using the above test (the Gist) and a high value of count inside onEnterFrame, I can actually still reproduce the problem - but it's less severe than before and probably shouldn't be an issue any more in realistic scenarios. That's where my false alarm from yesterday came from.

@MadNoMad0, please still post your results here, so that we're sure about the fix. Thanks in advance!

[Stortof]

I made a quick test here and for me the issue is gone :)
That a really good news for me as I encountered so much problem with this issue...

[PrimaryFeather]

Fantastic! Thanks for trying it out, @Stortof!

[PippoApps]

This is great news. I am just about to start a wall screen project involving multiple users interacting with the same app, and AIR is the only viable choice due to time constraint.