Improve accuracy and detail of touch gestures #5652

michaelDCurran opened this Issue Jan 5, 2016 · 2 comments

3 participants


All though the over all UX for NVDA touch input could be improved, first it is necessary to make improvements to the underlying touch gesture detection code itself.
Currently there are several limitations and bugs with touch trackers and gestures such as:

  • two fingers performing the same action are only merged into a multi finger action if the second finger contact starts after the first's start but before its end. This does not allow merging of fingers that overlap in other ways.
  • It is possible for the user to get things like a tap and a tripple tap when trying to do a quadruple tap if multiple fingers are used.
  • multi-finger and plural tap and holds are not supported very well.
    • Hovers flood speech in input help mode.
  • It is impossible for scripts to find out information such as the coordinates of single fingers in a multiTouch gesture as it is lost in processing.
  • Gestures with a hold (E.g. 1 finger hold + tap) are either non-existant or impossible to bind due to Python set ordering not being used.

Incubated in fe46f4f.


Incubated in 1c5884d.

@jcsteh jcsteh added a commit that closed this issue Jan 22, 2016
@michaelDCurran michaelDCurran Fixes and improvements related to touch screen gestures.
* All taps that overlap in time are correctly merged into one multi-finger tap. Previously tap A and B would be merged only if B started after A's start but before its end.
* Taps are only merged into plural (double, tripple, quadruple) taps if there are no other unknown trackers at the time. This stops a tap quickly followed by a tripple tap for example, from incorrectly turning into a quadruple tap.
* Plural tap and holds (e.g. double tap and hold) are now detected.
* Hover downs are no longer fired when part of a tapAndHold.
* Hovers are no longer reported in input help.
* MultitouchTracker objects now contain a childTrackers property which contains the multiTouchTrackers it is made up of (I.e. 2 finger double tap contains 2 2-finger taps. The 2-finger taps themselves contain 2 taps).
* MultiTouchTracker objects now also contain a rawSingleTouchTracker property, if this tracker is the result of one single finger doing a tap, flick or hover. The SingleTouchTracker allows access to the underlying Operating System assigned ID for the finger and whether or not the finger is still in contact at the current time.
* Added doc strings to MultiTouchTracker and SingleTouchTracker classes explaining all available properties
* Provide __slots__ on MultiTouchTracker and SingleTouchTracker classes to decrease their size.
* The Touch input Gesture class's doc string now explains all possible actions that can occur
* Touch input gestures now have x and y properties, removing the need to access the tracker in very simple cases.
* Gesturs now contain a preheldTracker property, which is a multitouchTracker object representing the other held fingers while this action was being performed.
* Gesture identifier IDs are now output in Python set order ensuring correct comparisons, allowing gestures such as "1finger_hold+hover" to be correctly bound.
* A generalized identifier with finger count removed is now included for holds (e.g. hold+hover for 1finger_hold+hover).

Fixes #5652.
@jcsteh jcsteh closed this in d7014eb Jan 22, 2016
@jcsteh jcsteh added this to the 2016.1 milestone Jan 22, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment