feat(web): the gesture-recognition engine 🐵

[jahorton]

This feature branch is devoted toward development of a module to facilitate #5029 in addition to our existing OSK gestures.

By developing an isolated module for gesture support, we'll be able to demo and test gesture behavior outside of KeymanWeb. One of my design goals with this is actually to enable gesture unit tests by recording tracked input sequence data. This may be achieved by keeping the core logic headless and replicating recorded input sequences in a headless environment.

This branch should not be merged into master until the new module is sufficiently ready and has been integrated into KeymanWeb properly. Until then, it should live as a feature branch.

---

Phase 1: the base GestureRecognizer

This section will be devoted to establishing the interface for the basic infrastructure needed to properly support gestures given the constraints needed by KeymanWeb and ensuring that future work on this module will be testable independently of Web.

Goals:

• a host page that can replicate the needs of a touch-platform OSK, including screen edge limitations, etc.
  • Such a page will be quite useful for demoing gesture behavior without the need to worry about how it will affect Web.
• Either the same page or an extended page that allows recording of inputs received, for use in unit-test development (much like web/tools/recorder's role for KeymanWeb's engine tests)
• The ability to support tracking of multiple touchpoints.

PRs for this phase:

• feat(web): the gesture-recognition engine 🐵 #6842 (this one!)
  • For future reference: through commit d5222df.
• feat(web): start of the GestureRecognizer class, use of event-based paradigm 🐵 #6843
• feat(web): start of input boundary generalization 🐵 #6844
• feat(web): first gesture-recognizer host page 🐵 #6878
  • Hooray for something interactive!
  • Has an initial set of user tests.
• feat(web): start of support for multi-touchpoint tracking 🐵 #6914
• feat(web): Gesture Recognizer input sequence recorder 🐵 #6952
  • Adds an input-sequence recorder page useful for unit-test preparation
• feat(web): Gesture Recognizer unit testing 🐵 #6970
  • Actual unit tests!

Phase 2: the GestureSegmenter

This section will be devoted to analyzing raw input-event data and mapping those time-based input sequences to gesture "segments" - small-scale gesture components that will serve as "building blocks" for complex gesture types, their differentiation, and state modeling.

• feat(web): gesture touchpath segmentation core 🐵 #7058
• feat(web): touchpath segment objects 🐵 #7142
• feat(web): headless segmentation unit tests 🐵 #7157
• Possibly one more (for 'integration'-style, input-event through to segmentation tests)
  • There's a strong chance I'll skip this aspect for now unless requested otherwise.
  • They're definitely possible to write, but this feature's implementation is running slower than originally projected.

Phase 3: the GestureSynthesizer

Once the "building block" logic is sound and stable, we may then use those components to construct the complex gestures needed by KeymanWeb's OSK and the internal state machine(s) needed for differentiation, detection, recognition, cancellation, and resolution.

• TBD.

Phase 4: Keyman Engine for Web integration

Once the module is ready to go, we need to integrate it with our Web engine's OSK.

Related Issues

• chore(web): extend gesture-recognizer testing to support reproduction of single inputs with multiple touchpoints 🐵  #7067

---

For this initial PR, the focus is on:

• establishing baseline infrastructure for this module
• ensuring that CI builds verify that the module builds properly
• copying over the relevant base classes from KeymanWeb as a starting point, without unnecessary changes.

As the actual TypeScript code here is unchanged from KeymanWeb, outside of a few blocks being commented out due to external references, do not feel a need to consider it during the review process.

@keymanapp-test-bot skip

[keymanapp-test-bot]

User Test Results

Test specification and instructions

User tests are not required

Test Artifacts

• Android
  • Keyman for Android apk
  • FirstVoices Keyboards for Android apk
  • KMSample1 apk
  • KMSample2 apk

[jahorton]

Now that we're moving this to a more "proper" feature branch, it's time 'merge' this and let the already-merged stuff sit in its new home.