Old and obsolete multi-key selection thought: batch submission of keyboard buttons – Variable dwell time necessary for activation

[JeffKang]

Old and obsolete multi-key selection thought: batch submission of keyboard buttons – Variable dwell time necessary for activation

Hey, amazing job on that multi key select option!
Around last year, I was thinking of what could be a faster typing method, and the multi key is like a successful version of what I pictured.

I made a draft for it, but I scrapped it because it started to not make sense.

If it's alright to put up a junk post, I'll leave it here for the thought process:

Slower eye-typing methods

I’ve seen that some of the methods of eye-typing on a virtual/soft keyboard can be slow, as you have to dwell on each letter for a certain amount of time

For example, if you want to produce the word, “testing”, and you fixed the dwell time at 1 second, you would stare at the letter “T” for 1 second, stare at the letter “E” for 1 second, and so on, and so on.

Ordinarily, if a user wants to go as fast as possible, they would try to time it so that they move their eyes away to the next letter at very close to 1 second.
If they leave at 0.95 seconds, they would have to go back, losing the time and distance that was involved in gazing away too soon.

An example of eye typing can be seen at 5:26 of the PCEye video: https://youtu.be/6n38nQQOt8U?t=5m26s

Batch submission of keyboard buttons – Variable dwell time necessary for activation

One idea to speed things up is to let the dwell time necessary for activating elements be variable, and have it be chosen by analyzing which letters were fixated on the longest.

For example, using the same word, “testing”, you would start by having your eyes go to “T”.
On your way there, you might glance at “Y” or “G”, two letters next to “T”, for a brief amount of time, such as 0.2 to 0.3 seconds before dwelling on “T” for 0.5 seconds
On the way to the second letter of “E”, you might skim by “R” for another short period of time, like 0.1 seconds
Once your gaze is on “E”, you might dwell on it for 0.7 seconds, and then dwell on the third letter of “S” for 0.6 seconds.

At this point, a decision can be to activate a button that will evaluate the dwell times that were just recorded.
The program would see that three letters had dwell times of 0.5, 0.7, and 0.6 seconds, which would be distinctly longer than the logged fixation times of the rest and most likely majority of the letters (0.1 to 0.3 seconds), and would thus select the letters that had longer times, and then output them.

As long as certain buttons have times that are larger than the average time of the batch plus a certain amount of standard deviations, they would be the ones to be submitted.

Low number of dwell times that keep average too high - add some short times as filler to lower average

If a user wants to evaluate a small batch of only a couple of letters, some 0.1s could be added to lower the average, since the target buttons with their longer dwell times might be more of the majority, and the average might be too high, which could exclude one of the longer dwell times.

The amount of 0.1s needed to be added could decrease as the batch of letters increase, as there would probably be enough skimmed letters to add their short dwell time, and lower the average.

Automatic submission upon a maximum dwell time

Besides dwelling on a separate submission button when you’re ready to send, another possible option could be to have an automatic evaluation and submission once a maximum dwell time of something like 1.4 seconds is reached on a particular button.

Resume/accumulation

If any jumpiness from eye tracking causes the fixation on an intended letter to be briefly interrupted, you would need to recognize a quick resumption of the dwelling, and continue building the time for that particular letter.
Any jumps to, and quick resumptions of the nearby unintended letters during a quick resumption of an intended letter would have to not cause a buildup of dwell time of unintended letters.

There could be a minimum dwell time requirement of something like 0.3 seconds before a button is a candidate for accumulation.

Or, if the target button is small, or it’s extra jumpy, it could be a minimum of 0.17, a quick return to the button, and another 0.17.
A quick return to the button lowers the resumption requirement from 0.3 to 0.17, as a return signifies that the particular button might be one if the desired targets.

Buttons that obtain a fixation that indicates interest (higher than prespecified threshold, or sufficiently higher than average) will remove all partial fixations.

(GazeTalk (free, predictive text entry system by Gaze Group – http://wiki.cogain.org/index.php/Gazetalk_About) has an "Accumulate dwell time" function to “avoid the re-setting of a button”.

GazeTalk also a way to disable consecutive dwell selections, and I’m guessing it’s so that there’s no disambiguation between wanting an return for accumulation, and wanting a repeat).

Repeat

Having a “repeat” button could help here, where you follow the “repeat” with a number (how many times you want to repeat), and then follow that with the button that you want to repeat).

Dwell animation

An example of the animation that takes place while dwelling could involve something like random blotches of paint covering more and more of the button so that the user can see that their intended letters are being sufficiently tagged.
Paint quickly disappears from letters as the user gazes across the next letters.

Conclusion

Variable dwell times could allow a user to move from button to button at a faster pace, as they’re not restricted to waiting for a set, bound time.
As long as the fixation time is reasonably long enough to show intention, it will work.

Counterargument

A potential issue with a batch submission: a user would have to get a good feel for estimating a batch size of letters that would give a good chance of surfacing the desired predicted word.

One thing that could be done is to increase the number of predicted words that are displayed until a person becomes more skillful at submitting the necessary letters.

(GazeTalk has a “Number of words suggested” preference).

Final example

I want to type "seventy".
I'm looking at a qwerty layout.

Pass by E on the way to S.
E 0.1s

S 0.8s

0.8 s is now the standard to signify interest.

Pass by F on the way to V
F 0.1s

V 1.1s

D 0.2s

E 0.4s

But it unintentionally breaks off to W for 0.2s.
0.3s is the set minimum time to allow a resumption of fixation (E 0.4s is enough).

E 0.4s + return for additional 0.7s = 1.1s

If unintended W meet then resumption requirement of 0.3 seconds, it could possibly build, and disrupt.
However, the E 1.1s is seen to belong to the other higher recorded times (sufficiently higher than the average so far:
E0.1 + S0.8 + F0.1 + V1.1 + D0.2 + E0.4 + W0.2 + E0.7
/
8 targets so far (2 E targets count for 1 E letter)

3.6s / 8 targets so far = 0.45).

Once a relatively longer time is logged, all other targets that could be resumed will not be able to resume, and they will be logged as is.

Pass by
F 0.1s
B 0.2s.

N 1.3 seconds.

You decide that SEVEN is enough to make SEVENTY surface in the autocomplete choices.
Fixate on a final "Submit" button to see the auto compete words that will appear.
The required fixation length here is hard set by the user.
It isn't flexible like the "targets of interest" time, which depends on all the times that are logged.

E0.1 + S0.8 + F0.1 + V1.1 + D0.2 + E0.4 + W0.2 + E0.7
F0.1 + B0.2 + N1.3

5.2s / 11 total targets = 0.47 average.

Add a certain percentage to the average to create a threshold that must be reached.

S 0.8s

E 0.7s

V 1.1s

E 0.4s + return for additional 0.7s 1.1s

N 1.3 seconds

Hopefully, they all make it though.

[JuliusSweetland]

Moving to BACKLOG

[alexandre-mbm]

• See automate fixation mode on Project IRIS