spec(developer,web): support "hint" property for touch layouts 🐵

[mcdurdin]

Introduction

This is needed in order to support the LDML spec, ref #5078.

We need to be able to specify a "hint" property for touch layouts. This hint would ideally be a single character, printed quite small, on the top right of the key.

A default hint source can be specified for each platform, from any of key's sub-elements - longpress, flick or multitap. This default can be overridden on a key-by-key basis, allowing for any text to be specified for any key. To hide a default hint, the author can specify a single space for the key hint.

The hint is specified as an additional property of a key in the .keyman-touch-layout file, and both the hint and defaultHint properties will be editable from the Keyman Developer touch layout editor.

The compiler will use the defaultHint data to insert the hint data onto each key cap in the compiled touch layout file, so KeymanWeb does not need to calculate this. However, KeymanWeb will need to check the defaultHint value to determine whether or not to show the "." hint to match functionality in earlier versions of Keyman.

KeymanWeb must present the hint on touch layouts.

For desktop-style On Screen Keyboards, there may also be a need for additional hints. At present the .keyman-touch-layout format does not fully support desktop, but this should be considered as part of this design, as we aim to deprecate .kvk and .kvks in future versions of Keyman.

There is conceptual overlap between the "underlying key" and the "hint" feature, but the source of data for the key cap text in each case is different. The "underlying key" may be displayed as well as the "hint", if the keyboard author decides to do so. (The "underlying key" data is algorithmically determined, from the "underlying keyboard"; currently US English is supported.)

When we introduce theming, the user will have the ability to decide which components to hide and/or move on a keycap.

This proposal does not need to support the concept of multiple hints for a given key. While there have been some requests in the past for extra letters on desktop OSK key caps, these all relate to alternate modifiers and can be found algorithmically.

The keyboard author can also decide whether or not to display the default "." hint which is currently shown when a long-press menu is visible, by selecting the default hint source of "dot".

C5079.1 File Formats: .keyman-touch-layout changes

1. Add a hint property to the key object. If present, this must be a string.
2. Add a defaultHint property to the platform object (alongside displayUnderlying). If present, this must be a string. Default is "dot".

• "none": show no hint on the key cap.
• "dot": show a dot on top-right of key cap if a longpress menu is
  available, default. Matches behavior for Keyman 15.0 and earlier.
• "longpress": show the default longpress key cap, if it exists.
• "multitap": show the first multitap key cap, if it exists.
• "flick": show the first flick key cap, selected in clockwise order,
  starting from North.
• "flick-n": show the key cap from the North flick, if it exists.
• "flick-ne": show the key cap from the North-East flick, if it exists.
• "flick-e": show the key cap from the East flick, if it exists.
• "flick-se": show the key cap from the South-East flick, if it exists.
• "flick-s": show the key cap from the South flick, if it exists.
• "flick-sw": show the key cap from the South-West flick, if it exists.
• "flick-w": show the key cap from the West flick, if it exists.
• "flick-nw": show the key cap from the North-West flick, if it exists.

Backward compatibility

Touch layout files that contain this feature will be backwardly compatible with earlier versions of Keyman, with a graceful degradation of functionality.

• The hint and defaultHint properties will be ignored by older versions of KeymanWeb.
• However, older versions of Keyman Developer will not load or compile files that contain these properties, because the files will be treated as invalid.

C5079.2 Changes to Developer

• Required: The touch layout editor needs to surface the hint property in the key cap properties.
• Required: The touch layout editor needs to surface the defaultHint property in the platform properties.
• Required: The compiler will pre-process the .keyman-touch-layout source to add hint data to each key from all sources except "dot". If the defaultHint value is "dot", then remove defaultHint property at compile time. This simplifies the logic within Keyman Engine for Web to a simple check for the presence of defaultHint.

C5079.3 Changes to Keyman Engine for Web

• Required: Draw the extra hint text onto the key cap.
  • If the hint property is present for a key, draw the text from the hint property using the same font as the key text (size to be determined).
  • Otherwise, check defaultHint for the platform: if it is present, draw nothing for the hint text -- because the hint property has been calculated already by the compiler.
  • Otherwise, as defaultHint is missing, use the Keyman 15 model (drawing a "dot" hint if the key includes a long-press).
• For 16.0, the hint should be drawn at top-right of the key.

Future:

• When we introduce theming (scheduled for 17.0), the theme will determine position and/or visibility (see feat(android/ios): user option for toggling OSK underlying characters on/off #4894) of the text, giving the end user additional control.

Related issues and discussions

• spec(developer): LDML to keyman-touch-layout tool 🙀 #5078
• feat(web): Display longpress preview on touch devices #2963
• feat(android/ios): user option for toggling OSK underlying characters on/off #4894
• feat(developer): underlying character customizable? #4879
• feat(Android/iOS): a toggle setting to show and unshow underlying characters on the OSK on mobile #6921
• https://community.software.sil.org/t/custom-display-underlying/5618
• https://community.software.sil.org/t/yiddish-keyboard-on-android/6416/5

[jahorton]

Given that LDML has been pushed back to 16.0, we can probably triage this to Future.

[mcdurdin]

See also #2963.

[rc-swag]

See also #4894 which has the request to enable/disable hints

[mcdurdin]

See also #4879.

[jahorton]

See also #4879.

Isn't that for something completely different, though? That one's more about the underlying key cap, as opposed to signalling the existence of longpress keys or the like.

[mcdurdin]

See also #4879.

Isn't that for something completely different, though? That one's more about the underlying key cap, as opposed to signalling the existence of longpress keys or the like.

If you read the description, they're really asking for something akin to hints, rather than the underlying layout. The meaning of the hint on a key could be dependent on the keyboard. No reason why you couldn't use hints in the way they are asking (IPA hints on Thai keyboard).

[jahorton]

Isn't that for something completely different, though? That one's more about the underlying key cap, as opposed to signalling the existence of longpress keys or the like.

If you read the description, they're really asking for something akin to hints, rather than the underlying layout. The meaning of the hint on a key could be dependent on the keyboard. No reason why you couldn't use hints in the way they are asking (IPA hints on Thai keyboard).

That's true for the November 26 comments, but not so much the comments before that, so far as I can tell. This is reasonable to do for those Nov 26 ones, at least.

As for the "IPA hints on Thai keyboard" idea... you're suggesting we could allow hints on keys without longpresses? That certainly seems to be implied here, but I want that to be explicitly stated if so. And if I'm inferring correctly, this is the first I've seen of that sort of suggestion... and I worry that this may cause confusion when looking at other keyboards that use it to signal longpresses.

[mcdurdin]

From #4879:

A user asked whether or not the underlying characters on touch layout and OSK be modified. If possible, the intention is to use IPA characters instead of the English characters because it may be useful to users to learn to type.

So while they looked at an existing property of the key cap (underlying key), and wanted to change that property, it actually makes more sense to look at implementing this as a hint, rather than changing the meaning and design of that existing property. I'm taking their feature request and responding with a slightly different solution.

you're suggesting we could allow hints on keys without longpresses

The hint is not tied to the longpress (that was just one way a hint could be used). The hint is an arbitrary string that can assigned to a key, just as the key cap is an arbitrary string as well -- neither have any functional meaning and it is up to the keyboard developer to use them appropriately. We may include tooling in Keyman Developer to fill in the hint from longpress (that would be useful!) and guidance in the documentation on this, but the intent was never to hard-code the link between hint and longpress.

This clause from the first paragraph in the spec may have been confusing:

that would override the default dot that appears on the top right when a key has a longpress

So I have updated that sentence and hopefully it is clearer!

[MayuraVerma]

Is it possible to implement hint for desktop app also

[mcdurdin]

Is it possible to implement hint for desktop app also

We are planning to consolidate the OSK views in 18.0 or 19.0 -- to be determined -- at which point, functionality like hints will work there as well.

[jahorton]

Already done on the feature-gestures branch; going ahead and closing this one.