Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(web): osk inner-frame abstraction (help text vs std OSK) 🍕 #5430

Merged
merged 10 commits into from
Jul 23, 2021
21 changes: 15 additions & 6 deletions common/core/web/keyboard-processor/src/keyboards/keyboard.ts
Original file line number Diff line number Diff line change
Expand Up @@ -98,24 +98,33 @@ namespace com.keyman.keyboards {
}

/**
* HTML help text which is a one liner intended for the status bar of the desktop OSK originally.
* HTML help text, as specified by either the &kmw_helptext or &kmw_helpfile system stores.
*
* Reference: https://help.keyman.com/developer/language/reference/kmw_helptext
* Reference: https://help.keyman.com/developer/language/reference/kmw_helptext,
* https://help.keyman.com/developer/language/reference/kmw_helpfile
*/
get helpText(): string {
return this.scriptObject['KH'];
}

get hasHelpHTML(): boolean {
/**
* Embedded JS script designed for use with a keyboard's HTML help text. Always defined
* within the file referenced by &kmw_embedjs in a keyboard's source, though that file
* may also contain _other_ script definitions as well. (`KHF` must be explicitly defined
* within that file.)
*/
get hasScript(): boolean {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a comment, similar to the one above for helpText, referencing kmw_embedjs

return !!this.scriptObject['KHF'];
}

/**
* Replaces the OSK with custom HTML, which may be interactive (like with sil_euro_latin).
* Embeds a custom script for use by the OSK, which may be interactive (like with sil_euro_latin).
* Note: this must be called AFTER any contents of `helpText` have been inserted into the DOM.
* (See sil_euro_latin's source -> sil_euro_latin_js.txt)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR gave me a great opportunity to dive into sil_euro_latin and see how it really ticks.

The KHF function wraps the keyboard's embedded script.

https://github.com/keymanapp/keyboards/blob/1c9365083c10acb758289c5ffb140f5fdc3a4526/release/sil/sil_euro_latin/source/sil_euro_latin.kmn#L15

store(&KMW_EMBEDJS) 'sil_euro_latin_js.txt'

https://github.com/keymanapp/keyboards/blob/1c9365083c10acb758289c5ffb140f5fdc3a4526/release/sil/sil_euro_latin/source/sil_euro_latin_js.txt#L1-L4

this.KHF=function(e) { 
  var h = document.getElementsByTagName('head');
  var stext = 
  "#keyboard_europeanlatin_help { text-align: left; padding: 0; background: #f5e3de; font-size: 8pt; font-family: Tahoma; cursor: default; padding: 4px; }"+

Sadly, it seems that Github doesn't like to expand cross-repo permalinks. ☹️

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side-note: that KHF definition has some nasty stuff, some of which will make Shadow DOM tricky to perform for such a "help page" approach in the future.

Like, direct insertion of a style sheet to the element, as seen here.

It also blatantly ignores the provided e parameter, preferring to directly retrieve the one it wants via document.getElementById() on an element defined in its help-text section.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How could we chart a way forward to using shadow dom? What sort of changes would we need to make to keyboards that use kmw_embedjs? (sil_euro_latin, Chinese, Japanese, Korean, perhaps others).

Copy link
Contributor Author

@jahorton jahorton Jul 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chinese and Japanese are deprecated, .js-only keyboards so far as I can tell. Neither defines a KHF function. korean_rr is not deprecated, and while it does use kmw_embedjs, it doesn't actually use KHF either. (Instead, it defines the IME aspects that the Developer compiler can't provide.) Help text is available, but the keyboard doesn't render a "help page" like sil_euro_latin does.

tibetan_ewts_to_unicode (also in legacy) is the only other keyboard that defines a KHF.

My proposition:

We pass KHF a ShadowRoot (when possible) specific to the keyboard, rather than the OSK's root _Box.

  • If necessary, JS code can detect whether or not it received an actual shadow root: https://developer.mozilla.org/en-US/docs/Web/API/Node/getRootNode
    • Not available for old browsers / pre-Chromium Edge, but that in itself - the function's availability - gives a strong hint.
  • From there:
    1. Build <style> element
    2. If ShadowDOM available, insert to shadow root
    3. Else, insert to document (also needed for compat with older KMW versions)
    4. Rinse & repeat with other components.

If the keyboard is able to utilize Shadow DOM stuff, great! If not... I think we kind of have to support the old, non-shadowed functionality for now - both in keyboards and in the engine. I don't see any reliable way to shoe-horn non-shadow-aware JS into Shadow DOM stuff.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chinese and Japanese are deprecated

Legacy, not deprecated. Big difference.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shadow DOM discussion captured in #4881.

*
* Reference: https://help.keyman.com/developer/language/reference/kmw_helpfile
* Reference: https://help.keyman.com/developer/language/reference/kmw_embedjs
*/
insertHelpHTML(e: any) {
embedScript(e: any) {
// e: Expects the OSKManager's _Box element. We don't add type info here b/c it would
// reference the DOM.
this.scriptObject['KHF'](e);
Expand Down
2 changes: 1 addition & 1 deletion web/source/kmwbase.ts
Original file line number Diff line number Diff line change
Expand Up @@ -696,7 +696,7 @@ namespace com.keyman {

PKbd = PKbd || this.core.activeKeyboard;

return com.keyman.osk.VisualKeyboard.buildDocumentationKeyboard(PKbd, argFormFactor, argLayerId, this.osk);
return com.keyman.osk.VisualKeyboard.buildDocumentationKeyboard(PKbd, argFormFactor, argLayerId, this.osk.getKeyboardHeight());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Internal-only method; the external API that uses it remains unchanged.

}
}
}
Expand Down
6 changes: 5 additions & 1 deletion web/source/kmwembedded.ts
Original file line number Diff line number Diff line change
Expand Up @@ -255,7 +255,11 @@ namespace com.keyman.text {
* correctOSKTextSize handles rotation event -- currently rebuilds keyboard and adjusts font sizes
*/
keymanweb['correctOSKTextSize']=function() {
if(osk && osk.vkbd && osk.vkbd.adjustHeights(osk)) {
let osk = keymanweb.osk;
if(osk?.vkbd?.adjustHeights(osk.getKeyboardHeight())) {
var b: HTMLElement = osk._Box, bs=b.style;
bs.height=bs.maxHeight=osk.vkbd.computedAdjustedOskHeight(osk.getHeight())+'px';

osk._Load();
}
};
Expand Down
17 changes: 17 additions & 0 deletions web/source/osk/emptyView.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
/// <reference path="keyboardView.interface.ts" />

namespace com.keyman.osk {
export class EmptyView implements KeyboardView {
readonly element: HTMLDivElement;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly certain KMW currently keeps the OSK hidden whenever this is the active "inner frame" type. But... it does bother to construct it, so it gets its own proper class.


constructor() {
let Ldiv = this.element = document.createElement('div');
Ldiv.style.userSelect = 'none';
Ldiv.className='kmw-osk-none';
}

// No operations needed; this is a stand-in for the desktop OSK when no keyboard is active.
public postInsert() { }
public updateState() { }
}
}
34 changes: 34 additions & 0 deletions web/source/osk/helpPageView.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
/// <reference path="keyboardView.interface.ts" />

namespace com.keyman.osk {
export class HelpPageView implements KeyboardView {
private readonly kbd: keyboards.Keyboard;
public readonly element: HTMLDivElement;

private static readonly ID = 'kmw-osk-help-page';

constructor(keyboard: keyboards.Keyboard) {
this.kbd = keyboard;

var Ldiv = this.element = document.createElement('div');
Ldiv.style.userSelect = "none";
Ldiv.className = 'kmw-osk-static';
Ldiv.id = HelpPageView.ID;
Ldiv.innerHTML = keyboard.helpText;
}

public postInsert() {
if(!this.element.parentElement || !document.getElementById(HelpPageView.ID)) {
throw new Error("The HelpPage root element has not yet been inserted into the DOM.");
}

if(this.kbd.hasScript) {
// .parentElement: ensure this matches the _Box element from OSKManager / OSKView
// Not a hard requirement for any known keyboards, but is asserted by legacy docs.
this.kbd.embedScript(this.element.parentElement);
}
}

public updateState() { }
}
}
22 changes: 22 additions & 0 deletions web/source/osk/keyboardView.interface.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
namespace com.keyman.osk {
/**
* An abstract representation for visualizations of the active keyboard within an
* OSKManager / OSKView. Most keyboards will default to use of a VisualKeyboard,
* though some will use HelpPage for certain form factors.
*/
export interface KeyboardView {
readonly element: HTMLDivElement;

/**
* Evaluates code that must be run _after_ the KeyboardView has been inserted into
* the DOM hierarchy.
*/
postInsert(): void;

/**
* Code that updates the state of the KeyboardView whenever the OSK itself needs to be
* refreshed or updated with new state information.
*/
updateState(): void;
}
}
172 changes: 84 additions & 88 deletions web/source/osk/oskManager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@
/// <reference path="layouts/targetedFloatLayout.ts" />
// Generates the visual keyboard specific to each keyboard. (class="kmw-osk-inner-frame")
/// <reference path="visualKeyboard.ts" />
// Models keyboards that present a help page, rather than a standard OSK.
/// <reference path="helpPageView.ts" />
/// <reference path="emptyView.ts" />

/***
KeymanWeb 10.0
Expand Down Expand Up @@ -142,11 +145,6 @@ namespace com.keyman.osk {

this.loadRetry = 0;

if(this.desktopLayout) {
this.desktopLayout.titleBar.setTitle('KeymanWeb'); // I1972
}


this._Visible = false; // I3363 (Build 301)
var s = this._Box.style;
s.zIndex='9999'; s.display='none'; s.width= device.touchable ? '100%' : 'auto';
Expand Down Expand Up @@ -191,69 +189,54 @@ namespace com.keyman.osk {
this._Box.onmouseover = this._VKbdMouseOver;
this._Box.onmouseout = this._VKbdMouseOut;

// TODO: find out and document why this should not be done for touch devices!!
// (Probably to avoid having a null keyboard. But maybe that *is* an option, if there remains a way to get the language menu,
// such as a minimized menu button?)
if(activeKeyboard == null && !device.touchable) {
const layout = this.desktopLayout = new layouts.TargetedFloatLayout();
// START: construction of the actual internal layout for the overall OSK
let layout: layouts.TargetedFloatLayout = null;

// Add header element to OSK only for desktop browsers
if(util.device.formFactor == 'desktop') {
layout = this.desktopLayout = new layouts.TargetedFloatLayout();
layout.attachToView(this);
this.desktopLayout.titleBar.setTitleFromKeyboard(activeKeyboard);
this._Box.appendChild(layout.titleBar.element);
}

Ldiv = util._CreateElement('div');
Ldiv.className='kmw-osk-none';
this._Box.appendChild(Ldiv);
} else {
var Lhelp='';
this._Box.className = "";
if(activeKeyboard != null) {
// Note: must exist in order for insertHelpHTML to be used!
Lhelp=activeKeyboard.helpText;
}
// Add suggestion banner bar to OSK
if (this.banner) {
this._Box.appendChild(this.banner.element);
}

// Generate a visual keyboard from the layout (or layout default)
// Condition is false if no key definitions exist, formFactor == desktop, AND help text exists. All three.
if(activeKeyboard && activeKeyboard.layout(device.formFactor as utils.FormFactor)) {
this._GenerateVisualKeyboard(activeKeyboard);
} else if(!activeKeyboard) {
this._GenerateVisualKeyboard(null);
} else { //The following code applies only to preformatted 'help' such as SIL EuroLatin
//osk.ddOSK = false;
const layout = this.desktopLayout = new layouts.TargetedFloatLayout();
layout.attachToView(this);
this._Box.appendChild(layout.titleBar.element);
this._Box.appendChild(this.banner.element);

//Add content
var Ldiv = util._CreateElement('div');
Ldiv.className='kmw-osk-static';
Ldiv.innerHTML = Lhelp;
this._Box.appendChild(Ldiv);
if(activeKeyboard.hasHelpHTML) {
activeKeyboard.insertHelpHTML(this._Box);
}
}
let kbdView: KeyboardView = this._GenerateKeyboardView(activeKeyboard);
this._Box.appendChild(kbdView.element);
if(kbdView instanceof VisualKeyboard) {
this.vkbd = kbdView;
}
kbdView.postInsert();

if(this.desktopLayout) {
this.desktopLayout.titleBar.setTitleFromKeyboard(activeKeyboard);
// Add footer element to OSK only for desktop browsers
if(this.desktopLayout) {
if(kbdView instanceof VisualKeyboard) {
this._Box.appendChild(layout.resizeBar.element);
}
// For other devices, adjust the object heights, allowing for viewport scaling
} else {
this.vkbd.adjustHeights(this.getKeyboardHeight());

let b: HTMLElement = this._Box, bs=b.style;
bs.height=bs.maxHeight=this.vkbd.computedAdjustedOskHeight(this.getHeight())+'px';
}

// END: construction of the actual internal layout for the overall OSK

// Correct the classname for the (inner) OSK frame (Build 360)
var kbdID: string = (activeKeyboard ? activeKeyboard.id.replace('Keyboard_','') : '');
if(keymanweb.isEmbedded && kbdID.indexOf('::') != -1) {
// De-namespaces the ID for use with CSS classes.
// Assumes that keyboard IDs may not contain the ':' symbol.
kbdID = kbdID.substring(kbdID.indexOf('::') + 2);
}
var innerFrame=<HTMLDivElement> this._Box.firstChild,
kbdClass = ' kmw-keyboard-' + kbdID;
if(innerFrame.id == 'keymanweb_title_bar') {
// Desktop order is title_bar, banner_container, inner-frame
innerFrame=<HTMLDivElement> innerFrame.nextSibling.nextSibling;
} else if (innerFrame.id == 'keymanweb_banner_container') {
innerFrame=<HTMLDivElement> innerFrame.nextSibling;
}
innerFrame.className = 'kmw-osk-inner-frame' + kbdClass;

const kbdClassSuffix = ' kmw-keyboard-' + kbdID;
kbdView.element.className = kbdView.element.className + kbdClassSuffix;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kbdView.element is equivalent to the old innerFrame - but this time, we don't need to search _Box's hierarchy to find it.


this.banner.appendStyles();

Expand Down Expand Up @@ -300,54 +283,58 @@ namespace com.keyman.osk {
}
}.bind(this);

private _GenerateKeyboardView(keyboard: keyboards.Keyboard): KeyboardView {
let device = com.keyman.singleton.util.device;

if(this.vkbd) {
this.vkbd.shutdown();
}

this._Box.className = "";

// Case 1: since we hide the system keyboard on touch devices, we need
// to display SOMETHING that can accept input.
if(keyboard == null && !device.touchable) {
// We do not (currently) allow selecting the default system keyboard on
// touch form-factors. Likely b/c mnemonic difficulties.
return new EmptyView();
} else {
// Generate a visual keyboard from the layout (or layout default)
// Condition is false if no key definitions exist, formFactor == desktop, AND help text exists. All three.
if(keyboard && keyboard.layout(device.formFactor as utils.FormFactor)) {
return this._GenerateVisualKeyboard(keyboard);
} else if(!keyboard /* && device.touchable (implied) */) {
// Show a basic, "hollow" OSK that at least allows input, since we're
// on a touch device and hiding the system keyboard
return this._GenerateVisualKeyboard(null);
} else {
// A keyboard help-page or help-text is still a visualization, even not a standard OSK.
return new HelpPageView(keyboard);
}
}
}

/**
* Function _GenerateVisualKeyboard
* Scope Private
* @param {Object} PVK Visual keyboard name
* @param {Object} Lhelp true if OSK defined for this keyboard
* @param {Object} layout0
* @param {Number} kbdBitmask Keyboard modifier bitmask
* @param {Object} keyboard The keyboard to visualize
* Description Generates the visual keyboard element and attaches it to KMW
*/
private _GenerateVisualKeyboard(keyboard: keyboards.Keyboard) {
if(this.vkbd) {
this.vkbd.shutdown();
}
private _GenerateVisualKeyboard(keyboard: keyboards.Keyboard): VisualKeyboard {
let device = com.keyman.singleton.util.device;

let util = com.keyman.singleton.util;
this.vkbd = new VisualKeyboard(keyboard, util.device);
// Root element sets its own classes, one of which is 'kmw-osk-inner-frame'.
let vkbd = new VisualKeyboard(keyboard, device);

// Ensure the OSK's current layer is kept up to date.
let core = com.keyman.singleton.core; // Note: will eventually be a class field.
core.keyboardProcessor.layerStore.handler = this.layerChangeHandler;

// Set box class - OS and keyboard added for Build 360
this._Box.className=util.device.formFactor+' '+ util.device.OS.toLowerCase() + ' kmw-osk-frame';

let layout: layouts.TargetedFloatLayout = null;

// Add header element to OSK only for desktop browsers
if(util.device.formFactor == 'desktop') {
layout = this.desktopLayout = new layouts.TargetedFloatLayout();
layout.attachToView(this);
this._Box.appendChild(layout.titleBar.element);
}

// Add suggestion banner bar to OSK
if (this.banner) {
this._Box.appendChild(this.banner.element);
}
this._Box.className=device.formFactor+' '+ device.OS.toLowerCase() + ' kmw-osk-frame';

// Add primary keyboard element to OSK
this._Box.appendChild(this.vkbd.kbdDiv);

// Add footer element to OSK only for desktop browsers
if(layout) {
this._Box.appendChild(layout.resizeBar.element);
// For other devices, adjust the object heights, allowing for viewport scaling
} else {
this.vkbd.adjustHeights(this);
}
return vkbd;
}

/**
Expand Down Expand Up @@ -822,8 +809,17 @@ namespace com.keyman.osk {
// TODO: Move this into the VisualKeyboard class!
// The following code will always be executed except for externally created OSK such as EuroLatin
if(this.vkbd && this.vkbd.ddOSK) {
// Always adjust screen height if iPhone or iPod, to take account of viewport changes
// Do NOT condition upon form-factor; this line prevents a bug with displaying
// the predictive-text banner on the initial keyboard load. (Issue #2907)
if(device.touchable && device.OS == 'iOS') {
this.vkbd.adjustHeights(this.getKeyboardHeight());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This particular call (adjustHeights) is no longer included in VisualKeyboard's show / updateState method.


var b: HTMLElement = this._Box, bs=b.style;
bs.height=bs.maxHeight=this.vkbd.computedAdjustedOskHeight(this.getHeight())+'px';
}
// Enable the currently active keyboard layer and update the default nextLayer member
this.vkbd.show(this);
this.vkbd.updateState();

// Extra style changes and overrides for touch-mode.
if(device.touchable) {
Expand Down
Loading