Creating and Using Dynamic Keyboards

About Dynamic Keyboards

In addition to the many built-in keyboard layouts, Optikey supports the ability to define your own keyboards. These keyboards are defined in an XML source file and offer unique features as well as support for most, but not all, of the functionality available in the built-in keyboards.

The easiest way to get started is to modify one of the sample keyboard files found in your Optikey installation folder Resources\DynamicKeyboards. The full path may vary depending on your version of Windows and choices you made during installation, but will generally be something like C:\Program Files (x86)\Optikey\Resources\DynamicKeyboards. You can also view available keyboard files inside of the OptiKey repository: Dynamic Keyboards on github.

Enabling Dynamic Keyboards at runtime

In order to enable a dynamic keyboard, we need to either choose the Custom Keyboard option as our startup keyboard and specify the path to the keyboard XML file or choose the list of Dynamic Keyboards as our startup keyboard and specify a directory where our custom XML keyboard files can be installed (literally copied). The configuration panel (or Management Console / Settings) can be accessed by right-clicking on the OptiKey keyboard and selecting the first option from the resulting popup menu. An example of where to locate these startup keyboard settings in the console is shown below:

If you have followed these instructions and the keyboard is too large or too small for your screen, then you may need to adjust the Height value in the XML file (see section "Skeleton of the XML Keyboard Files" below). Since the meaning of Height varies depending on screen resolution, a value that works for one user may not work for another.

We are now ready to write software and hack source code with OptiKey!

Writing your own Dynamic Keyboard

A new tool is available which allows you to visually design a dynamic keyboard: Click here to launch the dynamic keyboard design tool in your web browser

Please note that this is currently under development and may not reflect the complete set of features available in Optikey's dynamic keyboards.

Skeleton of the XML Keyboard Files

In this section we will explore the necessary components of the dynamic keyboard XML file so users can write and/or edit their own variations of the provided examples.

Keyboard Header

First, using the Demo_Keyboard as our model, we have the following options in the preamble of the file:

<Keyboard>
    <Name>Demo_Keyboard</Name>
    <HideFromKeyboardMenu>False</HideFromKeyboardMenu>
    <PersistNewState></PersistNewState>  
    <WindowState>Floating</WindowState>
    <DockSize>Full</DockSize>
    <Position>BottomLeft</Position> 
    <Width>90%</Width>
    <Height>40%</Height>  
    <HorizontalOffset></HorizontalOffset>
    <VerticalOffset>-80</VerticalOffset>
    <BackgroundColor>Transparent</BackgroundColor>
    <BorderColor>Transparent</BorderColor>
    <BorderThickness>1,1,1,1</BorderThickness>
    <SymbolMargin>4</SymbolMargin>
    <Grid>
        <Rows>6</Rows>
        <Cols>12</Cols>
    </Grid>
    <KeyGroup Name="DefaultGroup" LockOnTime="100" CompletionTimes="900,300" KeyDisabledOpacity=".5" />
    <KeyGroup Name="SlowKeys" LockOnTime="100" CompletionTimes="1500,750" KeyDisabledOpacity=".5" />
    <KeyGroup Name="FastKeys" LockOnTime="100" CompletionTimes="400,800,200" KeyDisabledOpacity=".5" />
    <KeyGroup Name="GrayKeys" BackgroundColor="gray" KeyDisabledBackground="gray" KeyDownBackground="blue" />
    <Content>
    <!-- ... Individual Key Definitions Go Here ... -->
    </Content>
</Keyboard>

HideFromKeyboardMenu - Controls whether or not this keyboard will be listed in the dynamic keyboard selection menu. You may wish to set this to False for smaller utility-type keyboards.
PersistNewState - (true or false) indiates whether the size and position of this keyboard will be saved
WindowState - (Floating, Docked, Maximised) if not declared the previous value or saved default will be used
DockSize - (Full or Collapsed) if not declared the previous value or saved default will be used
Position - (TopLeft, Top, TopRight, Left, Right, BottomLeft, Bottom, BottomRight) if not declared the previous value or saved default will be used
Width and Height - (-9999 to 9999) value is in pixels unless % is used in which case the value is a percent of screen. If negative then use the sum of the value and the screen's dimension i.e. if a Width of -300 is sent and the screen is 1920 wide then the keyboard width will be set to 1620 by way of ((-300) + (1920) = (1620))

Note that the dimensions are specified as a function of your screen size and resolution. Even when using an existing keyboard, you may need to adjust the Width, Height and offset values to find a size that works well for your screen. Similarly, if using a different screen resolution for any reason, the values may need to be manually adjusted.
Horizontal and Vertical Offset - (-9999 to 9999) value is in pixels unless % is used in which case the value is a percent of the screen. If negative then offset left or up
BackgroundColor and BorderColor - for valid colors refer to https://docs.microsoft.com/en-us/dotnet/api/system.drawing.color
KeyGroup - A collection of attributes that can contribute to the appearance and behavior of keys. Keys are not limited to any one group, rather they can belong to any combination of groups. Every key is automatically included as a member in the ALL group. Each KeyGroup may have the following optional attributes:
- BackgroundColor
- KeyDisabledBackground
- KeyDownBackground
- ForegroundColor
- KeyDisabledForeground
- KeyDownForeground
- Opacity - A numeric value between 0 and 1 where 0 is transparent and 1 is fully opaque
- KeyDisabledOpacity
- KeyDownOpacity
- AutoScaleToOneKeyWidth - Only required to modify scaling behavior when key width is greater than 1, otherwise it will default to "true"
- AutoScaleToOneKeyHeight - Only required to modify scaling behavior when key height is greater than 1, otherwise it will default to "true"
- SharedSizeGroup - Only required to break out a key, or set of keys, to size separately, otherwise size grouping is determined automatically
- UsePersianCompatibilityFont
- UseUnicodeCompatibilityFont
- UseUrduCompatibilityFont
- LockOnTime - Milliseconds from when the gaze is detected on a key to the start of the progress animation
- CompletionTimes - Milliseconds from the start of the progress animation to the moment the key is triggered. It is possible to set different timespans for each consecutive press of the same key by using a comma-delimited list. (i.e. CompletionTimes="1000,100,200")

Example of Continuous Gaze on Key "x"

Keyboard Body

Content:

There are four available content types: DynamicKey, Scratchpad, SuggestionRow, and SuggestionCol. The first type, DynamicKey, is host to a plethora of functionality to be explained in detail later, whereas the latter three are basic single-purpose items. A scratchpad is the visual display of whatever string of text has been typed and can interact with other keys to be copied, spoken, or sent to a plugin. A suggestion row or column is a collection of word predictions. Content items may have the following optional attributes:

Row - if not declared it will be set by the positioning algorithm
Col - if not declared it will be set by the positioning algorithm
Width - if not declared it will default to 1
Height - if not declared it will default to 1
Any of the attributes of a KeyGroup may be declared here, but not all are actually applied. If declared here they will act as override values for this item.

Example:

<Scratchpad Row="0" Col="0" Width="8" Height="1" BackgroundColor="black" Opacity="1" KeyDownBackground="gray" KeyDownOpacity="1" />

Positioning Algorithm:

Position all content items that have both row and col defined
Position all remaining items in listed order
If an item has a row or column defined jump forward to that row or column and mark any/all skipped spaces as empty

DynamicKey

DynamicKey may have any of the following optional attributes:

Row - if not declared it will be set by the positioning algorithm
Col - if not declared it will be set by the positioning algorithm
Width - if not declared it will default to 1
Height - if not declared it will default to 1
Any of the attributes of a KeyGroup may be declared here and will act as override values for this key.

DynamicKey may have 0 or 1 occurrence of the following visual elements:

Label - this value becomes the text value on the created Key.
ShiftDownLabel - only required to display an alternate text value when the shift key is down
Symbol - this is the image on the key

DynamicKey may have 0, 1 or many occurrences of the KeyGroup element:

In a DynamicKey the KeyGroup element serves two independent purposes: first, it allows key attributes to be specified and shared, and second, it allows keys to be referenced by KeyUp commands.
- When declared in the heading the attributes affect the appearance and behavior of keys.
- When only found within individual key definitions it only serves as a token to identify the key, or keys when used by more than one.

DynamicKey may have 0, 1 or many occurrences of the following command elements:

Action - an action key
ChangeKeyboard - a built-in or dynamic keyboard
KeyDown - a character or action key to press and not release
KeyToggle - a character or action key change state
KeyUp - a character, action key, or KeyGroup to release. Any down keys having a matching KeyGroup will be released
Loop - a wrapper used to enclose a set of commands which are to be performed repetitively
Text - a character or string of characters to type
Wait - time to wait in milliseconds before executing the next command

ChangeKeyboard commands may have the following optional attribute:

BackReturnsHere - if "true" the back key on the destination keyboard will return here. Default to "true"

Loop commands replicate DynamicKey functionality and may contain all the same command elements: Action, ChangeKeyboard, KeyDown, KeyToggle, KeyUp, Loop, Text and Wait. In addition, the following rules apply to Loop commands:

Count - the number of executions with 1 being the default and 0 acting as a signal to lock down the key and loop perpetually until the key is triggered to release
Wait - if a wait is not declared somewhere within the loop then a default delay will be applied to throttle the loop

Example:

<DynamicKey>
	<Label>Space Loop</Label>
	<Symbol>SpaceIcon</Symbol>
	<Loop Count="0">
		<Text>&#32;</Text>
		<Wait>500</Wait>
	</Loop>
</DynamicKey>

Text Commands

A Text command is a command that is associated with a text character or string of text which is generated when the Optikey application signals that the key has been pressed. Notice that the Text field, which signifies the text generated by Optikey on keypress, can be a string or a specially reserved HTML entity such as > or &#xx;where xx is the ASCII code of the character. The Label contains the text to be displayed on the key. If different text should be displayed when the shift key is held down, then ShiftDownLabel should be specified as well.

In place of a Label, you may instead specify a Symbol to be displayed on the key. If both a Symbol and a Label are specified for the same key, then the default behavior is to display the symbol until the user looks at the key. While the gaze remains on the key, the label will be displayed instead. A list of keywords that can be used to specify the Symbol icons are found in this source file. Common other icon keywords include: TabIcon, EnterIcon, BackOneIcon, LeftArrowKeyIcon, RightArrowKeyIcon, DownArrowKeyIcon, UpArrowKeyIcon, and MenuIcon, among others found in the default Optikey keyboards.

<DynamicKey Row="4" Col="0">
     <Text>a</Text>
     <ShiftDownLabel>A</ShiftDownLabel>
     <Label>a</Label>a
</DynamicKey>
<DynamicKey Row="6" Col="5" Width="2">
     <Symbol>SpaceIcon</Symbol>
     <Text>&#32;</Text>
</DynamicKey>
<DynamicKey Row="8" Col="9">
     <Label>FPRINTF</Label>
     <Text>fprintf</Text>
</DynamicKey>

Action Commands

An Action command associates a specific action with a keypress. Valid actions are listed in this source file. Common actions include: ArrowUp/Down/Left/Right, Escape, LeftCtrl, LeftAlt, LeftShift, BackOne, MenuKeyboard, and LeftWin. Some example Action entries are given below:

<DynamicKey Row="5" Col="10">
     <Action>ArrowUp</Action>
     <Symbol>UpArrowKeyIcon</Symbol>
</DynamicKey>
<DynamicKey Row="6" Col="0">
     <Action>LeftCtrl</Action>
     <Label>Ctrl</Label>
</DynamicKey>

To get to the built-in menu which provides a list of all Dynamic / Custom Keyboards that can be loaded at runtime we use the Action titled DynamicKeyboard:

<DynamicKey>
     <Action>DynamicKeyboard</Action>
     <Symbol>KeyboardIcon</Symbol>
</DynamicKey>

When you press on this key, it brings up a keyboard menu resembling the following screenshot output of our installed Optikey keyboard selections:

Currently once you exit the Dynamic / Custom Keyboards selection in favor of one of the built-in Optikey keyboards, i.e., navigating via a custom key with the one of the Actions Alpha1Keyboard, Alpha2Keyboard, CommuniKateKeyboard, ConversationAlpha1Keyboard, ConversationAlpha2Keyboard, ConversationCommuniKateKeyboard, ConversationNumericAndSymbolsKeyboard, Currencies1Keyboard, Currencies2Keyboard, Diacritic1Keyboard, Diacritic2Keyboard, Diacritic3Keyboard, LanguageKeyboard, MouseKeyboard, NumericAndSymbols1Keyboard, NumericAndSymbols2Keyboard, NumericAndSymbols3Keyboard, PhysicalKeysKeyboard, SizeAndPositionKeyboard, or WebBrowsingKeyboard, be cautious in that you may not be able to get back to the selection of Dynamic / Custom Keyboard layouts without a restart of Optikey. The core Optikey developers are aware of this problem and are working on a sane way to fix it for all users -- including those who do not use the custom XML keyboard layouts at all during runtime of the application.

To get to the main keyboard menu, you can also include the following key specification in your custom keyboard:

<DynamicKey>
     <Action>MenuKeyboard</Action>
     <Symbol>KeyboardIcon</Symbol>
</DynamicKey>

After pressing this key, a screen resembling the following keyboard will be displayed:

ChangeKeyboard Commands

These keys are navigational entries used within the Optikey layout to change between different dynamically loaded keyboards. It happens that users can navigate to the stock, hardcoded Optikey keyboards such as Alpha1 or Currency1 after loading a dynamic keyboard at startup by specifying an appropriate ActionKey from the FunctionKeys enum in the Optikey source linked in the last section. On the other hand, ChangeKeyboardKeys are used to navigate directly to another XML keyboard in the directory setup in the configurations console (Settings). If we have another XML keyboard file named HackerKeyboardSymbols2.xml in our keyboards directory, then the following key specification will allow us to load it on-the-fly from our first keyboard:

<DynamicKey> 
     <Symbol>NumericAndSymbolsIcon</Symbol>
     <ChangeKeyboard BackReturnsHere="True">HackerKeyboardSymbols2</ChangeKeyboard>
</DynamicKey>

The BackReturnsHere determines the behavior of the back button (when the destination keyboard contains a back key). If set to True, then pressing the back key on the destination keyboard will bring the user back to this keyboard. Otherwise, the current keyboard will be skipped, and the back button will return the user to the keyboard accessed before this one. The False option is useful when constructing multiple keyboards with large keys that are meant to function conceptually as a single keyboard. In this use case, arrows at the edges of the keyboard allow the user to "scroll" to the next block of keys. This approach is used frequently in the EyeMine project.

Wait Commands

These key commands allow pauses between other commands to slow down execution and create custom timings.

<DynamicKey> 
	<Label>Doe, a deer</Label>
	<Text>Doe, </Text>
	<Wait>500</Wait>
	<Text>a deer, </Text>
	<Wait>500</Wait>
	<Text>a female deer</Text>
</DynamicKey>

Key Down/Toggle/Up Commands

These key commands are intended for special use cases where there may be a desire to lock down a key. This is often important if you want to send key presses to a game, for instance. In the first example below, OptiKey would press and hold the "a" key for one-half second and then release it. In the second example OptiKey would press and hold the "a" key indefinitely until the user performs a secondary activation to manually toggle the key back to up.

<DynamicKey Row="1" Col="12" Width="2" Height="2" BackgroundColor="blue" KeyDownBackground="gray" Opacity=".6" KeyDownOpacity=".9">
	<Label>A burst</Label>
	<KeyDown>a</KeyDown>
	<Wait>500</Wait>
	<KeyUp>a</KeyUp>
</DynamicKey>

<DynamicKey Row="3" Col="11" Width="2" Height="2" BackgroundColor="green" KeyDownBackground="gray" Opacity=".6" KeyDownOpacity=".9">
	<Label>A</Label>
	<KeyToggle>a</KeyToggle>
</DynamicKey>

Special keys

The key name for a KeyUp / KeyDown / KeyPress can be one of the following (interpreted in this order)

A single character, such as <KeyUp>a</KeyUp>
The name of a builtin FunctionKey with associated VirtualKeyCode from this list: FunctionKeysExtensions.cs such as <KeyUp>LeftShift</KeyUp>
The (case-insensitive) name of any key from InputSimulator's VirtualKeyCodes, for instance <KeyUp>SNAPSHOT</KeyUp> or <KeyUp>Snapshot</KeyUp>

Loop Commands

These key commands provide the ability to have OptiKey perform a set of commands for a defined count, or indefinitely until manually stopped by a secondary activation of the key.

The first example is a simple three burst loop, and the second example shows loops nested within loops. In the third example, the loop Count is set to zero which means it will loop indefinitely. The spacebar will be pressed repeatedly until stopped by the user. Because there is no Wait defined the application will automatically impose a wait to throttle the loop.

<DynamicKey Row="1" Col="12" Width="2" Height="2" BackgroundColor="blue" KeyDownBackground="gray" Opacity=".6" KeyDownOpacity=".9">
	<Label>A burst</Label>
	<Loop Count="3">
		<KeyDown>a</KeyDown>
		<Wait>100</Wait>
		<KeyUp>a</KeyUp>
	</Loop>
</DynamicKey>

<DynamicKey Row="3" Col="7" Width="2" Height="2" BackgroundColor="black" KeyDownBackground="gray" Opacity=".6" KeyDownOpacity=".9">
	<Label>Nested Loops</Label>
	<Text>beforeloop&#13;</Text>
	<Loop Count="1">
		<Text>loop once&#13;</Text>
		<Loop Count="2">
			<Text>this loops twice&#13;</Text>
			<Loop Count="3">
				<Wait>1000</Wait>
				<Text> 3</Text>
			</Loop>
			<Text>&#13;still in loop twice&#13;</Text>
		</Loop>
		<Text>after loop twice&#13;</Text>
	</Loop>
	<Text>afterloop</Text>
</DynamicKey>

<DynamicKey Row="1" Col="7" Width="2" Height="2" BackgroundColor="black" KeyDownBackground="gray" Opacity=".6" KeyDownOpacity=".9">
	<Label>Space Loop</Label>
	<Symbol>SpaceIcon</Symbol>
	<Loop Count="0">
		<Text>&#32;</Text>
	</Loop>
</DynamicKey>

Custom Key Styling

In addition to the keyboard background and border colors, you may (optionally) also specify color and opacity for any Key in your XML keyboard. If specified this will override the base Optikey theme for that key only. Custom values are allowed for: BackgroundColor, ForegroundColor, KeyDisabledBackground, KeyDownBackground, KeyDownOpacity, and Opacity

For valid colors refer to https://docs.microsoft.com/en-us/dotnet/api/system.drawing.color

Custom Key Sizing

Default Optikey behaviour

Optikey attempts to automatically size and render the font and symbols of all keys by organising the keys in "groups" which are scaled the same amount. Optikey groups keys in the following ways:

All keys with symbols (so that the symbols across all keys are roughly the same size).
All keys with a single letter/character on them (e.g. "F")
All keys with a short string of characters on them (e.g. "ALT").
All keys with a long string of text (e.g. "Some long text").

Some built-in keyboards also use other predefined "shared size groups" to manually group keys together.

Once keys have been assigned a group, the text and symbols of all keys in this group are scaled to fit the width and height of the smallest key in this group.

Overriding key sizing in dynamic keyboards

If the default sizing isn't to your liking, you can define separate size groups on keys by specifying the SharedSizeGroup attribute explicitly on each key. With this property you can group keys that you want sized together by using the same SharedSizeGroup on multiple keys. The value of the SharedSizeGroup attribute is any string of your choice, for instance:

<DynamicKey SharedSizeGroup="MyNewSizeGroup">
     <Label>Back</Label>      
     <Action>BackFromKeyboard</Action>
</DynamicKey>

You may find that the scaling of text and symbols is still too small, for example if you have a wide key with long text. In this case you can disable the "size group" scaling by setting two other properties (XML attributes): AutoScaleToOneKeyWidth and AutoScaleToOneKeyHeight. For example:

<DynamicKey AutoScaleToOneKeyWidth="false" AutoScaleToOneKeyHeight="false">
     <Label>Back</Label>      
     <Action>BackFromKeyboard</Action>
</DynamicKey>

⚠️ Note that if your key is part of a KeyGroup in the Dynamic keyboard, then this will override the AutoScaleToOneKey[Width/Height] attributes. In this case, you need to turn off the behaviour in both the key group and the individual keys, for example:

<KeyGroup Name="NavKeys" BackgroundColor="#907090" AutoScaleToOneKeyWidth="false" AutoScaleToOneKeyHeight="false"/>    
<Content>    
    <DynamicKey AutoScaleToOneKeyWidth="false" AutoScaleToOneKeyHeight="false">
        <KeyGroup>NavKeys</KeyGroup>
        <Symbol>BackIcon</Symbol>      
        <Action>BackFromKeyboard</Action>
    </DynamicKey>
</Content>

Key Text Fonts

Some text requires specific fonts in order to render correctly. Optikey supports a number of explicit fonts which you can specify on the key:

UsePersianCompatibilityFont - set to true for Persian characters

UseUnicodeCompatibilityFont - set to true when you wish to display a diacritic character around a placeholding circle, e.g. an accent hovering over a dotted circle to show how it would display on a character. An example of a key label where this would be required is <Label>◌́</Label>

UseUrduCompatibilityFont - set to true for Urdu characters