roadmap for improving handwriting functionality Better

[Mokhyy]

@wasi-master

I have a few ideas for this section, which I'll write down for you. Read them and share your thoughts. Also, if you have any ideas, please mention them here. This section has great potential, especially for students and university students.

First of all, the handwriting feature provided by Google's ML Kit digital ink is a closed-text engine that can't be significantly enhanced, and it hasn't been updated for several years. This made me think: what if we train our own engine using this engine so that it can be upgraded in the future?

Now, how do we do this?

The way to do this is exactly like what Heliboard does for creating machine learning , for swipe ( glide ) typing .
Let's create a section within the settings where users can use the handwriting feature and send us their data (metadata).

This section needs a canvas connected to digital ink and a part that displays text, which retrieves words from the dictionary of that language and displays them. It also needs a button for changing the language (changing the language will also change the dictionary language and the digital ink language).

To start this process, we place a "Start" button. When the user clicks "Start," the digital ink engine in the background turns on, and the relevant dictionary for that language becomes active, displaying its words.

The user sees these words and draws them on the canvas. Then, several pieces of information from different parts are saved in a log for training purposes.

First of all, the stroke data drawn by the person is stored. That is,
x
y
t
and

You can even save speed and acceleration for more accuracy for the engine.

Then, the word drawn is scored by digital ink and saved in the log. You can also retrieve up to 3 to 4 subsequent options that digital ink recognizes and save them in the log to have a set.

---

The second idea is to do something to make the lite version also able to use digital ink.

Exactly like what LeanType does, connect the digital ink library as a separate plugin to the app. Definitely look at the method LeanType uses for this work; it will help you a lot.

Definitely visit this repository
https://github.com/LeanBitLab/Leantype-Handwriting-Plugin
and check the app's implementation to see how it works offline, downloading models and so on.