Different character sets for BangleJS 2

[espruino-discuss2]

Posted at 2022-02-21 by abbradar

Greetings from Russia! My device has finally arrived and I'm having a lot of fun with it.

One immediate issue that I noticed is lack of Unicode support for notifications. It's understandable that full Unicode character set is infeasible (both support and fonts). However, there are also known local character sets, like ISO8859-5, CP1251 or KOI8-R for Russian.

I was wondering how complex would it be to:

• Replace font in Messages with one containing Russian letters on expected code points for one of the encodings above;
• Add Gadgetbridge support to encode strings as expected.

Where should I start with this on BangleJS side? Can I somehow make font in Messages customizable and configurable from app store? Ideally one would upload a desired font separately (or just use the system one), and then select needed encoding in Gadgetbridge settings.

[espruino-discuss2]

Posted at 2022-02-22 by @gfwilliams

Hi!

It'd be interesting to see what gets sent by Gadgetbridge (maybe with the Gadgetbridge Debug app) but I have a feeling that rather than sending UTF8, it might convert everything to ISO Latin, in which case even if you changed the font (which you can do) Gadgetbridge will still need modification.

A few months ago I added something to Espruino that allows g.drawString (so any text drawn) to include images. You just include character code 0 followed by the image as a string (http://www.espruino.com/Graphics#strings).

So... what we could do is in Gadgetbridge we check for any characters that aren't in the ISO Latin char map, then we render those (ideally as a whole word) to a bitmap and then include it in the string we send over. We actually already have code for the bitmap conversion as this was on my list of things to do: https://codeberg.org/gfwilliams/Gadgetbridge/src/commit/c597adf27fb4628ca523c40a99e78aa5a613b21e/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/service/devices/banglejs/BangleJSDeviceSupport.java#L639

And there is even some code that's already been tried at espruino/BangleApps#754 which renders everything to an image, so actually it shouldn't be too hard to modify to get it to render just the relevant characters/words instead.

It's been on my list of jobs for a while though, so if you were willing to take a look that'd be amazing

---

Posted at 2022-02-22 by abbradar

Interesting approach! This could allow us to show any Unicode codepoint, not only a limited set.

I went into another approach, which is to store localized fonts in the device's memory and decode UTF-8 to them according to charmaps. This potentially doesn't require any changes in Gadgetbridge at all. Another benefit is (admittedly, not measured) performance gain from sending strings instead of bitmaps over BLE (especially for long localized messages).

This, however, requires user to upload a special font and a charmap together into the device. Can we in theory package them together as some sort of "font" in the app store, and then select and use this "font" in Messages?

I have a prototype working here: https://github.com/abbradar/banglejs-ru . It also requires a patched Font Converter, which can create fonts according to a charmap : https://github.com/abbradar/EspruinoDocs (pre-compiled HTML at https://abbradar.moe/me/banglejs-font.html ). I consider Font Converter code ready to make a PR if you agree that this can be useful.

One small issue is performance of conversion from Uint8Array to String here: https://github.com/abbradar/banglejs-ru/blob/ffb62e592e63dd1f3a826480789e9f56138a0717/decode.js#L133 We don't have TextEncoder implemented, but is there any better way? Otherwise I consider decodeUtf8 performance to be good enough for notification messages.

What do you think about this approach in general? Should I try integrating this with Messages? We could also use hybrid approach: support local alphabets as fonts and show other symbols (emoji!) as inline bitmaps.

---

Posted at 2022-02-22 by @gfwilliams

This potentially doesn't require any changes in Gadgetbridge at all.

Well, if Gadgetbridge sends UTF-8 - but I don't think it does so this still won't work without Gadgetbridge changes anyway.

Your approach would at least work well on iOS though where UTF8 is sent and we have no choice.

Can we in theory package them together as some sort of "font" in the app store, and then select and use this "font" in Messages?

Yes, potentially we could.

One small issue is performance of conversion from Uint8Array to String here:

Have you seen E.decodeUTF8? It's built-in already and I feel like that should be faster and would avoid a bunch of code.

What do you think about this approach in general?

I'm afraid I prefer the Gadgetbridge option. If we do that, we can do exactly what you suggest with emoji, but it'll work for any emoji, or any language, without requiring any special uploads or setup. I feel like that's got to be a huge benefit.

Your changes for the character map could be a good idea to pull in though (although it'd be nice to have some examples of what that mapping JSON file might look like)

---

Posted at 2022-02-22 by abbradar

Well, if Gadgetbridge sends UTF-8 - but I don't think it does so this still won't work without Gadgetbridge changes anyway.

Interesting - I expected it to do just that.

Have you seen E.decodeUTF8? It's built-in already and I feel like that should be faster and would avoid a bunch of code.

Wow, I completely missed it! This does exactly what I need. We could mention it in https://www.espruino.com/Fonts#character-sets -- I didn't expect that kind of function in E.

I'm afraid I prefer the Gadgetbridge option. If we do that, we can do exactly what you suggest with emoji, but it'll work for any emoji, or any language, without requiring any special uploads or setup. I feel like that's got to be a huge benefit.

Makes sense, yeah. I'm still worried about performance, but we could measure it and think of improvements afterwards. One possible way would be to implement some sort of on-device LRU cache for glyphs which is managed from Gadgetbridge side.

I'll see if I can make something work.