Add a fallback alias opcode

[rimas-kudelis]

I'm in the process of finishing a fallback table which will map all accented Latin letters to their non-accented equivalents. However, I think it would probably become even more practical and useful if instead of defining each such letter as a dot combination (e.g. noback uplow Àà 1), I could define it as a "fallback alias", meaning that "this character should be treated as this other character (or this sequence of characters), but only if there are no other definitions for it". This would allow to achieve two things:

1. re-use such fallback mappings in verbatim form regardless of what dots target table uses (e.g. Latvian maps U, V and Z to different dots than usual)
2. put these definitions anywhere in the target table, without worries that they will override target alphabet definitions.

The second point is less important, as the correct order could be specified in the documentation, but in either case, care should be taken to not allow such aliases automatically override other definitions. At the very least, they should follow the usual position-dependent precedence pattern.

[rimas-kudelis]

BRLTTY already has a similar opcode, which is called alias.

I'm saying "similar" here because I'm not sure how it behaves when contradicting opcodes are specified (e.g. alias a b followed or preceded by lowercase a 1).

[bertfrees]

http://brltty.com/doc/TextTables.html#the-alias-directive

[rimas-kudelis]

Yep. The documentation doesn't tell what happens in case of conflict though. :)

[bertfrees]

Just putting the link here for my own information.

[rimas-kudelis]

One more thing: according to documentation, BRLTTY only allows to define one character as alias target. I would suggest allowing more. This would allow to alias letters like þ to th for example, or even implement transliteration.

[DrSooom]

CC: @egli, @bertfrees, @LeonarddeR, @Andre9642, @school510587 and @zstanecic. And due to the NVDACon 2019 also CC @derekriemer, @Adriani90 and @JulienCochuyt.

Two days ago I held a talk during the NVDACon 2019 about the HUC Braille Tables. After @zstanecic contact me via Signal yesterday, I was thinking about a fallback opcode as a quick fix for the multiple definitions issues, which occur when the HUC Braille Tables are included into other braille tables. In issue #688 and #689 we already spoke about a new opcode for the HUC Braille Tables. But maybe this isn't required right now due to the fallback opcode.

Actual behavior for single Unicode code points:

sign \x0041 17
sign \x0041 13678-245678-1345

Depending on the primary used braille table(s) and the state of the NVDA option "Expand to computer braille for the word at the cursor", ⣥⣺⠝ (HUC8 style) instead of ⡁ is displayed on the braille display. See also question No. ⢅ and No. ⠁ in the HUC Braille Tables documentation.

Expected behavior for single Unicode code points:

sign \x0041 17
fallback sign \x0041 13678-245678-1345

As sign \x0041 was already defined before, the fallback line will be fully ignored. Thus ⡁ is still shown on a braille display. Note the every tbi file contains all 65,536 code points of one single Unicode plane. I plan to release an extended version of the HUC Braille Tables – yes, I'm really adding 32 additional tbi files – to improve the UTF-16 surrogate representation for UTF-16 end applications. Maybe the total file size of all files together could increase to 140 or 150 MB. When I'm compressing all them into a single 7z archive with LZMA2, then the file size of the 7z archive could be between 3 and 5 MB.

Example No. 1 for UTF-16 surrogate pairs:

sign \xd83d 13678-2378-147
sign \xde00 13678-28-245678
fallback always \xd83d\xde00 134678-367-245678

Even if I don't think that there are braille tables out right now, which define low and high surrogates (@school510587: I know about your try here in issue #664), we should handle such conflicts for UTF-16 end applications. UTF-32 end applications such as NVDA 2019.3 (and later) aren't affected here.

Example No. 2 for UTF-16 surrogate pairs:

always \xd83d\xde00 25-1245-1235-24-1345-124-25
fallback always \xd83d\xde00 134678-367-245678

The fallback line will be completely ignored. ⠒⠛⠗⠊⠝⠋⠒ instead of ⣭⡤⣺ is displayed. This representation for an emoji could be a solution for nvaccess/nvda#9213. But I still prefer to replace the code point of this Unicode character with ":grinf:" by the end application like NVDA itself. Thus Liblouis just have to translate the seven code points and not the two code points of the UTF-16 surrogate pair. The ⠒ for the colon comes from "de-de-comp8.ctb". @ABuffEr, I'm not working on this.

Please note that nvaccess/nvda#8702 is still the final goal for the HUC Braille Tables. But before we – or only you, as I haven't the skills for it – are trying to realize this possible solution, which I mentioned here in issue #689, having this fallback opcode for the HUC Braille Tables would solve the most annoying issues with them at once. For compatibility reasons I will add the fallback opcode first in the third version of the HUC Braille Tables, so the improved second one can still be used with NVDA 2013.2 on Windows XP, which I really tested a few weeks ago on a very old notebook. The second version of the HUC Braille Tables are planned to be released in the first half of 2020. Creating the new 32 tbi files takes much time and I firstly still have to do quite a lot regarding the HUC Braille Tables documentation.

[bertfrees]

@DrSooom Two questions.

• You seem to have a different idea about what a alias opcode would mean than Rimas. In fact, apart from the name, I don't see any resemblance. So maybe create a new issue for it?
• I don't really see a need for an alias opcode the way you define it. As far as I understand it's more a matter of documenting and possibly changing the precedence rules, and then making sure that the primary table and the HUC table are included in the right order. What would be the big benefit of your new opcode?

[DrSooom]

@bertfrees: Could it be that issue #868 already answers your questions? If this isn't the case, please let me know. Thanks.

[bertfrees]

Yes, thanks. This was just an old comment (from 2019) that I edited, you probably got a notification because of that. Sorry for bothering you.