Re-integrate work on capitals, emphasis and numbers in UEB by APH

[bertfrees]

• EndEmphasisPatch.txt can be integrated right away, Mike's further work on emphasis will not rely on that patch.
• Eventually integrate Mike's mrg-emphasis mrg_ueb_update branch. APH will try to keep their fork in sync with master.

---

Some relevant posts/threads on mailing list:

• Proposal for capital and emphasis in UEB
• Capital/Emphasis update
• UEB update
• UEB numeric mode update
• Problems with the letter sign in grade 1 and grade 2
• letter indicator upate
• Script indicator
• Mappings
• UEB NumericModeChars
• Michael Gray, how to you test your branch?
• Questions about UEB patches by Michael Gray
• word_reset as opode
• Chat between APH and Nordic Braille in DAISY Pipeline 2 project
• UEB patches: Summary of issues with examples
• Inconsistent behavior with begcaps and endcaps
• Indicator scope and backtranslation
• Documentation

Some related issues:

• [#99] Combine emphasis opcodes
• [#103] Update the Python wrapper to support the new UEB typeforms
• [#116] Add emphmodechars opcode
• [snaekobbi#10] Ordering of the open and closing "tags" in braille

See also the wiki article https://github.com/liblouis/liblouis/wiki/New-opcodes-for-UEB

---

Overview of the changes:

• Fixed the behavior of emphasis opcodes.
• Extended the set of opcodes to cover more use cases.
• Handle capitalization like other emphasis.
• Handle script indication like other emphasis.
• Added 5 other typeform indications (correspond to 5 transcriber-defined indicators in UEB).

firstword*
lastwordbefore*
lastwordafter*
len*phrase
firstletter*
lastletter*
singleletter*
*word
*wordstop

* = {ital, bold, under, caps, script, trans1, trans2, trans3, trans4, trans5}

• Introduced the typeform bits "passage_break" and "word_reset" to give the user more control.
• Added the opcode numericmodechars for indicating that a character does not terminate the numeric mode.
• Added various opcodes as a replacement for letsign.

nocontractsign
seqdelimiter
seqbeforechars
seqafterchars
seqafterpattern
numericnocontchars

• Changed input-output mapping for end indicators.
• Deprecated some opcodes.

capsign
begcaps
lenbegcaps
endcaps

[bertfrees]

Some questions from @dkager (w.r.t. the current status of branch dkager_dutch_ueb: snaekobbi/liblouis@d22a56d...8c46f33):

• Is there a way to specify which chars don't cancel an all-caps phrase?
• Why are the wrong (parts of) words being emphasized? (maybe write a separate test case and small table for this)
• Do you need numericmodechars, or is decpoint broken?

[bertfrees]

• Is there a way to specify which chars don't cancel an all-caps phrase?

There is the word_reset typeform bit for manually specifying "that a word indicator stops at that point in the word and will need to be repeated if it continues", as explained in Re: Capital/Emphasis update. That could work. I suggested to have an opcode, e.g. called wordresetsign or nowordresetsign (Re: Capital/Emphasis update), or caplike as I suggested later (Re: UEB numeric mode update), for defining characters that generate a word_reset bit. But I'm not sure this is implemented or not.

[bertfrees]

• Why are the wrong (parts of) words being emphasized? (maybe write a separate test case and small table for this)

This issue was solved by adapting the Python bindings. See Re: Questions about UEB patches by Michael Gray and #103.

[egli]

I think this is basically done. There are some issues still with ueb numeric mode but these will have to be tracked separately