Change USE Category for Brahmi Virama U+11046

[punchcutter]

As stated in UTS Old Tamil Brahmi has a special use of the virama:

Tamil Brahmi puḷḷi (virama) had two functions: to cancel the inherent vowel of consonants;
and to indicate the short vowels [e] and [o] in contrast to the long vowels [e:] and [o:] in Prakrit and Sanskrit. As a consequence, in Tamil Brahmi text, the virama is used not
only after consonants, but also after the vowels e (U+1100F, U+11042) and o (U+11011,
U+11044). This puḷḷi is represented using U+11046 brahmi virama.

To work with USE shaping the category for U+11046 should be changed. I propose overriding the IndicSyllabicCategory to be Bindu which will make the USE category VMAbv instead of H unless there is a better suggested override. Then USE cluster validation will work correctly for these situations.

Test string: 𑀓𑁂𑁆 𑀓𑁄𑁆 𑀓𑁂𑁆𑀓𑀼 𑀓𑁄𑁆𑀓𑀼
U+11013,U+11042,U+11046,U+11013,U+11044,U+11046,U+11013,U+11042,U+11046,U+11013,U+1103C,U+11013,U+11044,U+11046,U+11013,U+1103C

Current behavior

Desired behavior

[dscorbett]

Changing the category of U+11046 would make it no longer work like a virama, so conjuncts would be split into multiple clusters.

U+11046 is similar to U+0D4D MALAYALAM SIGN VIRAMA, which can either be a virama or a vowel modifier. The solution for U+0D4D in hb-ot-shape-complex-indic-machine.rl was to allow any virama after any vowel sign: matra_group = z{0,3}.M.N?.(H | forced_rakar)?;. Something similar could be done in hb-ot-shape-complex-use-machine.rl, though it would unfortunately allow many nonsense sequences. It would be nice if it were easy to add new Indic categories to HarfBuzz so that U+11046 could be in its own category without affecting other viramas.

[punchcutter]

Yes, I know! Argh. If it were a simple USE category change I would have submitted a pull request, but since it's not so simple I opened this. Maybe it would be nice to allow multiple categories for situations like this. So U+11046 could be both H and VMAbv. A very rare thing, but like you mention with Malayalam that's going to fail once Indic 3 tags are working and it gets run through USE.

[Richard57]

Multiple uses are not so rare - U+1A7A TAI THAM SIGN RA HAAM is both a vowel killer and a final consonant, and in loanwords from English (and sometimes from Siamese) it can also function as a consonant killer. Vowel killer was the least bad option. Peter Constable has been calling one Thai vowel killer a 'nukta' - because that is how it is sometimes used.
Allowing vowel + conjoiner at the end of the syllable also makes sense for incompletely typed Tai Tham and Khmer - which is why invisible stackers tend to be rendered as a subscript '+' sign. (They're always invisible in grammatically correct text.)

[Richard57]

As I said, another solution is to allow H near the end of a standard cluster. Not only would this allow Brahmi and mly3, it would also alleviate the problems of Tai Tham. Tai Tham CVHC would be wrongly interpreted as two non-defective orthographic syllables, CVH|C, as opposed to a standard cluster plus defective syllable(s) CV|HC (CV|H|C in Uniscribe) but that should merely require a second round of subscript formation after the dissolution of syllable boundaries. Allowing Tai Tham CVH should not be a problem - the subscript '+' already indicates a problem.

[behdad]

@punchcutter can you summarize what I need to do here?

[punchcutter]

@behdad I'm not really sure the best way to handle this. This virama needs to also work as not-a-virama. It all worked before USE came along because there was no strict cluster validation, but now USE clusters expect the virama only in a couple specific places and this situation needs something else. If a character could be assigned multiple USE categories that would work and might be useful in some other scripts, but it definitely complicates everything.

[Richard57]

Well, if you insist on leaving institutionally racist syllable analysis in place (note that it messes up non-Indo-Aryan languages in Indic scripts), you could always use the following process:

1. vowel + Brahmi virama > vowel + Brahmi virama + ZWNJ
2. Brahmi virama + ZWNJ > U+110000 (utterly private plane)
3. Make U+110000 a pure_killer with InPC=top.
4. Split syllables.
5. Convert U+110000 back to Brahmi virama + ZWNJ.

Andrew Glass has [url=https://unicode.org/pipermail/unicode/2018-August/006941.html]floated[/url] the idea of adding pulli to the Brahmi character repertoire, which is what U+110000 above would be.

[behdad]

Ok, we split the Halant category to add a Halant_Or_Vowel_Modifier. This meant modifying the USE machine. Should be properly reported to Unicode and USE to add new category.