Use of CGJ

[r12a]

4.2 Override Mechanism for Exceptions
http://www.unicode.org/reports/tr53/

The default display order implemented by the UAOA will be correct for most uses. However, these are situations where a different mark order is desired. In these cases, U+034F COMBINING GRAPHEME JOINER (CGJ) can be used to achieve the desired display order.

Question:
It isn't really clear to me, but i'm guessing that the expectation is that content authors will use the CGJ in their content prior to application of the UAOA rules, rather than as an additional step in ordering text per the UAOA rules.

Editorial suggestion:
If that assumption holds, I'm surprised that the UAOA algorithm in section 3.2 doesn't mention that CGJ is a non-starter. For example, the step "Move any shadda characters ... to the beginning of S" should presumably not apply to a shadda that is preceded by CGJ.

[behnam]

I submitted a long individual feedback. Here's the part related to this issue:

---

4. Consequences of the Algorithm: Semantics

With UAOA applied on text during rendering, some strings collapse into a single sequence. Basically, there are plenty of strings X and Y, where toNFC(X) ≠ toNFC(Y), but UAOA(toNFC(X)) = UAOA(toNFC(Y)).

Basically, this is changing the semantics of existing text encoded in Unicode, since the rendering will be different afterwards. The document is not clear about this semantic change and only claims to “correcting” all the problems.

The proposal is suggesting to use CGJ to preserve the old semantics when needed. The document needs to be more clear about how to preserve the semantics. In fact, there should be a clear algorithm to convert a string X to preserve the semantics when changing the (rendering) interpretation, since for a couple of decades users have been storing text in the current semantics of the encoding, which has been the only recommended way to do so by Unicode.

[r12a]

The original comment above may be sent after UTR53 has been updated, if it is still relevant. Setting a status of Waiting for now.

[r12a]

Resend the comment.