UNSAFE_TO_CONCAT [Was Clarify UNSAFE_TO_BREAK and expose OS2V2Tail::usMaxContext as context.]

[bungeman]

The current documentation clearly states what UNSAFE_TO_BREAK (or it's absence) can be interpreted to mean. However, it should also be documented as to what it is not, since it's very easy to be lured into a false sense of security with it.

The current behavior (as I understand it) is that one can shape a sequence of code points into glyphs, pick any two safe to break locations in the resulting buffer, reshape the associated sub-sequence of code points in isolation and get the same buffer output as is seen between the original two safe to break locations. Essentially, any pair of safe to break locations is stating the the larger context of the original sequence of code points did not affect the shaping of that run. (This is actually a bit stronger of a statement than the current documentation implies, please correct if this is not true.)

However, it is important to note that the inverse is not true, it is not the case that such a run of code points will always shape this way; any modification to the original sequence of code points can arbitrarily modify the overall shaping. For instance I can make a font which creates ligatures of all the sentences in the works of Shakespeare such that "There is a written scroll" will come back with many safe to break locations, but "There is a written scroll!" will shape into a single glyph which visually represents the icon for the Prince of Morocco. In other words, adding to or modifying any of the original sequence of code points invalidates everything about the shaping before doing so (including any safe to breaks).

In order to avoid requiring complete re-shaping when editing (especially simply adding to the end of a paragraph), it is necessary to let the user know the maximum affected area. For OpenType this is the purpose of OS2V2Tail::usMaxContext, in theory even TrueType knows the answer to this question for at least appending (keep track of the last ground state). With this sort of information it should be possible for a sophisticated user to only re-shape a portion of the overall sequence of code points by dirtying the maximum context area and finding the safe to break locations surrounding that area. The pathological Shakespeare font I propose above would obviously have quite a large value for the maximum context, but for most fonts this value is three or less.

In between these two cases (breaking at safe to break locations and mutating the sequence of code points) there is the case of determining the extent of re-shaping if truncating the original sequence of code points. Is it enough to find some safe to break location before the truncation and shape only the remaining portion? I cannot construct an example to answer in the negative, but it is not obvious that this is always allowable either. I think this is equivalent to asking if any shaping operation can operate by negative look ahead in a way which doesn't affect safe to break. With the maximum context information it seems this could be at least bounded, but it would be nice to know if this common case can be handled even more efficiently (may be common when line breaking).

With all of that as background, the two actual requests here are:

Can general and/or end of shaping maximum context information be exposed to reduce re-shaping when editing?

Can anything be assumed about code point sequence truncation and safe to break information?

[khaledhosny]

I’m not sure usMaxContext can e trusted, HarfBuzz does not use it and I don’t know any other open source library or application that makes use of it (the only application that I know that makes use of such information, LibreOffice, actually uses a hard-coded value for max context https://github.com/LibreOffice/core/blob/148beecad4ead6378e3c92c0b48d37ad2f0ecfc0/sw/source/core/text/itrform2.cxx#L2702-L2705). As such I wouldn’t trust fonts to be setting this value correctly, since any bugs in it would go unnoticed.

[bungeman]

Hmmm... yes, I see less than enthusiastic comments about its value, for example fonttools/fonttools#466 (comment). Of course, there's always the slow way of just calculating it, which may actually be fine for an editor where there aren't actually that many fonts in play but layout is going to be done many times.

Without some sort of value like this correct editing (and sometimes line breaking) is more like n^2 on the length. I happen to know the guy who wrote the first code for detecting ground states in the TrueType state machine shaping to determine how far back to re-shape when appending. Just picking a number like LibreOffice seems to do seems as bogus as using a semi bogus number from the font. Also, until UNSAFE_TO_BREAK was introduced in HarfBuzz there wasn't any reason to know about this value since it was necessary to re-shape the entire run anyway for correctness. This would explain why no open source project was ever interested in this value.

I am interested if there is a example where truncation of the of the initial code point sequence requires full re-shaping. It's super tempting on truncation to simply find the previous safe to break position and just re-shape from there to the truncation point. If this is always possible it would be great to state that, if not a counter example should be stated in the docs to scare implementors away from doing it.

[behdad]

Thanks Ben. You make the use-csae very clear.

1. The current usMaxContext cannot be relied on. It's easy/fast enough to ask HarfBuzz to compute it. We need to implement that at some point anyway.

2. usMaxContext cannot reliably be used however, given that lookups can skip marks, and HarfBuzz also skips default-ignorable-characters. So, to reliably use usMaxContext one also needs to duplicate that logic.

However, all of the above is irrelevant. You are assuming that appending a char to a buffer can affect at most usMaxContext chars before it, and as such one can find any unsafe-to-break point before that range and reshape from there forward?

There's at least one very easy reason why that's completely unsafe: ReverseChainContextSubst subtables. With those, eg. you can have a sequence of 'a's shape to themselves. But as soon as you append a 'b', all the 'a's before it also change to 'b'. The usMaxContext for that substitution is short (1 or 2), but the effects propoagate backwards.

'morx' table can also do this easily as morx subchains can process input in reverse.

Even without ReverseChainContextSubst subtables, the usMaxContext needs to be multiplied by the number of lookups, since having 100 lookups each of which can look forward 5 glyphs, when combined, can look forward 500 glyphs and substitute....

In summary, I suggest you don't try to get this information out of shaping / font. IMO the best way to handle this is to reshape from last line break.

[drott]

CC @kojiishi

[bungeman]

This is great information. I suppose that means https://cs.chromium.org/chromium/src/third_party/blink/renderer/core/layout/ng/inline/ng_line_breaker.cc?l=525&rcl=664f94acf2faf985bd33be531b5a9a0f8fcaff46 may need to be reconsidered?

So, restating as I understand it, safe to break is good only for partitioning an unchanging sequence of code points at safe to break points. Changing the sequence of code points in any way will require re-shaping any shaping runs which the change affects (often an entire paragraph). Attempting to partition at an unsafe to break point (e.g. while line breaking) will require reshaping from the beginning of the sequence of code points (which is often the beginning of the current partition / line). In theory a real maximum context could be computed from the font, but this number may be very large and generally not useful. At any given point a tighter maximum local context could be computed, but that may be more expensive that just re-shaping (and a lot of code).

[behdad]

I think the best way forward is for HarfBuzz to expose a new flag bit that does exactly what you want. Since most fonts don't have back-to-front lookups, this will be effective for most fonts. If font has ay back-to-front lookups, we'll mark the entire run unsafe. Let me think about exactly how to implement that efficiently.

Need a name for the new flag.

[behdad]

I think I know how to implement this. Next step is to prototype it. @kojiishi / @bungeman would you be able to test it in LayoutNG?

[kojiishi]

Like this example site, reported at crbug.com/479370?

I think I was under the assumption that UNSAFE_TO_BREAK includes these cases. If not, and if we're adding a new flag, it's easy for LayoutNG to switch to it. We could optimize further by distinguishing two cases as @bungeman pointed out, but I think I'd like to do it incrementally, when we know better if it's causing the perf problems. /cc @eaenet in case he has different opinions.

[behdad]

In order to avoid requiring complete re-shaping when editing (especially simply adding to the end of a paragraph) ...

While in theory true, I doubt that the implementation will use the flag I'm going to add properly to avoid reshaping when text changes. That requires a lot of change tracking....

[kojiishi]

Currently LayoutNG tracks changes in nodes, but not part of text nodes. In Phase 1, we will not ship editing part though, we switch to the current engine for editable elements. The flag may help fixing cases such as:

<div id=e>a</div>
e.appendChild(document.createTextNode(' b'));

though I'm not sure how common the problem is, and Blink will need some additional work in addition to using the new flag.

We will start working on editing after phase 1. At that point, we will know better how much text-offset-level change tracking is needed. /cc @yosinch

[bungeman]

As a potential counter-example without explicit backward running features, consider that I construct a font with a simple 1:1 mapping of characters to glyphs except for two features, one of which (say 'rlig') which has a lookup 1 which maps "ived" with a special glyph and another (say 'liga') which has lookup 2 which maps "tri" to a different special glyph and shape the string "contrived" with this font. (The same effect might happen with one feature with two lookups.)

As I understand it, the feature lookups run in the order of the lookup list (after determining which features to run in a given pass). As a result, "contrived" should map to [c o n t r ived] with 'ived' marked as safe to break. However, should there be a possible line breaking point found between "e" and "d" such that we should be shaping "contrive" we should end up with [c o n tri v e]. However, this would not be the result if going back to just before 'ived' and just re-shaping the "ive" into [i v e].

It seems like this is a general issue any time a prefix of an early lookup is a suffix of a later lookup. If this is a valid example, it seems that it would be more difficult to take this sort of behavior into account than simply looking to ensure no backward looking features are in the font. I suppose most fonts don't have this property, though now I think I'm going to create this one just to see the effect.

[kojiishi]

IIUC we all agree you're correct, and appreciate to bring this problem into our focus. It's even easier to hit than your example. Consider, user typed "f", shape, then type "i". If apps is smart enough to know "f" is unchanged, because after "f" is safe-to-break, the app will not reshape "f" and fail to get the ligature.

Behdad's point is that apps is not that smart today, and he's right on that regard. Blink reshapes all the text as long as they're in a same node. The Blink code you quoted above is only for removing trailing spaces of line end, so it should be ok.

Blink may want the smartness when we work on editing in post phase 1 if the current per-node change tracking hits the performance bad enough. If Blink switch to per-character change tracking, this case will be much more common.

[behdad]

Okay. I thought more about this. There are some things that I misunderstood before about our current unsafe-to-break flag. I like to correct that understanding, and together we can design a new flag for stronger guarantees.

The way I previously suggested the unsafe-to-break can be used was this: full paragraph shaped, then line-break position determined, and if line-break position is unsafe-to-break, then walk back from it to the first safe-to-break, and shape the tiny resulting piece and append it to the shaped results for the chunk of line before it. This, unfortunately doesn't work.

What safe-to-break guarantees, and only thing it guarantees, is that if you break at this point, and shape the two sides separately and append the results, it would be same as shaping the whole thing. But if you break here and then change one of the pieces, eg by truncating it, then there's no guarantee that shaping can be done in pieces anymore. Ie. if "abcd" as text results in glyphs "AB|CD" where "|" means safe-to-break, then it must be the case that "ab" shapes to "AB" and "cd" shapes to "CD". Let's imagine "c" also shapes to "C", but there's no guarantee from these that "abc" shapes to "ABC". It might shape to "XYZ".

Back to line-breaking. The optimization can be changed to this: if chosen line-break is safe-to-break, then good, just break the glyphstring and move on. Otherwise, the line needs to be reshaped. In a separate commit I'll talk about how to design a flag that can give stronger guarantees.

[behdad]

It seems like this is a general issue any time a prefix of an early lookup is a suffix of a later lookup. If this is a valid example, it seems that it would be more difficult to take this sort of behavior into account than simply looking to ensure no backward looking features are in the font. I suppose most fonts don't have this property, though now I think I'm going to create this one just to see the effect.

Oh I didn't mean that any font that doesn't have backward features is safe... I was designing a new flag that can be implemented to discover such break positions. I'll expand on it in more detail separately.