Add support for block level shear.

[palemieux]

tts:fontShear currently specifies that "for the purpose of determining applicability of this style property, each character child of a p element is considered to be enclosed in an anonymous span." Combined with the fact that the style property is inheritable, this means that the shear transformation is applied independently to each character child of a p element.

Based on D-Cinema practice, a p should instead be sheared as a block (instead of each of its character elements independently) in order to preserve character alignment across lines esp. in the case of ruby text (see https://github.com/w3c/imsc-tests/blob/imsc-1.1/imsc1_1/png/fontShear001/3.000000.png).

Furthermore, shearing of inline elements is not supported by CSS.

I recommend making tts:fontShear applicable to p only, and specify that it yields a transformation of the entire block.

Alternatively, #fontShear could be split into #fontShear-block and #fontShear-inline, and the semantics of tts:fontShear clarified to specify that how it applies differently to inline and block areas.

[skynavga]

@palemieux Unfortunately, this interpretation of the intent of application is incorrect. It applies to both, but in entirely different ways, which does need to be clarified in the spec. In particular, the font* properties that are said to apply to p have the sole purpose of being used to compute the default value of the line height that applies to the line areas generated by formatting the p. In the case of fontShear, the shearing transformation is NOT intended to be applied to the block areas generated by formatting the p. Rather, the shearing transformation is intended to be applied to individual glyph area descendants of these block areas.

[palemieux]

Rather, the shearing transformation is intended to be applied to individual glyph area descendants of these block areas.

That is unfortunate. It looks like a new property will be needed to specify a shear applied to entire block, if tts:fontShear cannot be used. Would tts:shear work?

[skynavga]

Yes, that would work, but also begs the question of whether we need tts:rotate and perhaps tts:translate. Since we have not previously discussed the application of block area level shearing that appears to be used in D-Cinema, I wonder if we should hold off trying to address this in TTML2, particularly since it is late in the process, and we are already well past the deadline to add new features.

[palemieux]

I wonder if we should hold off trying to address this in TTML2, particularly since it is late in the process

Unfortunately there was a misunderstanding that is surfacing only now that tests and implementations are being created. We should focus on getting to the bottom of this in the next two weeks.

[skynavga]

OK, but there is at least one implementation (TTPE) with tests that adhere to the current specification text. My thinking at this point is that we should resolve #785 now (for CR2), then label this issue for resolution in ttml.next and close it (for this repo).

[palemieux]

@skynavga As I understand it, the lack of a block shear is a blocker for TTML2 publication. This ticket can be renamed support for block shear instead of closing it and reopening a new one.

[skynavga]

@palemieux where did that come from? blocker for whom? since that is an entirely new feature, I would dispute this notion

[palemieux]

since that is an entirely new feature

This is perhaps an entirely new feature to you, but not to other readers of the CR.

I am trying to get to the bottom of this, i.e. which of the two styles of shear/oblique/slant/shatai is required for subtitling, and suggest you do the same.

In the meantime, let's keep the issue open.

[skynavga]

This is perhaps an entirely new feature to you, but not to other readers of the CR.

That would be called reading between the lines.

The current specification is based on an analysis of the content specification for LambdaCAP Japanese subtitling format. An English translation of the relevant section clearly shows shear (slanting/italic) being applied to individual glyph areas, and not line areas or larger block areas.

[palemieux]

@skynavga The example above does not show the desired effect when ruby annotations are present.

[palemieux]

@skynavga In the case to Tate-chu-yoko, wouldn't you expect the horizontal writing to be sheared with the rest of the block?

[palemieux]

As another example, shearing the ruby text together with the ruby base preserves character alignment.

[skynavga]

@palemieux Can you point at a reference that shows standing conventions when shear is combined with ruby or tate-chu-yoko? I have not yet found such reference.

[palemieux]

@skynavga I am working on it. Let me know if you find some as well.

[skynavga]

Change title as suggested by @palemieux. Which block areas the shear are to be applied to remain undetermined at this point. For example, would it be individual line areas (which are block areas) or the parent block area of consecutive line areas?

[akiseino]

Hi, I'm Aki Seino from IMAGICA, post prodction including
localization service (Subtitle) for OTT and Digital Cinema
in Japan. And I'm Japanese native speaker.

I got the link from Pierre and let me add my comment.

We have same issues at digital cinema (DCP) with Japanese subtitle but
Sheared as block looks natural.

At above two samples Pierre mentioned,
The ruby by individual, is moved to left side. But it's center by block.
The base (bottom) line of Tate-chu-yoko "34" by individual and it's not oblique at "4". But it's same oblique "34" by block.

[skynavga]

@akiseino thanks for your input; can you refer us to a specification or document that describes

• the behavior of shearing (also called slanting or obliquing) transformation to Japanese text in both horizontal and vertical layout when used with ruby or emphasis (kendot)
• the same for vertical layout when used with tate-chu-yoko

It is OK if the specification or document is in Japanese.

[akiseino]

Hi skynavga, thank you for your comment.
Let me try to find the specification or document.
In the actual work with those in digital cinema, we will align to be natural looking by mixing
parameter setting and show our client for approval.

[palemieux]

@skynavga It is entirely possible that the decision to shear by block or by character will turn out to be an aesthetic one, and thus both approaches would need to be supported.

[skynavga]

@palemieux agreed; both approaches are reasonable, and clearly both have been implemented in different contexts;

[akiseino]

@palemieux @skynavga I also agreed both approaches to have option for director / production team intent.

[skynavga]

@palemieux @akiseino In the above descriptions of what you have been calling block shear, you have not made it clear which block (areas) you are referring to. In the context of a formatted paragraph, there are three theoretical levels of block areas (from most to least nested):

• line areas
• intermediate block areas
• outer block areas

In the context of TTML, the last two are not distinguished since there is no concept of page or column breaks, so we can say there are two block areas in the context of paragraphs, which I will label thus:

• line areas
• paragraph-level block areas, or just (multi-line) block areas for short

where the line areas are the children of the block areas.

Now, we note that the effect of applying shear to line areas is different than applying shear to block areas. Consider a paragraph having two line areas. If shear is applied to individual line areas, the starting and ending edges of those line areas will have the same position in the inline progression dimension. In contrast, if shear is applied to the outer (multi-line) block area, then the line areas will have different start and end positions.

As a consequence of these differences, we have to consider two possible applications of shear at the block (line or paragraph block) level, and, therefore, a single style property could not be used for both purposes.

I conclude from this that, in the TTML case, THREE potential independent styles are required to support all design choices for shear, which I will call:

• fontShear - applies shear to individual glyph area descendants of a line area
• lineShear - applies shear to individual line area descendants of a paragraph block area
• shear - applies shear to (outer) block areas, i.e., block area ancestors of line areas

The last of these could also be potentially applied to elements generating higher level block areas as well, such as div and body.

There are also other aspects of shear which have not been raised, and which apply to these THREE applications of shear:

• how does shear affect line measure computation?
• how does shear affect inter-glyph spacing?
• how does shear interoperate with the overflow property?

I will briefly describe these points:

Line Measure Computation

When allocating space for a line area, the layout system must take into account the inline progression dimension of the available content rectangle of a line area's parent (paragraph-level block area). If the line measure is held constant, independently from shear, then, enabling line shear will reduce the available measure (in order to wholly contain the sheared line area), and, therefore, may result in different line break decisions or different whitespace adjustments (e.g., when using justified text alignment).

The same consideration holds when applying font shear to individual glyph areas, since at shear segment and line break points, the shear angle will have the effect of increasing the overall IPD of the glyph area, i.e., the last (spacing) glyph on the line will require an additional fontSize*sin(θ) increase in IPD.

Inter-Glyph Spacing

When applying shear to individual (spacing) glyph areas, it is possible (likely) that not all glyph areas on a line will share the same shear angle; consequently, an adjustment in inter-glyph spacing is necessary to handle these boundary conditions in order to prevent glyph bounding box overlap or excessive inter-glyph spacing.

There is also the case of a boundary between vertical and horizontal glyph layout, for example, as in the case of tate-chu-yoko as supported by textCombine. In such boundary cases, a shear applied to glyphs on one or the other side of such a boundary must be take into account to prevent glyph bounding box overlap or excessive whitespace.

Handling Overflow

If overflow is set to visible, then it may be permissible to allow shear effects to extend outside the nominal line area, in which case adjustments to line measure computation may not be required. Otherwise, if overflow is hidden, then such adjustments may be needed to prevent overflow due to shear effects.

In conclusion, if we are to define either (or both) tts:lineShear or (and) tts:shear, we will need to address these points, or leave them implementation dependent for the present.

I am concerned that introducing either of these new styles will result in a delay in the TTML2 publishing schedule. In particular, if we have to publish more CRs after CR2, then we will be in danger for reaching REC this year.

[cconcolato]

We also think that the behavior of fontShear on tate-chu-yoko or ruby has to be the one indicated in "sheared as a block".

[palemieux]

Thanks for the analysis!

In the above descriptions of what you have been calling block shear, you have not made it clear which block (areas) you are referring to.

I had the the block generated by the p in mind.

shear - applies shear to (outer) block areas, i.e., block area ancestors of line areas

I have not seen a need to apply it to a block area other than that generated by a p,

Line Measure Computation

Not affected in the case of shear, right?

Inter-Glyph Spacing

Not affected in the case of shear, right?

Otherwise, if overflow is hidden, then such adjustments may be needed to prevent overflow due to shear effects.

tts:overflow does not control impact whether there is overflow, only whether the overflow should be hidden. I therefore do not see the value of tts:overflow resulting in adjustments.

In conclusion, if we are to define either (or both) tts:lineShear or (and) tts:shear,

I would focus on tts:shear

I am concerned that introducing either of these new styles will result in a delay in the TTML2 publishing schedule. In particular, if we have to publish more CRs after CR2, then we will be in danger for reaching REC this year.

imscJS implements tts:shear (see alpha build at http://sandflow.com/imsc1_1/index.html). I do not see a way to implement tts;fontShear in CSS -- other than a fallback to "oblique".

[skynavga]

On Sun, May 27, 2018 at 9:43 AM, Pierre-Anthony Lemieux < ***@***.***> wrote: Thanks for the analysis! In the above descriptions of what you have been calling block shear, you have not made it clear which block (areas) you are referring to. I had the the block generated by the p in mind.

So you are fine with having each line in a sheared (outer) block starting at a different offset from the start edge of the outer block? Because that is what you will get by shearing the outer block. None of the examples I have seen (like D-cinema) show more than one line in a block, so I can't tell if they intend line shear or (outer) block shear.

…

shear - applies shear to (outer) block areas, i.e., block area ancestors of line areas I have not seen a need to apply it to a block area other than that generated by a p, Line Measure Computation Not affected in the case of shear, right? Inter-Glyph Spacing Not affected in the case of shear, right? Otherwise, if overflow is hidden, then such adjustments may be needed to prevent overflow due to shear effects. tts:overflow does not control impact whether there is overflow, only whether the overflow should be hidden. I therefore do not see the value of tts:overflow resulting in adjustments. In conclusion, if we are to define either (or both) tts:lineShear or (and) tts:shear, I would focus on tts:shear I am concerned that introducing either of these new styles will result in a delay in the TTML2 publishing schedule. In particular, if we have to publish more CRs after CR2, then we will be in danger for reaching REC this year. imscJS implements tts:shear (see alpha build at http://sandflow.com/imsc1_1/index.html). I do not see a way to implement tts;fontShear in CSS -- other than a fallback to "oblique". — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#784 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXCb0hpTXKkuKJO-PN96Hh635m6AKQTks5t2smtgaJpZM4UHtAv> .

[skynavga]

On Sun, May 27, 2018 at 1:18 PM, Pierre-Anthony Lemieux < ***@***.***> wrote: Wouldn't that require the author to know where the implementation would place line breaks during normal line layout processing? Yes. I was under the impression that line breaks had to be reasonably well-controlled for Japanese subtitles to look acceptable... but maybe not? Or does imscJS pass an entire paragraph to CSS and let it do line breaking? imscJS needs to apply individual styles to lines, e.g. linePadding, rubyPosition, etc... so it needs to determine line breaks, and could conceivably apply shear per line, although I have not tried it and cannot anticipate side effects.

At this point, I think we don't have adequate community input on whether outer block shearing is preferred over line area shearing. Have you seen any D-cinema examples that have multiple lines? As an aside, how does imscJS determine line breaks? Does it use HTML/CSS to format a paragraph and then (somehow) extract the line breaks from the results?

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#784 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXCb7etrWgyC08RbdYq0uQ-HAmfl7Hgks5t2vv0gaJpZM4UHtAv> .

[palemieux]

Have you seen any D-cinema examples that have multiple lines?

More input would be good, and we can give ourselves 1-2 weeks for such input.

I have seen examples with ruby annotations before and after, but not with multiple lines (in the CSS/XSL sense).

If you are uncertain, you can always specify both line- and p-based shear, and we can sort it out in IMSC.

What is getting clearer in my mind is that glyph-based shear is not going to be sufficient.

As an aside, how does imscJS determine line breaks? Does it use HTML/CSS to
format a paragraph and then (somehow) extract the line breaks from the
results?

Yes.

[skynavga]

On Sun, May 27, 2018 at 2:04 PM, Pierre-Anthony Lemieux < ***@***.***> wrote: Have you seen any D-cinema examples that have multiple lines? More input would be good, and we can give ourselves 1-2 weeks for such input. I have seen examples with ruby annotations before and after, but not with multiple lines (in the CSS/XSL sense). If you are uncertain, you can always specify both line- and p-based shear, and we can sort it out in IMSC. What is getting clearer in my mind is that glyph-based shear is not going to be sufficient.

I think we have general consensus on this last point. I will create a PR that adds both tts:lineShear and tts:shear, marking both as at risk.

As an aside, how does imscJS determine line breaks? Does it use HTML/CSS to format a paragraph and then (somehow) extract the line breaks from the results? Yes.

I'm curious. How do you get the line break character offsets back from HTML/CSS/JS? Do you perform multiple formats by adding a character at a time until you see when the height of the resulting block is increased (meaning that a new line area was added)?

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#784 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXCb1Fz3lQPqrHIzKzbAQVxSkwpPMy4ks5t2wa4gaJpZM4UHtAv> .

[palemieux]

I'm curious. How do you get the line break character offsets back from HTML/CSS/JS?

Wrap every character in a <span> and query their bounding boxes.

[skynavga]

On Sun, May 27, 2018 at 2:23 PM, Pierre-Anthony Lemieux < ***@***.***> wrote: I'm curious. How do you get the line break character offsets back from HTML/CSS/JS? Wrap every character in a <span> and query their bounding boxes.

Nice. BTW, that method will (or may) not work with ligatures, e.g. ff vs , non-spacing characters, and joining scripts (e.g., Arabic). Chrome and Safari have a long-standing bug (from KHTML) the breaks apart joining Arabic characters if there is a span boundary.

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#784 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXCbz2ZyJU8R9TwkW37SBJw5C12Z1q9ks5t2wtbgaJpZM4UHtAv> .

[palemieux]

Chrome and
Safari have a long-standing bug (from KHTML) the breaks apart joining
Arabic characters if there is a span boundary.

Yes.

Hopefully uptake of captions on the web will encourage UA vendors to implement needed features and fix their bugs.

[akiseino]

@palemieux @skynavga
Here is my input for below.

None of the examples I have seen (like D-cinema) show more than one line in
a block, so I can't tell if they intend line shear or (outer) block shear.

We made shearing sample of vertical Japanese subtitle by "Font Italic" for Digital Cinema (XML).

First one is one line in line areas.

Second one is two lines in each line areas.

Third one is two lines in block.

Second one and Third one seems same result.
I hope it would be help for you.

[palemieux]

@akiseino Thanks for the submissions.

None of the examples I have seen (like D-cinema) show more than one line in
a block, so I can't tell if they intend line shear or (outer) block shear.

Yes, unfortunately line wrapping is not supported by the D-Cinema standard.

[skynavga]

@akiseino @palemieux this would seem to argue that we use line shear, and not outer (paragraph level) block shear, although from Pierre's comment I gather that in D-Cinema these are the same (due to lack of line wrap functionality)

[cconcolato]

@skynavga can you clarify if the inline areas generated by the ruby annotations are considered to be part of the line area? i.e. how the lineShear would apply?

[palemieux]

this would seem to argue that we use line shear, and not outer (paragraph level) block shear, although from Pierre's comment I gather that in D-Cinema these are the same (due to lack of line wrap functionality)

Yes there is no automatic line wrap in cinema.

The key question is my mind is whether two-line shear is regularly used, and, if so, is the expected result this:

or this:

or perhaps both are tolerable.

[skynavga]

@cconcolato yes, inline areas generated for the purpose of presenting ruby and emphasis are children of the line area with which they are associated; however, they are special in the sense that they do not consume space in the inline progression dimension but consume space in the block progression dimension

[skynavga]

@palemieux your example shows what I call (outer) block shear in the first image and line shear in the second image; I consider them to be independent, just as I consider font (glyph) shear to be independent; conceivably, all three could apply in the same multi-line paragraph, and would involve three shear (or skew) transforms to render

[cconcolato]

inline areas generated for the purpose of presenting ruby and emphasis are children of the line area with which they are associated;

Can the above be added to the spec?

however, they are special in the sense that they do not consume space in the inline progression dimension but consume space in the block progression dimension

I thought this had to be taken care of when setting tts:lineHeight?

[skynavga]

Merged early per WG resolution.