LWSP in attribute and value expressions?

[palemieux]

I apparently missed the three day window on #310.

As I understand it, and please correct me if I am wrong, expressions of the form rgba( 233, 1,0 ) are now permitted.

If so, this will require TTML1 implementations to be revised and tested, without stated benefits as far as I can tell.

[skynavga]

It turns out that XSL-FO and CSS2 permits this. Furthermore, this is consistent with behavior regarding LWSP appearing around delimiters that goes back to the original CSS2 YACC grammar. However, I'm open to having a discussion about this point.

[palemieux]

To minimize impact on implementations and divergence from TTML1, I would not introduce LWSP if another delimiter is already present between adjacent non-terminal components.

The TTML1 test file FontFamily009.ttml is the first TTML document I have seen with extra spaces added where another delimiter is already present between adjacent non-terminal components -- it is not clear if the space was intended or not.

[palemieux]

Furthermore, TTML1 explicitly states in 8.3 that in the syntax representations defined in this section, no linear whitespace (LWSP) is implied or permitted between tokens unless explicitly specified

TTML2 should do the same unless the absence of LWSP has proven an obstacle to the use of TTML, something I have seen no evidence of.

[skynavga]

@palemieux please review the language found in the preambles of TTML2 8.3, 9.3, 10.3, and 13.3; the restriction in these sections

[andreastai]

Although there are the benefit to get better compatibility with other specs, it could make (as @palemieux commented) a document that uses an otherwise unchanged feature of TTML1 non-conformant against TTML1. A TTML processor would possibly fail processing a document where whitespace are added around "non space" separator. If for example @tts:color is set to rgb(255 ,255,255) this should also fail validation (e.g. it fails validation by ttv modul of the Timed Text Toolkit), so the document may already be rejected during a QC check.

Apart from that, the affected features in TTML2 (as #color) could not have the same feature designator in TTML1 (but this seems currently be the case).

Taking the missing backward compatibility into account the change brings more disadvantages than advantages and should possibly be reverted.

[skynavga]

My position is that we should allow a less restricted syntax for whitespace in all COMMA separated lists in TTML2. Let's review what uses a comma separated list:

TTML1

• @tts:color - in rgb() and rgba() expressions
• @tts:fontFamily

In both these cases, it can be argued that lwsp is permitted, based on XSL-FO and CSS2 syntax; however, there is scant evidence that implementations support lwsp in tts:color, but there is evidence of support for lwsp in tts:fontFamily. As a consequence, TTML1 may be considered both ambiguous and inconsistent in treatment of lwsp around COMMA as a delimiter.

TTML2

• @condition - in function argument lists
• @keySplines - tt:animate specific attribute
• @range - in unicode range - tt:font specific attribute
• @tts:border - radii() argument list
• @tts:color - rgb(), rgba() argument lists
• @tts:fontFamily

In addition, TTML2 makes use of a number of SEMICOLON separated lists:

• @keySplines - tt:animate specific attribute
• @keyTimes - tt:animate specific attribute
• @tts:* - from <animation-value-list> specific to tt:animate

We need to ensure that we resolve any ambiguities about lwsp in these contexts, and we need to apply a single set of rules for consistency (and reduce special cases in code and test content).

[nigelmegitt]

We need to ensure that we resolve any ambiguities about lwsp in these contexts, and we need to apply a single set of rules for consistency (and reduce special cases in code and test content).

+1

[nigelmegitt]

This is somewhat related to #191 - one of the proposals I made there is to add feature designators for the support/non-support in documents/processors of additional lwsp. I'm not really delighted by that bifurcation but it would at least allow profiles and processors to be clear about what they do and do not deal with.

[andreastai]

We need to ensure that we resolve any ambiguities about lwsp in these contexts, and we need to apply a single set of rules for consistency (and reduce special cases in code and test content).

For sure it is not a question with a straight forward answer. To be more consistent is always good. To break existing implementations may be worse. To have an additional "whitespace" feature as @nigelmegitt suggests maybe a way out (depending on what exactly goes into it).

In any case the use of whitespace in comma separated value lists for attributes need to be clarified for TTML1.

@skynavga @nigelmegitt Do you also propose an Errata on this for TTML 1 and if yes, what would it say?

[palemieux]

We need to ensure that we resolve any ambiguities about lwsp in these contexts

TTML2 is not broken without LWSP around delimiters AFAIK, so I do not see a clear motivation to add them. The downside is significant amount of additional testing and a bifurcation with TTML1.

[andreastai]

Regarding the use of spaces in tts:fontFamily: If you take the following text from Section 8.2 in TTML...

Unless explicitly stated otherwise, linear white-space (LWSP) must appear between adjacent non-terminal components of a value of a TT Style or TT Style Extension Property value unless some other delimiter is permitted and used.

...I can see how you may come to the conclusion that if other delimiters are used (e.g. the COMMA) then spaces are not allowed (because they are neither required nor explicitly allowed).

Unfortunately the TTML1 spec fells short here and is not unambiguous. But regardless of the "legal" discussion it is clear that it led to different interpretations.

At least for tts:fontFamily I had always the assumption that space in comma separated lists of atomic values are allowed. Others had the same interpretation. So for example the TTML profile EBU-TT-D-Basic-DE has samples with spaces (page 3 and 16). According to this specifications the open source framework SCF also uses spaces (see the corresponding XSLT, Code line 66). As some broadcasters use this framework for the subtitles of their online playout there are already a lot of files with spaces in this attribute.

But also others had the same interpretation. After a first check I found for example the BBC Subtitle Guidelines where you can find an example with three font family names in one attribute and in addition to the COMMA separator there are spaces (28.5.1 tts:fontFamily, Document Requirements.

[nigelmegitt]

Thanks @TairT for the additional data points. I'm beginning to feel that there are enough examples of spaces around commas in tts:fontFamily that we cannot now prohibit them in TTML1 or TTML2.

How about this proposal:

• in TTML1 we add an informative note saying that both interpretations exist, and
• in TTML2 we explicitly permit spaces around list delimiters.

?

[palemieux]

Re: TTML1, I see two options:

• forbid LWSP around commas in document (SHALL NOT be present), but recommend that processors accept them (SHOULD accept them). The idea is encourage the authoring of most compatible documents, but encourage processors to be tolerant given the existence of documents with LWSP in them

• explicitly permit LWSP around commas of tts:fontFamily components only, and recommend not using them (SHOULD NOT be present) since some implementations might not accept them

Feedback on both options should be sought on public-tt.

[palemieux]

in TTML2 we explicitly permit spaces around list delimiters.

Do you mean LWSP around commas of tts:fontFamily only, or something more?

[nigelmegitt]

Do you mean LWSP around commas of tts:fontFamily only, or something more?

My preference would be a single rule applicable across all style attributes, and possibly parameter attributes if applicable, but I could accept a per-attribute restriction if there's a good reason for it.

[palemieux]

but I could accept a per-attribute restriction if there's a good reason for it.

The reason is to avoid burdening implementers and creating confusion by introducing new requirements that do not solve an actual interoperability and/or use case.

The absence of LSWP between rgba components is not a documented problem AFAIK.

[nigelmegitt]

The requirement for no LWSP between rgba components is an interoperability issue with CSS.

CSS3 Color Module Level 3 specifically permits spaces:

White space characters are allowed around the numerical values.

It is a burden on document authors used to CSS to have to learn a new extra restriction that, for example, may prevent CSS color values from being pasted into TTML color style attributes. I expect there are more document authors than implementers, so I would rank their needs higher.

[palemieux]

It is a burden on document authors used to CSS to have to learn a new extra restriction that, for example,
may prevent CSS color values from being pasted into TTML color style attributes. I expect
there are more document authors than implementers, so I would rank their needs higher.

If compatibility with CSS is truly a requirement, then we have much bigger issues than LWSP since we will need to examine every new feature in that light.

Compatibility with CSS has not been an issue re: rgba values so far. I suggest we do not make it an issue now.

[skynavga]

On Wed, May 24, 2017 at 6:56 AM, Andreas Tai ***@***.***> wrote: Regarding the use of spaces in tts:fontFamily: If you take the following text from Section 8.2 in TTML... Unless explicitly stated otherwise, linear white-space (LWSP) must appear between adjacent non-terminal components of a value of a TT Style or TT Style Extension Property value unless some other delimiter is permitted and used. ...I can see how you may come to the conclusion that if other delimiters are used (e.g. the COMMA) then spaces are not allowed (because they are neither required nor explicitly allowed). Unfortunately the TTML1 spec fells short here and is not unambiguous. But regardless of the "legal" discussion it is clear that it led to different interpretations. At least for tts:fontFamily I had always the assumption that space in comma separated lists of atomic values are allowed. Others had the same interpretation. So for example the TTML profile EBU-TT-D-Basic-DE <http://www.irt.de/webarchiv/showdoc.php?z=NjMzOSMxMDA2MDE4I3BkZg==> has samples with spaces (page 3 and 16). According to this specifications the open source framework SCF also uses spaces (see the corresponding XSLT <https://github.com/IRT-Open-Source/scf/blob/master/modules/EBU-TT-D2EBU-TT-D-Basic-DE/EBU-TT-D2EBU-TT-D-Basic-DE.xslt>, Code line 66). As some broadcasters use this framework for the subtitles of their online playout there are already a lot of files with spaces in this attribute. But also others had the same interpretation. After a first check I found for example the BBC Subtitle Guidelines where you can find an example with three font family names in one attribute and in addition to the COMMA separator there are spaces (28.5.1 tts:fontFamily, Document Requirements <http://bbc.github.io/subtitle-guidelines/#tts-fontFamily>.

SKYNAV has always assumed whitespace is permitted around COMMA in tts:fontFamily, which is born out by at least four implementations and numerous test files. However, as I reported previously, we did not make that assumption about tts:color, even though in retrospect we see it is permitted by the same grammar constructs in XSL-FO and CSS2.

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#315 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXCb68q5BJWBPflDT4BIXrtv-NfOAx_ks5r9CjpgaJpZM4NdXR5> .

[skynavga]

On Wed, May 24, 2017 at 8:42 AM, Nigel Megitt ***@***.***> wrote: Thanks @TairT <https://github.com/tairt> for the additional data points. I'm beginning to feel that there are enough examples of spaces around commas in tts:fontFamily that we cannot now prohibit them in TTML1 or TTML2. How about this proposal: - in TTML1 we add an informative note saying that both interpretations exist, and Agreed. - in TTML2 we explicitly permit spaces around list delimiters. There is a somewhat broader question here, which is whether to extend this

to *all* COMMA and SEMICOLON separated lists, including, in particular tts:color. As I have pointed out, we are going from two contexts of use in TTML1 to nine contexts of use in TTML2 (as currently drafted). My position is that we should have a consistent approach in TTML2 meaning that whitespace be allowed around delimiters in all of these nine contexts, including tts:color.

…

? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#315 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXCb2NuPkysa0Amn_EkGjyf7-gRsGABks5r9EHFgaJpZM4NdXR5> .

[skynavga]

On Wed, May 24, 2017 at 9:01 AM, Pierre-Anthony Lemieux < ***@***.***> wrote: Re: TTML1, I see two options: - forbid LWSP around commas in document (SHALL NOT be present), but recommend that processors accept them (SHOULD accept them). The idea is encourage the authoring of most compatible documents, but encourage processors to be tolerant given the existence of documents with LWSP in them this is clearly not feasible since it would invalidate numerous existing

TTML1 documents, i.e., would make conforming documents non-conforming

- - explicitly permit LWSP around commas of tts:fontFamily components *only*, and recommend not using them (SHOULD NOT be present) since some implementations might not accept them yes to the first part, no to the second; I prefer Nigel's suggestion, to

add note indicating both types of implementation exist; further, I would suggest going further and recommend (in a note) that content processors accept whitespace here even though the original specification was ambiguous

- Feedback on both options should be sought on public-tt.

Presumably, readers of public-tt already have visibility to this conversation and can chime in at any time.

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#315 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXCb0tdZTNChPVGJAW89Bk2e_6hukIYks5r9EZIgaJpZM4NdXR5> .

[skynavga]

On Wed, May 24, 2017 at 9:49 AM, Nigel Megitt ***@***.***> wrote: The requirement for no LWSP between rgba components is an interoperability issue with CSS. CSS3 Color Module Level 3 <https://www.w3.org/TR/css3-color/#rgb-color> specifically permits spaces: White space characters are allowed around the numerical values. It is a burden on document authors used to CSS to have to learn a new extra restriction that, for example, may prevent CSS color values from being pasted into TTML color style attributes. I expect there are more document authors than implementers, so I would rank their needs higher.

Agreed

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#315 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXCb6SnwzXgSos-dfey1AtNLxPoFx2iks5r9FF1gaJpZM4NdXR5> .

[palemieux]

the same grammar constructs in XSL-FO and CSS2.

Let's be factual here: they are not the same.

CSS2 has an extensive grammar and lexical scanner specifications Appendix D, there is no equivalent in TTML1.

They are perhaps similar.

[skynavga]

On Wed, May 24, 2017 at 11:38 AM, Pierre-Anthony Lemieux < ***@***.***> wrote: the same grammar constructs in XSL-FO and CSS2. Let's be factual here: they are not the same.

What is your point here? Of course they are not the same documents. But TTML1 makes normative reference to XSL-FO and is clearly based on certain sub-grammars of XSL-FO, which is in turn, based on sub-grammars of CSS2 1998. The syntax of rgb() expressions is one such sub-grammar.

CSS2 has an extensive grammar and lexical scanner specifications Appendix D <https://www.w3.org/TR/REC-CSS2/grammar.html>, there is no equivalent in TTML1.

So what? It makes direct or indirect reference to Appendix D.

…

They are perhaps similar. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#315 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXCb2ZV6U2ZlOwMcvI4bQJK6z5vAol8ks5r9GsTgaJpZM4NdXR5> .

[cconcolato]

In discussing this with @palemieux and @skynavga, we recommend the group to consider the following:

• in TTML1, follow the fontFamily path for color, i.e. say that are allowed (because they were allowed from day 1 in CSS) but not recommend them (because some implementations may not support it)
• in TTML2, allow lwsp everywhere, specified in one global section (e.g. 2.3 conventions).

[skynavga]

@cconcolato suggests that when incorporating whitespace syntax rules into 2.3 that we should sub-divide 2.3 into multiple subsections since 2.3 is getting rather long;

[css-meeting-bot]

The Working Group just discussed LWSP in rgba expressions? ttml2#315, and agreed to the following resolutions:

• RESOLUTION: Specify explicitly where lwsp is permitted.

The full IRC log of that discussion

<nigel> Topic: LWSP in rgba expressions? ttml2#315
<nigel> github: https://github.com//issues/315
<nigel> Cyril: I'll create an issue for TTML1 for the equivalent of this, in color.
<nigel> .. It's w3c/ttml1#322.
<nigel> Nigel: When you say "allow lwsp everywhere", what exactly is "everywhere"? Is it all tta, ttp and tts attributes?
<nigel> Glenn: Pretty much, yes. I'm not proposing to allow lwsp between value names and ( for example.
<nigel> Chris: That matches what is in CSS too.
<nigel> Glenn: I should change `<color>` to combine the `"rgb" "("` into `"rgb("` and `"rgba" "("` into `"rgba("`
<nigel> Nigel: In `<number>`, `<non-negative-number>` and `<percentage>` there shouldn't be any lwsp allowed.
<nigel> Chris: Implementers prefer no space between a number and %.
<nigel> Nigel: So that applies between number and pitch-units in `<pitch>` too.
<nigel> Chris: Yes
<nigel> Cyril: It may be clearer and easier to put `<lwsp>` explicitly everywhere it is allowed.
<nigel> .. We should do that.
<nigel> Glenn: I may have to accept that.
<nigel> Nigel: `<position>` already does it.
<nigel> Glenn: `<condition>` does too.
<nigel> Glenn: There are some animation value expressions too that need it.
<nigel> RESOLUTION: Specify explicitly where lwsp is permitted.

[nigelmegitt]

Note that a partial change was made that does not address this issue, in #525.