Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apply improvements to aspect ratio and pixel semantics. #321

Merged
merged 5 commits into from May 24, 2017

Conversation

palemieux
Copy link
Contributor

This is an initial pass at rationalizing computation of aspect ratio and pixel dimensions.

Some highlights:

  • IMHO the ultimate goal is to determine (a) the intended display aspect ratio of the root container, i.e. what the viewer sees, and (b) the meaning of px within the document coordinate system

  • the mapping of the root container to the presentation viewport is left to profiles and/or applications, but can use the computed PAR, SAR, and DAR.

  • the document coordinate system is specified as an abstract Cartesian coordinate system, i.e. independently of the notion of pixels.

I expect additional editorial will be needed, at the very least, but wanted to get the review started.

@skynavga skynavga self-requested a review May 18, 2017 00:13
@skynavga
Copy link
Collaborator

I'm making an editing pass on this PR and will commit an update later this evening.

spec/ttml2.xml Outdated
@@ -855,7 +855,7 @@ means of a <att>designator</att> attribute or prose text in a specification of p
</def>
</gitem>
<gitem id="terms-display-aspect-ratio">
<label>[display aspect ratio]</label>
<label>[display aspect ratio (or DAR)]</label>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acronyms go into the Acronyms section.

spec/ttml2.xml Outdated
<loc href="#terms-root-container-region">root container region</loc> used to format and present a
<loc href="#terms-document-instance">document instance</loc>, more about which see
<specref ref="root-container-region-semantics"/>.</p>
<p>See <specref ref="root-container-region-semantics"/>.</p>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose of terminology section is to provide some definition, and not just vector to a new location. So this edit is too radical.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.

spec/ttml2.xml Outdated
@@ -1314,7 +1311,7 @@ A <loc href="#terms-default-region">default out-of-line region</loc> is implied
</def>
</gitem>
<gitem id="terms-pixel-aspect-ratio">
<label>[pixel aspect ratio]</label>
<label>[pixel aspect ratio (or PAR)]</label>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto on acronyms.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't the terminology section be the right place to define acronyms?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have an acronyms section. I am creating an edit now to be posted later this evening.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed the acronym section... so sounds good to move it there.

spec/ttml2.xml Outdated
@@ -1508,7 +1505,7 @@ by <specref ref="semantics-style-resolution-processing-sss"/>.</p>
</def>
</gitem>
<gitem id="terms-storage-aspect-ratio">
<label>[storage aspect ratio]</label>
<label>[storage aspect ratio (or SAR)]</label>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto on acronyms.

spec/ttml2.xml Outdated
@@ -5174,7 +5169,7 @@ numerator | demoninator
</tr>
</tbody>
</table>
<p>If not specified, then square pixels (i.e., aspect ratio 1:1) must be assumed to apply.
<p>If not specified, then both <code>numerator</code> and <code>demoninator</code> must be assumed to be equal to 1.
Copy link
Collaborator

@skynavga skynavga May 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is unnecessary and possibly overconstrained. In particular, the values of the numerator and denominator may be some other value than 1/1, e.g., 3/3 is equivalent. Reverting this change.

spec/ttml2.xml Outdated
attribute is specified, then the spatial extent of the <loc href="#terms-root-container-region">root container region</loc> is
considered to be determined by the <loc href="#terms-document-processing-context">document processing context</loc>.
The origin of the <loc href="#terms-root-container-region">root container region</loc> is determined by the <loc href="#terms-document-processing-context">document processing context</loc>.</p>
The origin of the <loc href="#terms-root-container-region">root container region</loc> is determined by the <loc href="#terms-document-processing-context">document processing context</loc>.
-->
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't dispense with this language, am revising slightly to refer to root container region semantics appendix.

spec/ttml2.xml Outdated
which has the effect of defining
the <loc href="#terms-document-coordinate-space">document coordinate space</loc>,
about which see also <specref ref="root-container-region-semantics"/>;</p></item>
of the <loc href="#terms-root-container-region">root container region</loc>;</p></item>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revising this change as it removes useful information, while adding nothing.

spec/ttml2.xml Outdated
</ulist>
</item>
<item><p>otherwise, a <code>px</code> corresponds with the semantics defined by <bibref ref="xsl11"/>, &sect; 5.9.13, and its mapping to the document coordinate space is undefined.</p></item>
</ulist>
Copy link
Collaborator

@skynavga skynavga May 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to vector this definition to root container region semantics sub-section on coordinate space rather than attempt to define in this context. I will revise accordingly.

spec/ttml2.xml Outdated
<item><p>aspect ratios (display, storage, pixel)</p></item>
<item><p>resolution</p></item>
<item><p>coordinate space</p></item>
</ulist>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revering this removal, as this is useful information for reader.

spec/ttml2.xml Outdated
<p>The coordinate space of the root container region, also known as the <loc href="#terms-document-coordinate-space">document coordinate space</loc>, is a cartesian coordinate system with:</p>
<ulist>
<item><p>its origin coincident with the upper left-hand corner of the <loc href="#terms-root-container-region">root container region</loc>; and</p></item>
<item><p>the point with coordinates (1, 1) coincident with the lower right-hand corner of the <loc href="#terms-root-container-region">root container region</loc>.</p></item>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is clearly wrong.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is wrong?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the coordinates of the lower right hand corner are <1,1>, then the entire coordinate space has only one pixel. Coordinates are in pixels, not percentages.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am close to committing an update on this branch, so please stand by for further changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point is that the coordinate system should not be in pixels since pixels are meaningless if tts:extent is not specified on tt.

Copy link
Collaborator

@skynavga skynavga May 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may have forgotten that TTML always assumes the coordinate space is in pixels, and that a document can use px units even without specifying SAR, i.e., without specifying tts:extent on <tt>.

I note that IMSC and a few other profiles rule out this scenario, but it still applies to TTML as such.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may have forgotten that TTML always assumes the coordinate space is in pixels,

Without extent in px when dimensions are expressed in px, it is not possible to determine the relative dimensions of a region with (50%,50%) extent and one with (20px, 20px) extent.

This has been an issue with TTML1, and should be fixed in TTML2.

Copy link
Collaborator

@skynavga skynavga May 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really. In this case, the document processing context can choose a resolution, say, e.g., the resolution of a related media object, or in case of no media, an arbitrary resolution, e.g., "1000px 1000px". Having done so, it is straightforward to resolve both [50%,50%] and [20px,20px]. If the results are unexpected, then it is the authors fault for failing to specify a resolution in the first place. We can always add a strong warning against the use of pixels without specifying resolution, but for backwards interoperability, we can't change TTML1 retroactively. The bottom line is that processing behavior is well defined, even in the absence of sensible authorial input (on resolution).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't agree that TTML assumes the coordinate space is in pixels. It may well define semantics should an implementation make that assumption, but it also seems entirely possible (as well as a good idea in some cases) to author and present TTML documents without making references to logical TTML pixels anywhere. For example rw and rh units, or cell units, can be used throughout, and mapped to a vector image format on output that also only uses relative units.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolution pending.

spec/ttml2.xml Outdated
<td>computed value of ttp:pixelAspectRatio</td>
</tr>
</tbody>
</table>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was expecting only that the existing specification text would be corrected if there was a substantive difference in technical understand of the desired results as we discussed in the London F2F. I did not expect a wholesale replacement of the existing text with a table that is not only difficult to interpret, but introduces a number of ambiguities, such has how to resolve over-constrained specified values. I am restoring the previous text and then editing it as needed to incorporate the results from the F2F in a manner I hope is consistent with the semantics of your replacement table.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but introduces a number of ambiguities,

What ambiguities?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See "such as ...".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In absence of a specific example, tt sounds like you did not even try to read it.

Copy link
Collaborator

@skynavga skynavga May 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There you go again, accusing me of bad faith. Of course I read it. More than once. For example, your table says that PAR is DAR/SAR when both DAR and SAR are specified; however, if PAR is also specified, then this is ambiguous in the case that PAR is not DAR/SAR. Your table also appears to imply that tts:extent on <tt> may be lengths that use non-pixel units, but it is required to be in pixels if specified as lengths.

From TTML2 (10.2.16)

If a tts:extent attribute is specified on the tt element, then the specified value is restricted to one of the following: (1) the auto keyword, (2) the contain keyword, or (3) two <length> specifications, where these specifications are expressed as non-percentage, definite lengths using pixel units. All other syntactically legal values must not be used in this context, and, if used, must be considered an error for the purpose of validation processing and must be ignored for the purpose of presentation processing, in which case the initial value (auto) applies.

From TTML1 (8.2.7)

If tts:extent is specified on the tt element, then the width and height must be expressed in terms of two <length> specifications, and these specifications must be expressed as non-percentage, definite lengths using pixel units.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

however, if PAR is also specified, then this is ambiguous in the case that PAR is not DAR/SAR.

In London, we had decided that DAR/SAR would override the specified PAR value.

Copy link
Contributor Author

@palemieux palemieux May 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your table also appears to imply that tts:extent on <tt> may be lengths that use non-pixel units,

As it stands it can also be: 1) the auto keyword or 2) the contain keyword.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't disagree that in that case PAR is overridden by DAR/SAR, but your table doesn't say that, and the reader may be confused. In any case, that was what I meant by ambiguity (as well as the non-pixel lengths on extent).

Yes, I agree that auto and contain are possible values, but your table didn't say that, so a reader might have thought that extent with a non-pixel length could apply.

@skynavga skynavga changed the title Initial pass at rationalizing aspect ratio and pixel dimensions Apply improvements to aspect ratio and pixel semantics. May 21, 2017
@skynavga
Copy link
Collaborator

@palemieux edit applied to branch.

@skynavga
Copy link
Collaborator

Closes #30.

@skynavga
Copy link
Collaborator

@palemieux do you have any comments on the PR edits? what I'm looking for primarily is if you have any technical issues with the edits with respect to the table you proposed as a replacement, and, more specifically, whether you feel that my incorporation of the table's semantics into the earlier text is consistent with your mental model of resolving aspect ratios.

@palemieux
Copy link
Contributor Author

@skynavga It is on my todo list.

spec/ttml2.xml Outdated
<p>Three aspect ratios apply to the <loc href="#terms-root-container-region">root container region</loc>:</p>
<glist>
<gitem>
<label>display aspect ratio (DAR)</label>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DAR acronym is already defined in the acronym section.

spec/ttml2.xml Outdated
<label>display aspect ratio (DAR)</label>
<def>
<p></p>
<p>The <emph>display</emph> aspect ratio of the root container corresponds with the displayed aspect ratio of the root container, i.e.,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aspect ratio is already defined in the definition. No need to repeat it here.

spec/ttml2.xml Outdated
<p>The <emph>display</emph> aspect ratio of the root container corresponds with the displayed aspect ratio of the root container, i.e.,
the actual (or intended) physical aspect ratio when it is (to be) presented on a visual medium;
this <emph>display</emph> aspect ratio may be specified by the
<loc href="#parameter-attribute-displayAspectRatio">ttp:displayAspectRatio</loc> attribute on the
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why try to list all ways that aspect ratio can be computed when the following explains it in detail. This is likely to introduce inconsistencies and make the document more difficult to maintain.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two parts work in concert: the first part defines how specified values of aspect ratios are determined, the second part defines how resolved values of aspect ratios are determined. It is a two pass process, like determining specified and then computed values of style properties. If you believe there is a technical inconsistency, then please share that information.

spec/ttml2.xml Outdated
<def>
<p></p>
<p>The <emph>storage</emph> aspect ratio of the root container corresponds with the logical aspect ratio of the root container
in storage or sample units;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are "storage or sample units"? Neither terms are used or defined anywhere else? Why is this relevant to the processing of the document?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The term storage aspect ratio, as well as its synonyms or near-synonyms, frame aspect ratio, image aspect ratio, and picture aspect ratio, typically make referent to storage units and/or sample units, namely, the units of information that correspond to pixels in the context of a stored frame/image/picture. I suppose we could substitute "logical pixels" in this context, since they have the same intended meaning here.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replacing with logical pixels in pending fix.

spec/ttml2.xml Outdated
<p>When the <loc href="#style-attribute-extent">tts:extent</loc> attribute is specified on the
<loc href="#document-structure-vocabulary-tt"><el>tt</el></loc> element, then the <code>SAR</code> is assumed to have been specified as
follows:</p>
<olist>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The means of determining DAR, SAR and PAR need to be in a single place and not sprinkled across the document, which makes reading it more complex and increases the chances of inconsistencies.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix pending (by vectoring definitions in terminology section to Appendix H).

spec/ttml2.xml Outdated
<def>
<p></p>
<p>The <emph>display</emph> aspect ratio of the root container corresponds with the displayed aspect ratio of the root container, i.e.,
the actual (or intended) physical aspect ratio when it is (to be) presented on a visual medium;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the difference between the visual medium and the presentation viewport?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The term visual medium is a term of art that goes back to CSS1 961217, where it is assumed and contrasted with non-visual media, expanded further in CSS2 980512 [2] as visual media group.

I am removing the term presentation viewport (and all other viewport related terms) since they are no longer referenced in the specification text.

[1] https://www.w3.org/TR/REC-CSS1-961217#appendix-e
[2] https://www.w3.org/TR/1998/REC-CSS2-19980512/media.html#media-groups

spec/ttml2.xml Outdated
<def>
<p></p>
<p>The <emph>pixel</emph> aspect ratio of the root container corresponds with the actual (or intended) physical aspect ratio of
each individual physical (as opposed to logical) pixel of the root container when it is (to be) presented on a visual medium;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PAR cannot be the aspect ratio of the pixel on the presentation medium since these are completely out of the control of the author. The PAR can only be the aspect ratio of the logical pixel used by "px" measurements, right?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix pending, by disconnecting display pixels from post-TTML processing.

spec/ttml2.xml Outdated
left-hand corner of <emph>R</emph>, and where positive pixels on the vertical axis extend downwards and positive pixels on the horizontal axis
extend rightwards.
Furthermore, the <loc href="#terms-width">width</loc> and <loc href="#terms-height">height</loc> of <emph>R</emph> is set to the
resolved resolution of the root container region.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the resolved resolution of the root container region determined? Is that in Section H.2?

spec/ttml2.xml Outdated
The origin of the <loc href="#terms-root-container-region">root container region</loc> is determined by the <loc href="#terms-document-processing-context">document processing context</loc>.
-->
</p>
determine by rules specified in <specref ref="root-container-region-semantics"/>.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Appendix H does not use the term "spatial extent" once.

Copy link
Collaborator

@skynavga skynavga May 24, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix pending.

spec/ttml2.xml Outdated
<olist>
<item><p>if a <loc href="#terms-related-media-object">related media object</loc> exists and has a defined <code>DAR</code>,
then the resolved value of <code>DAR</code> is the display aspect ratio of the
<loc href="#terms-related-media-object">related media object</loc> and the value of <code>SAR</code>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is application dependent and the DAR needs to be determined by the document processing context, even if a Related Media Object exists, which may not even be video.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix pending.

@skynavga
Copy link
Collaborator

skynavga commented May 24, 2017 via email

@skynavga skynavga merged commit ec0f1e1 into gh-pages May 24, 2017
@skynavga skynavga deleted the issue-0030-aspect-ratios-and-pixels branch August 21, 2017 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants