Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

foreign namespace usage is underspecified #213

Closed
mikedo opened this issue Feb 10, 2017 · 13 comments

Comments

Projects
None yet
4 participants
@mikedo
Copy link

commented Feb 10, 2017

The only conformance text on the subject is:

6.2 Foreign Element and Attributes
A Document Instance MAY contain elements and attributes that are neither specifically permitted nor forbidden by a profile.
A transformation processor SHOULD preserve such elements or attributes whenever possible.

G and H have some brief informative material.

The XML schemas are informative and make various assumptions not traceable to the spec.

FYI, TTML has no conformance language like either the above or the following additional provisions that I believe are needed for document interop when extending it with foreign namespaces.

  1. Where exactly can they be used? Everywhere?
    a. Specifically for , do they follow, lead, or can they be placed between any elements?
  2. What namespaces are permitted? Is ##other assumed or ##any?
  3. What processContents setting is to be used(this affects the validation behavior on future schemas and extended documents)?
  4. (probably other things I have not thought about like union)

@mikedo mikedo changed the title foreign namspace usage is underspecified foreign namespace usage is underspecified Feb 10, 2017

@skynavga

This comment has been minimized.

Copy link
Contributor

commented Feb 11, 2017

@mikedo

This comment has been minimized.

Copy link
Author

commented Feb 11, 2017

The cited text from TTML at best hints that foreign namespaces were contemplated. There is zero conformance language to support their inclusion in TTML. And, if it is already clearly enabled in TTML, then there would have been no need for the existing IMSC1 conformance language I cited. And, the fact that there is a disclaimer about XML Schemas being imperfect for validation, although true, is not relevant. This issue is about where foreign namespaces can be placed and what happens to derivative work schema processing when IMSC1 is extended (processContents affects the addition of foreign namespace extensions, not the base TTML and IMSC1). All this said, I suspect we may agree on the final answer. I will try to draft some specific "clarifying" text to test that theory...

@nigelmegitt

This comment has been minimized.

Copy link
Contributor

commented Feb 13, 2017

In TTML1 there is conformance language in §3.1, §3.2 and §4, and also relevant conformance statements in §4.1:

  1. pruning all element information items whose names are not members of the collection of element types defined by the associated Abstract Document Type, then

and

  1. pruning all attribute information items having expanded names such that the namespace URI of the expanded names are not listed in Table 1 – Namespaces,

Together this means that when processing documents foreign namespace elements and attributes are ignored.

There's something much more obvious for attributes which is that all the elements contain in the description of permitted attributes the following text:

{any attribute not in default or any TT namespace}

I'm not convinced that any further clarification is necessary, but maybe I haven't fully understood the issue that @mikedo is raising.

@mikedo

This comment has been minimized.

Copy link
Author

commented Feb 13, 2017

By "conformance" statements, I mean sentences with the conformance terms, (e.g. "shall, should, or may") to enable foreign namespaces. The cites that you and Glenn keep providing are about instance document conformance steps including the removal of foreign namespaces, only inferring foreign namespaces could exist, not enabling them. Where does it say: "Foreign namespace elements may be present anywhere (including every position in a )"? In fact the situation is worse than this since there is one instance where a foreign element is explicitly permitted (metadata, 12.1.1) thus more clearly communicating that they are not, in fact, permitted elsewhere. I simply don't see how a reader could infer that foreign namespace elements can be used everywhere..

Yes, foreign attributes are clearly permitted where that statement is present. I include attributes in this discussion since they are affected by processContents. Sorry for not being more clear.

@skynavga

This comment has been minimized.

Copy link
Contributor

commented Feb 14, 2017

@nigelmegitt

This comment has been minimized.

Copy link
Contributor

commented Feb 23, 2017

TTWG Meeting 2017-02-23: Agreed that @mikedo will draft a proposed informative paragraph explaining that foreign namespace elements are permitted on content elements.

@palemieux

This comment has been minimized.

Copy link
Contributor

commented Feb 24, 2017

I had the opportunity to review TTML1 following the telecon.

I am now thinking TTML1 is unambiguous: the XML Representation and prose for content elements clearly indicate that {any attribute not in default or any TT namespace} are permitted but that children element are within a very specific set, e.g. Metadata.class*, Animation.class*, div* for body. Only metadata allows ({any element in TT Metadata namespace}|{any element not in any TT namespace})*.

This is compatible with extension attributes and metadata elements introduced by SMPTE-TT and EBU-TT-D and IMSC1, AFAIK.

[EDIT: There is apparently an error in ST 2052-1 that implies that smpte:information is a direct child of head]

If anything, this means that the prose in IMSC1 (A Document Instance MAY contain elements and attributes that are neither specifically permitted nor forbidden by a profile ) is perhaps misleading since it implies that foreign elements can be present anywhere.

@nigelmegitt

This comment has been minimized.

Copy link
Contributor

commented Mar 2, 2017

I have also reviewed TTML1 and IMSC, and have lead myself down the following logical path (which I am happy to be challenged on!):

TTML1 §4 says that the first step in establishing validity is to prune element information items not in the collection of elements defined by the associated abstract document type.

The abstract document type for, say, the tt element or the head element, excludes elements defined in non-TTML namespaces. Therefore such elements are not in the collection of elements and are pruned for validity checking. This is not a prohibition of them being present, just a defined behaviour (prune them) when they are present.

The metadata element differs however in that those "foreign" namespace elements are included in the collection of elements in the abstract document type and are therefore not pruned. However it is left unstated how their validity should be assessed. The clause in §4 "the descendants of the document element satisfy their respective element type's content specifications," suggests that foreign namespace element types have some content specification somewhere but goes no further, which is probably reasonable. It does mean that validity checking is required on foreign namespace elements if and only if they are descendants of metadata elements.

Presumably any validator should inform when pruning foreign namespace element content but not warn about it, since it is excluded from the validity checking, but should warn about non-pruned foreign namespace elements (descendants of metadata) for which there is no content specification and therefore no validity checking is possible.

Presumably also any presentation processor will also ignore foreign namespace elements and also any metadata elements, since they are not expected to affect presentation semantics.

Now in the case of IMSC1 we are no longer talking about an abstract document instance but a concrete encoding. So the question becomes: should we permit within a concretely encoded document foreign namespace elements? And secondarily, if we do permit them, what should a processor do?

On the first question I am a little torn each way.

On the "yes, permit foreign namespace elements anywhere" side:

  • permitting foreign namespace elements allows for "mix ins" where that's deemed appropriate, which seems like a helpful thing (and is something that I would in general like to support, e.g. the use of EmotionML to add more information about elements, which could be in or out of the metadata space);
  • there is one data point of an example where this is explicitly specified, in SMPTE-TT. The SMPTE ST2052-1-2010 specification defines a smpte:information element that is required to be a child of the head element. So saying yes means it may be possible to create SMPTE-TT documents with this element that are also conformant IMSC1 documents.
  • since IMSC 1 already says "A Document Instance may contain elements and attributes that are neither specifically permitted nor forbidden by a profile." it follows that conformant processors should already deal with this scenario.

On the "no, only permit foreign namespace elements under metadata" side:

  • there may be (non-conformant?) IMSC 1 processors out there that will fail on unexpected content;
  • there are certainly profiles that prohibit it explicitly, such as EBU-TT and EBU-TT-D;
  • it does not preclude conformant processors from being forgiving by pruning or ignoring foreign namespace elements elsewhere if they find them.
  • it is slightly simpler to implement since no pruning is required.

I'm swayable on this but on balance it seems like the best thing is to continue with the existing phrasing in IMSC 1 §6.2 Foreign Elements and Attributes, i.e. make no substantive change, but possibly to strengthen it by adding some clarification points, in notes or normative text:

  • Foreign elements and attributes are excluded from validation as IMSC 1 document by processors except when present as descendants of the metadata element.
  • When validating foreign namespace element descendants of the metadata element a content specification should be provided for validation purposes.
  • When validating foreign namespace attributes a content specification should be provided for validation purposes.
  • Foreign namespace elements and attributes shall be ignored by presentation processors for the purpose of providing strictly conformant IMSC 1 presentation processing semantics; they may be used to modify the presentation where IMSC 1 presentation processing semantics allow for such freedom.

An example of the freedom referred to in the last bullet is if someone wants to add metadata to control the block progression scrolling of lines of subtitles that are added word by word, as opposed to making it a system setting. They could do so in principle by adding foreign namespace elements. I would not support defining them in the TTML or IMSC 1 specs however. This freedom of interpretation is explicitly called out in https://www.w3.org/TR/ttml1/#semantics-smooth-scrolling-recommendation by the way.

@skynavga

This comment has been minimized.

Copy link
Contributor

commented Mar 2, 2017

@mikedo

This comment has been minimized.

Copy link
Author

commented Mar 2, 2017

In principle, I agree with Glenn. That said, I (still) believe that TTML1 is quite clear that only metadata elements can contain foreign namespace elements. I do not subscribe to the position that silence is permission when the TTML1 spec is so very, very prescriptive on this topic. There are no known extensions by TTWG or others that violate this today and we should stay the course, clarifying it in IMSC1. I look forward to continuing this live in 23 minutes...

@mikedo

This comment has been minimized.

Copy link
Author

commented Mar 6, 2017

With s/children/descendants, it is not universally true. This explicitly enables nested use of foreign namespaces within foreign namespaces. This may or may not be true depending on each foreign namespace that is defined. For example, METADATA/FOREIGN1:META/FOREIGN2:META is only valid if FORIEGN1 itself permits foreign namespaces.

I'm a bit unclear what this proposal fixes, but in any event, it is now too broad I think without further qualifying text.

@palemieux

This comment has been minimized.

Copy link
Contributor

commented Mar 8, 2017

@mikedo See revised PR

@nigelmegitt

This comment has been minimized.

Copy link
Contributor

commented Jan 10, 2018

Following discussion in F2F today reopening since the group consensus now appears to be that foreign namespace elements are pruned prior to validation processing and therefore are in fact permitted anywhere by TTML. See w3c/ttml1#251.

@nigelmegitt nigelmegitt reopened this Jan 10, 2018

@palemieux palemieux self-assigned this Jan 10, 2018

@palemieux palemieux added the imsc1.0.1 label Jan 10, 2018

@palemieux palemieux modified the milestones: imsc1.0.1 WR, imsc1.0.1 PR Jan 10, 2018

palemieux added a commit that referenced this issue Jan 16, 2018

@palemieux palemieux added the pr open label Jan 16, 2018

@palemieux palemieux removed the pr open label Jan 29, 2018

palemieux added a commit that referenced this issue Jan 30, 2018

palemieux added a commit that referenced this issue Feb 6, 2018

palemieux added a commit that referenced this issue Mar 8, 2018

Port IMSC1.0.1 fixes
* Clarify fillLineGap semantics when applied to successive p elements (#263)
* Editorial tweaks (Close #271)
* Clarify forcedDisplay semantics (#284)
* Clarify the limits of Active Area per its syntax (#288)
* Clarify handling of foreign elements and attributes (#213)
* Clarify name of IMSC namespaces (#301)
* Use Root Container Region consistently (#302)
* Clarify style resolution procedure (#300)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.