Fade in/out

[PabloRFresco]

1. Proposal
   Working collaboratively with filmmakers in the production of subtitles, we have used fade in/outs, but only for open subtitles, burnt on the image, not for closed subtitles.
   We would like to propose that the user can select a subtitles (or a word, line, etc.) and set the duration for fade in/out. Another option would be to allow the use of keyframes. Video editing tools, such as Premiere and After Effects, come with some pre-sets which helps to add effects directly without the need of manually using keyframes. However, the use of keyframes gives more freedom to adjust the effect to your needs by changing the curves of speed to make the movement smooth or abrupt. The pre-set effects normally have a constant speed, but by using the keyframes and their speed curves, the speed can be set to accelerate with time or to decelerate with time, or start slow, accelerate and decrease when reaching the next keyframe (easy ease). For fading effects, the use of keyframes and adjusting the speed may not have a clear direct impact on the result but for other effects, such as changes in position, this is important, as it changes visualization and perception considerably. The user should be able to mark the keyframes where the effect is intended to occur and then be able to change the speed and acceleration (ideally using graphs to edit the curves).

2. Context and evidence
   When we work with filmmakers applying the accessible filmmaking model (integration of translation and media accessibility into the filmmaking process through the collaboration between filmmakers and translators), we present them with a range of options, including creative subtitles, and they choose the options that better suit the aesthetics, tone and content of the film. In the case of the award-winning documentary Notes on Blindness, for example, they asked us to use, amongst other features, fade in and out subs, which we did. The subtitles proved very successful and the viewers loved it. The film was then bought by Netflix (and the BBC, for their iPlayer) and they wanted to use the creative subs, but we could only give them the creative subs burnt in the image, as we didn’t find a way of sending them a closed caption file with all the features we had. While I understand that it may be hard to include all the features of creative subtitles that we used in the current version of TTML, it would be great if fade in/fade out could be considered, as it allows filmmakers to develop their creativity and to produce subtitles that, apart from the content, also carry meaning aesthetically and are in line with their vision of the film, thus ensuring that the foreign and deaf audience have as similar as possible an experience to the original audience.

Here is some evidence and research data to prove or back up the claims made in this message.

example of the creative subtitles used for Notes on Blindness (1:10-1:15 is a good example of the use of fade in/out): https://vimeo.com/archersmarkreview/review/277085498/bee5019e0a

impact campaign of the film Notes on Blindness, including explanation of how the accessible filmmaking model was used: https://drive.google.com/open?id=1_zKxiZ14C0eUNr5EEm0uKZ5olfgJuP3u

first article on accessible filmmaking: : https:/www.jostrans.org/issue20/art_romero.php

recent book by Wendy Fox with empirical data proving the efficiency of creative subtitles that use fades: http://langsci-press.org/catalog/book/187

recent ppt presentation by Josh Branson with further evidence of user satisfaction when using creative subtitles with fades: https://drive.google.com/open?id=1OaF3ohkB5NRrOwVFbOgH0ntZ2l3qQ_Ra

[skynavga]

This functionality is already supported in TTML2. For example, see the test file and example output [1][2], which uses a linear calculation mode; paced and spline calculation modes are also supported.

Regarding keyframes, media timing in TTML can be expressed using frame count or HH:MM:SS:FF, including fractional frames.

Overall, I think there is nothing new here unless I haven't read you correctly. If you are satisfied that this functionality is supported, then please close this issue.

[1] https://github.com/w3c/ttml2-tests/blob/master/presentation/valid/ttml2-prstn-animate-linear-opacity.xml
[2] https://github.com/w3c/ttml2-tests/blob/master/presentation/valid/ttml2-prstn-animate-linear-opacity.expected.zip

[PabloRFresco]

Many thanks for your reply -very useful indeed.

[nigelmegitt]

Thanks for raising this @PabloRFresco - I suggest that we should take encouragement from this for creating an example showing how to meet this requirement.

[nigelmegitt]

@skynavga your proposal to @PabloRFresco to close this on the basis that the requirement is met by TTML2 is too narrowly scoped - the requirements raised here go further than TTML and include other TTWG specifications.

I'm reopening this on the basis that we should consider if we also need to support it in IMSC. Currently IMSC only supports discrete animation using the set element from TTML1 - a smooth fade using the functionality available in TTML2 would require support for the animate element additionally.

[nigelmegitt]

@skynavga I've adjusted the labels here - we have not yet agreed the output documents in which we may choose to meet any of the requirements here, so rather than positively scoping it into a particular specification, I've added a "met by TTML2" label to indicate that it is certainly out of scope for that specification, i.e. exclude it where it clearly does not apply.

[andreastai]

@PabloRFresco First, I also want to thank you a lot for submitting this requierment! It is important that we get input from the creative side of subtitles and captions. Thanks also for the detailed description and for providing important background information. I think this is a very good example how standards can get feedback and requirements from the operational field.

I agree with @nigelmegitt that we should build an example to test for ourselves and with you that the requirement are met. I am absolutely not sure that they are. Even if there is some technical constellation how to meet all or part of the requirements, we need to ask if this is practical enough to be used.

[skynavga]

@nigelmegitt regarding

Currently IMSC only supports discrete animation using the set element from TTML1 - a smooth fade using the functionality available in TTML2 would require support for the animate element additionally.

Actually, one can always emulate animate via set, so, technically, even TTML1 supports this. Of course, it wouldn't be very inefficient, since it would require many set elements, possibly one for every frame.

[nigelmegitt]

@skynavga re using set (#11 (comment)) I don't consider that to be a viable approach to meeting the requirement, for two reasons. Firstly, as you mention, the verbosity is an issue. Secondly, it may actually not be permitted in IMSC if the HRM threshold requirements are not met, due to generating a large number of ISDs and repaints.

Of course, if we were to introduce continuous animation to IMSC we would need to apply some thought to the implications for the HRM.

[skynavga]

@PabloRFresco have you any further input on this issue? in particular, do you believe there is some technical aspect of fade-in/fade-out that cannot be accommodated by what is already defined in TTML2? do you have specific business requirements that this TTML2 functionality be added to the IMSC profile?

[andreastai]

The first step is to decide if this requirement should be covered by the next version of

1. TTML
2. IMSC

I think it should be covered. See also the explaination from @PabloRFresco why this needed. To have a practical impact it needs to be in IMSC.

If it is requirement, it can be tested if anything needs to be added or if all features are already present. This is not so time sensitive as the general decision about the requirement.

[skynavga]

@PabloRFresco @TairT To be clear, I have seen no language in this thread that suggests a new feature be added to TTML, and, as far as IMSC is concerned, the only question is whether IMSC should support a feature that already exists in TTML2 so as to facilitate use of this functionality in IMSC. Please correct me if you think otherwise.

[css-meeting-bot]

The Timed Text Working Group just discussed Fade in/out tt-rews#11, and agreed to the following:

• SUMMARY: Not currently scheduled for 2019, pending additional input of resource to test and implement.

The full IRC log of that discussion

<nigel> Topic: Fade in/out tt-rews#11
<nigel> s/rews/reqs
<nigel> github: https://github.com//issues/11
<nigel> Andreas: This has been contributed by someone who works in the field of subtitles.
<nigel> .. We have to ask if it is already solved, and if there is already some syntax or mechanics, is it appropriate
<nigel> .. and user friendly enough to be used for the requested feature.
<nigel> q+
<nigel> Glenn: I asked the original commenter two weeks ago for further input asking if they believe there is
<nigel> .. something not accommodated by TTML2. I haven't heard anything back since Dec 20 it looks like.
<atai> q+
<nigel> .. My current assumption is there is no technical feature being asked for, possible a desire to support
<nigel> .. in IMSC. I think we can close this issue. We can reopen it if the commenter comes back.
<nigel> ack nigel
<nigel> Nigel: I see there's no response from the commenter. I also observe that there is an overlap here between
<nigel> .. this, the karaoke and the CSS extensions requirements, so it may be that we meet the requirement
<nigel> .. by reference to one of those others. If it is needed I think it would be needed in IMSC also.
<nigel> .. It is not obvious to me how this can be done in a user friendly way at the moment.
<nigel> ack a
<nigel> Andreas: Glenn said that he asked Pablo Romero Fresco if he agrees that TTML2 already meets the
<nigel> .. requirements or if he objects. As he did not respond then he would like to close the issue. I encouraged
<nigel> .. Pablo to file this issue because he has the requirement and he is not familiar with TTML2. I also saw
<nigel> .. some value in what he requested. I also said to him that if he's not familiar with the technology then
<nigel> .. I would help out which I'm happy to do.
<nigel> .. First I also support this requirement and would not like to close it.
<nigel> .. The important thing for now is to say this requirement should be met by TTML or IMSC.
<nigel> .. I would support that. Until we have not proven that it could be met by existing vocabulary then it
<nigel> .. should not be closed. I agree with Nigel that the current spec text does not satisfy it completely,
<nigel> .. especially in a user friendly way.
<nigel> .. How this is done is a separate issue. I can imagine some dedicated vocabulary like fadeIn or fadeOut.
<nigel> .. Most important is to bring this into scope, for now I would leave it open.
<nigel> Pierre: Did you see the example that Glenn provided?
<nigel> Andreas: Yes
<nigel> Pierre: Is that too complicated?
<nigel> Andreas: Pablo asked for key frames and control over the speed so I'm not sure if this is enough.
<nigel> Glenn: We have spline and keyframe and interpolation modes. All that is being asked for is present.
<nigel> .. User friendliness is not a criteria that we have applied to any of this technology up to now.
<nigel> .. For me this needs a champion within our group, who would be responsible for determining the answer.
<nigel> Andreas: I will champion this.
<nigel> Glenn: I probably would not be inclined to support new shorthand properties for this but you could make
<nigel> .. the case that it is some higher level thing that could be translated into TTML syntax.
<nigel> .. I don't mind leaving it open if there's a champion.
<nigel> Andreas: I am happy to do it. I think you're right that we need a balance for user friendliness, and we may
<nigel> .. not need to add anything new. User friendliness does indeed play a role in TTML, otherwise there would
<nigel> .. not be different syntaxes for, say, color, but this could be delegated to another time.
<nigel> .. We first see if technically it meets the requirements.
<nigel> Glenn: There is a principle at stake called Maximum Orthogonality which we have applied as a strategy
<nigel> .. in TTML which says one should not define alternative ways to do the same thing.
<nigel> Andreas: But we do.
<nigel> .. One addition: I think the request is also to have this feature in a used profile of TTML and the only
<nigel> .. one I know is IMSC, so there would be a request to add it there.
<nigel> Glenn: On this particular issue there was some back and forth on labelling, I guess we can leave this
<nigel> .. open for the time being if you are going to take it forward Andreas.
<nigel> Andreas: Yes, there's no real policy on labelling so it does not represent a decision and I'm fine with any labels.
<nigel> Glenn: I didn't have an objection to Nigel's change here so I didn't make any noise on this issue.
<nigel> Pierre: In terms of the requirement is it on IMSC to support this at the end of the day?
<nigel> Andreas: Yes
<nigel> Pierre: The challenge I have is who will make the implementation effort to make it happen?
<nigel> .. Is the one interested party the tip of the iceberg or is that it?
<nigel> Andreas: I heard it also from other directions. Although it may not be a super critical feature.
<nigel> q+
<nigel> Pierre: I think we need those users to show up somehow.
<nigel> Andreas: The question of how we measure the weight of the feature - often it is sufficient if one member
<nigel> .. presents the case.
<nigel> Pierre: I haven't heard form Movielabs on this.
<nigel> Glenn: I haven't heard from Netflix on this.
<nigel> Nigel: There's an interaction here with responsive, CSS, karaoke: essentially we need to be able to specify
<nigel> .. or interpolate word timings and have some presentation effects dependent on those.
<atai> q+
<nigel> .. I think Cyril suggested time markers which could be relevant too.
<nigel> Glenn: Once upon a time we had a different timing model in TTML presented by John Birch and that took
<nigel> .. a lot of time, which in the end we had no implementations for. What you're bringing up now is
<nigel> .. asking if we need some small amount of that. I'm open to thinking about it again but it's a complicated
<nigel> .. area for sure. Writing in processing semantics for such heuristics is not straightforward.
<nigel> .. In the context of karaoke I think we are going to run into this again so I think we will have to bite the
<nigel> .. bullet and look at it.
<nigel> ack a
<nigel> Andreas: I hear what Pierre is saying and that there needs to be sufficient support for new features, so
<nigel> .. I would ask if other stakeholders would take it up, and if they think it is needed. It needs a balance of
<nigel> .. workload in what we can achieve this year.
<nigel> .. I think that counts of course for every feature. I would also look at the HTML group's process for how
<nigel> .. they consider new features for adoption.
<nigel> Pierre: We have to have this input.
<nigel> Glenn: Here's a link for dynamic flow
<nigel> Pierre: What do we do before we have critical mass of interest?
<nigel> Andreas: I would leave it open for now.
<nigel> Pierre: The question is do we schedule it for 2019? I would object to it unless we know there are people
<nigel> .. who will be testing it, implementing it, paying for it.
<glenn> https://www.w3.org/TR/2010/CR-ttaf1-dfxp-20100223/#style-attribute-dynamicFlow
<nigel> .. In the case of IMSC we have checked it is really what industry wants to do.
<nigel> Andreas: OK I will check that.
<nigel> Nigel: We wanted to come to a decision here but at the moment we don't have consensus to proceed
<nigel> .. with it or to say no.
<nigel> Andreas: What is the process if we do not have a consensus on a feature.
<nigel> q?
<nigel> Nigel: I propose that we proceed with this but note for ourselves that there is additional risk associated
<nigel> .. with this feature, which can be mitigated by getting more adoption or implementation input. Actually
<nigel> .. this risk applies to everything we do, but in this case it has been flagged up early. If we don't get
<nigel> .. enough input on this then the default should be we defer it until such time as we have enough.
<nigel> Pierre: How long will it take Andreas to get additional feedback?
<nigel> Andreas: End of March because I will be out of the office most of February.
<nigel> Pierre: Can we make it earlier if there's strong interest?
<nigel> Nigel: When we come back at the end of today we will have looked at karaoke and other additions and may
<nigel> .. have a change of view.
<nigel> Pierre: Right now I think we can't accept it. By the way I'm not questioning the applicability, as this is used
<nigel> .. today in digital cinema.
<nigel> Andreas: I think Nigel's compromise is a good one for now.
<nigel> Pierre: I'm saying that one person alone is not enough.
<nigel> .. If we can't get input on this until end of March let's not schedule it for 2019.
<nigel> Glenn: Some groups like CSS have an implicit operating rule which is to say that if no browsers implement
<nigel> .. then nobody will pay any attention to it. We should do something similar with respect to the content
<nigel> .. community, and if we have insufficient interest then it is not adequate.
<nigel> Andreas: Whatever approach we take it is quite similar, if we don't take it in now but take it in later. It is the same thing.
<nigel> .. For me there is not a clear basis and transparent basis for external people, when a feature has
<nigel> .. sufficient support and backup to be expected. We now discuss it, so we do not have a clear statement,
<nigel> .. and until then we cannot just say now we need more.
<nigel> Pierre: Ultimately it is the consensus of the group, the AC and the Director that counts.
<nigel> .. It could be that at the end of March this is a high priority thing.
<nigel> .. I would rather say without closing it that we don't schedule it in 2019 and revisit it when there's
<nigel> .. more input, which could be tomorrow, next week or at the end of March.
<nigel> Glenn: I want to remind everyone that new technical requirements need to be accompanied by test content.
<nigel> Andreas: I'm not happy with this, and maybe process-wise we need to deal with the case where we do not
<nigel> .. have consensus. Does that mean it is out of the list?
<nigel> Nigel: Consensus does mean no objections, normally. Here I think the only objection is due to lack of
<nigel> .. perceived interest, which could change. I mean, the BBC's requirements include transition effects too.
<nigel> .. They're in the CSS requirement.
<nigel> Andreas: I could take this if we apply the same rule to everything else. I cannot accept introducing new
<nigel> .. barriers now. We said that the requirements would be open for a certain time but we didn't say more.
<nigel> Glenn: The Chair is responsible for determining whether we have consensus and also whether to overrule
<nigel> .. an objection, and if he declares consensus in the presence of an objection then there's an appeals process for that.
<nigel> Nigel: That's correct in terms of process. For this specific thing I think we need a checkpoint later like
<nigel> .. FPWD where we know for certain if we have enough resource and input, and if we don't then we defer it.
<nigel> .. There are at least 2, maybe 3 or 4 sources for this requirement.
<nigel> Glenn: This requires TTML3 for significant features. We could easily put an example or a note in TTML2 2nd Ed
<nigel> .. describing how to use animation features in for fade in or fade out, giving us some place to point to
<nigel> .. for people who ask questions about it.
<nigel> .. I think I have somewhere an open issue that says "add more animation examples" so I may already have
<nigel> .. an issue for that, in the 75 open issues.
<nigel> Pierre: I don't agree putting this into IMSC in 2019 given the current level of backing today.
<nigel> Glenn: I would support Pierre's position on this.
<nigel> Pierre: I don't think it should be closed though, just not scheduled.
<nigel> Nigel: Okay so we can still make progress and schedule it in for 2019 later if there is backing.
<nigel> Pierre: I agree. There's huge testing effort but probably it is straightforward from a specification perspective.
<nigel> Nigel: Andreas, can you live with that?
<nigel> Andreas: Yes
<nigel> SUMMARY: Not currently scheduled for 2019, pending additional input of resource to test and implement.

[css-meeting-bot]

The Timed Text Working Group just discussed Fade in/out tt-reqs#11.

The full IRC log of that discussion

<nigel> Topic: Fade in/out tt-reqs#11
<nigel> github: https://github.com//issues/11
<nigel> Glenn: Any word from Andreas about following up on this?
<nigel> Nigel: I've not heard anything more yet.
<nigel> Glenn: The only potential action I can see is possibly adding support to IMSC for fade.
<nigel> .. I don't see any feature definitions coming out of that.