New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add subtitle positioning option on the editor and cues for VTT and DFXP #3023

Closed
franontanaya opened this Issue Nov 16, 2017 · 21 comments

Comments

Projects
None yet
4 participants
@franontanaya
Contributor

franontanaya commented Nov 16, 2017

Subtitle positioning is a base requirement for captioning. We'll support this in the editor like this:

  • hovering over the subtitle in the video player will change the cursor to a positioning cursor.

  • clicking and holding over the subtitle will make the two possible positioning regions (highlighted rectangles) appear on the video player.

  • when subtitles are dragged to the top region, they stick there.

  • subtitles positioned in the top of the video player are exported with a line cue:

    00:00:00.830 --> 00:00:03.153 line:1%

    • DFXP captions would add the attribute region="top" to the p container. This region is already defined in our layout header.
    • VTT is currently the main use case for positioning since it serves YouTube users and it's been requested several times, so we'd want to modify that export first.

https://www.rev.com/captions-editor/sandbox


Context:

FCC regulations:

(iv) Placement. Captioning shall be viewable and shall not block other important visual content on the screen, including, but not limited to, character faces, featured text (e.g., weather or other news updates, graphics and credits), and other information that is essential to understanding a program's content when the closed captioning feature is activated.

The main commercial Quality Control checks look for failures to this rule. On Amara, this currently means captions that obscure bottom third content need to be marked within the caption text content and then processed after export to replace the markings with actual cues.

Video sites were behind on positioning cue support, but since a year or so YouTube supports the "line" cue on WebVTT. Timed Text exports (DFXP) are the other frequent use case for positioning we know of. SCC and SSA support positioning but the first is legacy and the second is a niche format, so it's not so pressing.

@kslottow kslottow self-assigned this Nov 16, 2017

@kslottow kslottow added this to the Sprint 26 milestone Nov 16, 2017

@kslottow kslottow added the new editor label Nov 29, 2017

@kslottow kslottow changed the title from Add subtitle position toggle on the editor and cues for VTT and DFXP to Add subtitle positioning option on the editor and cues for VTT and DFXP Nov 29, 2017

@kslottow kslottow removed this from the Sprint 26 milestone Dec 15, 2017

@kslottow kslottow assigned bendk and unassigned kslottow Apr 4, 2018

@kslottow kslottow modified the milestones: Sprint 38, Sprint 36 Apr 4, 2018

@franontanaya

This comment has been minimized.

Contributor

franontanaya commented Apr 5, 2018

Aside from the visual display of the position on the player, we could try to have some indication on the caption itself. For example, if a captioner is moving around the caption list to review the positioning, they can locate those positions that way without screening the video.

Inspecting the VTT file itself is a workaround, but wherever it can be done without cluttering the UI it'd have a practical benefit.

@bendk

This comment has been minimized.

Member

bendk commented Apr 13, 2018

Good point @franontanaya . I made #3176 for that one. For this issue, I'm going to skip it because I think we need a couple design assets to do it well.

@franontanaya

This comment has been minimized.

Contributor

franontanaya commented Apr 16, 2018

Adding to this, we have this request:

Ideally, they want files that require positioning to be delivered as a .ITT file so that subtitles can be top-positioned when necessary.

ITT falls under the DFXP positioning part of the ticket, both are Timed Text.

@bendk

This comment has been minimized.

Member

bendk commented Apr 17, 2018

Implemented in the gh-3023 branch.

To change the subtitle position, drag the subtitle in the video player.

  • For DFXP, this was pretty simple, I just put the subtitles in the top region
    I have no clue how to test the DFXP, what's a player that supports DFXP regions?
  • Added VTT support for regions like this:
    • VTT export
      • The top region gets "line:1"
      • The default/bottom region has no positioning
    • VTT import
      • We consider subtitles in the top region if:
        • The line is a number <= 5
        • The line is a percent <= 20%
  • Are there other formats that we need to support?

@bendk bendk modified the milestones: Sprint 36, Sprint 37 Apr 17, 2018

@PCF-Testing

This comment has been minimized.

Member

PCF-Testing commented Apr 19, 2018

For me, the editor in the branch behaves inconsistently.

Suppose I add a new video and start transcribing it. With the first subtitle I am typing, I move it from the bottom (default) region to the top region and continue typing. The second and subsequent subtitles are also displayed in the top region (suppose I am happy with this location and decide to keep it this way). Then I finish all this typing and start syncing; the first subtitle is still displayed in the top region, but the 2nd and all the next are in the bottom region. The DFXP file says:
<body region="bottom"> <div> <p begin="00:00:01.489" end="00:00:03.814" region="top">aaa</p> <p begin="00:00:03.814" end="00:00:05.453">bbb</p> <p begin="00:00:05.453" end="00:00:07.224">ccc</p> <p begin="00:00:07.224" end="00:00:09.436">ddd</p> </div> </body>

I'd expect better predictability in one of the following possible ways:

  1. The next subtitle has the same position as the previous one unless the user drags it to the other region,
    or
  2. The default position is always at the bottom and the user always has to drag the subtitle upwards to change it if they need.

I like option 1 better, but AOD transcribers may have a different opinion based on their experience.

@PCF-Testing

This comment has been minimized.

Member

PCF-Testing commented Apr 19, 2018

Regarding subtitle file export, both DFXP and VTT are gracefully accepted by Vimeo and YouTube. Vimeo ignores all the positioning information, while YouTube honors the positioning in the VTT files downloaded from Amara.

Downloaded ITT files should also include information about positioning. They currently don't.

bendk added a commit that referenced this issue Apr 19, 2018

@bendk

This comment has been minimized.

Member

bendk commented Apr 19, 2018

  • ITT should work now.
  • Made it so if the current subtitle is on top and you hit enter, the next one is now on top.
  • Fixed a bunch of smaller issues.
@PCF-Testing

This comment has been minimized.

Member

PCF-Testing commented Apr 20, 2018

  • Uploading ITT fails on the branch. This goes not only about the "new" ITT format that supports regions, but ITT files downloaded from production fails as well. It is a regression compared to production.

  • Once the subtitles are synced, I don't see the option to change the subtitle position anymore. It feels awkward that the user cannot change their mind about the position of subtitles. Suppose the video gets modified after the subtitles are timed, and the changes require re-positioning of some subtitles?

  • We need a keyboard shortcut for toggling the position of the subtitle because switching to the mouse device while doing the typing requires pausing the video, drag-dropping the subtitle area, then clicking the link to add a new subtitle cell and continue typing - that is not super convenient. How about ALT + T (for Toggle) or ALT + P (for Position) to toggle the current position of subtitle to the opposite region?

  • When the editor is in the translation mode (i.e. the video is already transcribed and a new video is added), all the subtitles are positioned at the bottom region. Since re-positioning is usually done to prevent some part of the image obscured by subtitles, I think it is natural to expect that the translation would be by default positioned exactly as the reference language it is being translated from. E.g., if subtitle 3 in the original language is at the top position, I'd expect the editor to position subtitle 3 in the translation to the same place, etc. The users should have the option to change this position.

  • The video embedder should honor the subtitle position. It currently doesn't, in contrast to the player in the editor.

@bendk

This comment has been minimized.

Member

bendk commented Apr 20, 2018

 Once the subtitles are synced, I don't see the option to change the subtitle position anymore. It feels awkward that the user cannot change their mind about the position of subtitles. Suppose the video gets modified after the subtitles are timed, and the changes require re-positioning of some subtitles?

Can you explain this one more? When I'm testing I can change the position of synced subtitles by dragging them.

We need a keyboard shortcut for toggling the position of the subtitle because switching to the mouse device while doing the typing requires pausing the video, drag-dropping the subtitle area, then clicking the link to add a new subtitle cell and continue typing - that is not super convenient. How about ALT + T (for Toggle) or ALT + P (for Position) to toggle the current position of subtitle to the opposite region?

I totally agree with this, but maybe we should wait and do it when we add the menus. I think that could be a good time to think about our shortcuts generally.

@PCF-Testing

This comment has been minimized.

Member

PCF-Testing commented Apr 21, 2018

  • I have tried to screencast the behaviour where the position of the subtitle does not "stick" to the destination region at https://www.youtube.com/watch?v=JiXc4JlhSik. My understanding is that it is possible to change the subtitle position if you seek to the subtitle prior to changing its position (0:28), but if you just click in the next subtitle and try to drag-drop it (0:40), it won't stick to the new place.

  • If you position a subtitle to the top and then split it (CTRL+Enter) or insert a new subtitle below, the new subtitles will have the default (bottom) position. In the spirit of the decision that the subsequent subtitle should have the same position as the previous one, unless the user decides to change this, I think that the newly created subtitles should have the same position as those from which they got created. The only exception is "Insert above" action because it is too hard to predict where it should be positioned, so the default (bottom) position should be fine.
    Summary:

  • "Insert above" should create a new subtitle in the bottom region
  • "Insert below" should create a new subtitle positioned in the same region as the "parent" subtitle
  • "Split subtitle" should create a new subtitle positioned in the same region as the "parent" subtitle
  • The new ITT format has every new subtitle with the paragraph mark on. This means that if you download "normal" subtitles as a file in ITT format from the branch and upload it again as a different language, every subtitle will be marked as a new paragraph.
@PCF-Testing

This comment has been minimized.

Member

PCF-Testing commented Apr 21, 2018

Made #3185 for the keyboard shortcut.

@PCF-Testing

This comment has been minimized.

Member

PCF-Testing commented Apr 21, 2018

  • Maybe a nitpick, but I've got the impression that the video player has the top region subtitles displayed higher than in the editor. Compare:
  • in the editor:
    screenshot from 2018-04-21 06 02 03
  • in the player:
    screenshot from 2018-04-21 06 02 56

bendk added a commit that referenced this issue Apr 23, 2018

@bendk

This comment has been minimized.

Member

bendk commented Apr 23, 2018

Those should be fixed. We talked on the dev chat today a bit and decided to make insert above also copy the subtitle positioning. It's not clear if that's more likely, but it makes the UI more predictable to have it match insert below.

@PCF-Testing

This comment has been minimized.

Member

PCF-Testing commented Apr 24, 2018

There are still quirks with changing the position of previously positioned subtitles. Consider the following subtitle set:

WEBVTT

NOTE Paragraph

00:00:01.600 --> 00:00:04.106
bottom 1

00:00:04.106 --> 00:00:06.470
bottom 2

00:00:06.470 --> 00:00:08.860 line:1
top 3

00:00:08.860 --> 00:00:11.469 line:1
top 4

00:00:11.469 --> 00:00:14.129 line:1
top 5

00:00:14.129 --> 00:00:16.971
bottom 6

00:00:16.971 --> 00:00:19.419
bottom 7

00:00:19.419 --> 00:00:21.861
bottom 8

00:00:21.861 --> 00:00:24.600 line:1
top 9

00:00:24.600 --> 00:00:27.919 line:1
top 10

Play back the video. Each subtitle will be displayed where the subtitle text suggests.

Now rewind the video to the beginning. Start playing it and pause in the middle of subtitle "bottom 1". In the editing panel, click on "top 3" cell and drag the subtitle on the video player to the bottom position. Press TAB to resume playback. All the subtitles in the set, not just "top 3" and "bottom NN", will be displayed in the bottom area.

@bendk

This comment has been minimized.

Member

bendk commented Apr 24, 2018

I can't reproduce that exact issue. I'm pretty sure I followed the instructions, but subtitles were on both the top and bottom.

What I did notice was that if I didn't click away, then "top 3" would stay in the editing state, even after the video playback started. I just fixed that one. Hopefully it addresses what you saw too.

@PCF-Testing

This comment has been minimized.

Member

PCF-Testing commented Apr 25, 2018

Yes, whatever it was, it doesn't happen anymore.

One - hopefully, last - thing, while we're at it: in Syncing / Review mode, "+ New subtitle" button under the last subtitle appends a new untimed subtitle to the set. For consistency, could this new subtitle also copy the position from the previous subtitle?

@bendk

This comment has been minimized.

Member

bendk commented Apr 25, 2018

I'm not totally sure I agree that that's the best behavior, but I just implemented the above.

@PCF-Testing

This comment has been minimized.

Member

PCF-Testing commented Apr 26, 2018

Sorry, found one more thing - the teal strips are not displayed and drag-dropping does not work in MS Edge and MS Internet Explorer on Windows. I checked on a real Win 10 machine, but this can be reproduced in CrossBrowserTesting.

The rest looks good to me. Some details of the current behavior:

  • Subtitles that get automatically synced to YouTube, keep their position as specified on Amara.
  • Subtitles that get automatically synced to Vimeo are all displayed on Vimeo in the bottom position. This seems to be the peculiarity of the Vimeo player; the VTT subtitles stored on Vimeo keep the positioning info, and a VTT file with all this data can be downloaded.
  • DFXP and ITT files downloaded from Amara keep the positioning info, but these files uploaded to YouTube produce subtitles that are all displayed at the bottom.
@bendk

This comment has been minimized.

Member

bendk commented Apr 26, 2018

Just pushed a fix for Edge. Not sure if it will also fix IE. If not, I think we should just merge anyway and not support IE.

@PCF-Testing

This comment has been minimized.

Member

PCF-Testing commented Apr 27, 2018

Works well for me in MS Edge. For IE, the fix did not work; the console said:
SCRIPT65535: Unexpected call to method or property access.
editor.js (54008,17)

I also had trouble with Chrome 64bit on OS X - the dragged subtitles did not "stick" to the destination region. Not sure if it is a real problem or just some quirks of the simulator.

Moving to Needs PR queue since everything else looks good.

bendk added a commit that referenced this issue Apr 27, 2018

bendk added a commit that referenced this issue Apr 27, 2018

@PCF-Testing

This comment has been minimized.

Member

PCF-Testing commented May 3, 2018

Deployed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment