[Feature Request] live captions

[jmankoff]

It is becoming standard for accessibility reasons to show live captions when giving a presentation. It would be great if this could be added to remark as well. Looks like there are APIs for this (e.g. see https://github.com/google/io-captions-gadget)

When in presentation mode, captions for anything spoken by the presenter should show up in a footer at the bottom of the slideshow.

[dvberkel]

I think it is awesome to make presentations more accessible. I have seen various life captions and they help everybody in the audience.

The way I have seen this done is with an setup that interweaves various source, presentation, video, live caption. This was highly specific to the conference/venue. So it is not clear to me if this would be something that remark should support, because it is so specific to how the live captions are created.

@jmankoff do you have experience with live captioning? Could you provide more of a context?

[jmankoff]

Sure, I've actually got something working right now which I am trying to write up in a blog post. I am currently teaching a course that has a student who uses a screen reader and a deaf TA so I'm using remark in combination with Mermaid, MathJax, Jekyll, and webcaptioner.com to make the class accessible. mathjax and mermaid are integrated into my remark markdown. Jekyll's been modified to add a div about the slides showing live captions, and to add a special link only visible when you tab through links to the raw markdown.

Website: http://cse340-20wi.pages.cs.washington.edu/website/
Slides: linked off the schedule page.
Accessible version of slides: Tab through schedule page to see the raw markdown links
The addition of webcaptioner was done in this file: https://gitlab.cs.washington.edu/cse340-19sp/website/blob/CSE340-SP19-FINAL/_layouts/presentation.html

[dvberkel]

Thanks for the extensive references. The course seems very interesting.

I was not aware of webcaptioner.com. It seems a great resource. I will take a look into the details.

Just to be on the same page: you are requesting to have an option to enable live captioning for remark presentations with webcaptioner?

[jmankoff]

Correct, or any other captioning service really but that one certainly works! :)