WP affords time-based media and text

[TzviyaSiegman]

2.1.4 Time-based Media and Text
The notion of a Web Publication should enable specific publications like audio books, graphics books, and mixed media.
See #52

[HadrienGardeur]

Are we talking about synchronized text with multimedia? Otherwise, this duplicates #133

[TzviyaSiegman]

Thanks @HadrienGardeur. I was doing a lot of copy/paste. I've updated it now.

[jmulliken]

This seems similar in scope to #135, which has been proposed as one to close. Since the web allows for this kind of media, do we need to specify it here?

[iherman]

@jmulliken probably not...

[GeorgeKerscher]

Affordance name: Synchronized text and audio (134)

1. Short description: Synchronized text and audio is a feature set that has been in use for audio books. There are two variations:
   A: Full text and audio --there is the full text of the title and there is a human narrated recording that is aligned with the text. The synchronization most often will be on the paragraph or at the sentence level. A TOC is provided for navigation.
   B: TOC and full audio -- This is most often associated with an audio book, where the recording of the title is provided. A TOC is provided for navigation, but this is the only text available.

We commonly find that a novel is recorded and sold as an audio book. These are very popular and normally are produced by a person reading the book after it has been published.

Note: this is not the same as captions or an audio track in videos.

2. Dependencies: There must be a specification that is used to combine time-based media (audio files) with static text. The alignment may be on the ID of an element, e.g. p for paragraph level syncronization , or ID of a span for sentence level synchronization and the time offset in the audio file.

The Community Group where this is going on is at:
w3c/sync-media-pub: Repository of the Synchronized Multimedia for Publications Community Group
https://github.com/w3c/sync-media-pub
3. Use case Reference:
2.1.4 Time-based Media and Text
https://w3c.github.io/dpub-pwp-ucr/#r_time
Permalink for 2.1.4 Time-based Media and Text

Req. 4:
A Web Publication needs to support both time-based media and text.

A Web Publication needs to support time-based media, such as synchronized video, audio, captions or transcript, or sign language interpretation. A Web
Publication must also be able to enable a synchronized media experience while navigating through the publication, with sufficient level of granularity.

list of 1 items
Usage examples:
• Illyés has a cognitive disability and uses accommodated texts in the classroom, to help learn the content while improving his reading. His assistive
technology uses combined audio and highlighted text, which it obtained from the UA through the information provided in the Web Publication, to turn the
page for him while reading along in sync with the page currently open.
list end

4. Examples: ANSI/NISO Z39.86 and EPUB 3 Media Overlays.
   EPUB Media Overlays 3.1
   https://www.w3.org/Submission/2017/SUBM-epub-mediaoverlays-20170125/

This specification, EPUB Media Overlays 3.1, defines a usage of
[SMIL]
(Synchronized Multimedia Integration Language), the
Package Document,
CSS Style Sheets, and
EPUB® Content Documents
for representation of audio synchronized with the EPUB Content Document.

Digital Talking Books
http://www.daisy.org/z3986/2005/Z3986-2005.html

ANSI/NISO Z39.86-2005 (R2012)
Revision of ANSI/NISO Z39.86-2002
ISSN: 1041-5653
Specifications for the Digital Talking Book

Abstract: This standard defines the format and content of the electronic file set that comprises a digital talking book (DTB) and establishes a limited
set of requirements for DTB playback devices. It uses established and new specifications to delineate the structure of DTBs whose content can range from
XML text only, to text with corresponding spoken audio, to audio with little or no text. DTBs are designed to make print material accessible and navigable
for blind or otherwise print-disabled persons.

An American National Standard Developed by the National Information Standards Organization
Approved April 21, 2005 by the American National Standards Institute

5. Testing: A "Media Overlay test book has been produced that details the tests to be performed. EPUBTEST 0320 - Media Overlays Fundamental Accessibility Tests: and it can be found at: http://epubtest.org/testsuite/

[wareid]

Affordances addressed in the UCR document, as mentioned in the meeting on Feb 4 2019, I am closing this issue.