Reflowable Video Content

[AdamSobieski]

Hello. I recently thought of an idea while watching Bloomberg TV that I would like to share here. If you haven’t already seen Bloomberg TV, it presents news video in a quadrant of the screen while also presenting a number of rectangular widgets on-screen, e.g., listing upcoming scheduled content, and there are animated infographics, graphs and charts, on-screen as well.

The idea which I would like to share here includes that DASH-based streaming services could provide different video streams for different resolutions or ranges thereof.

For the above example, at high resolutions, one could provide the news video plus widgets plus infographics. At lower resolutions, one could provide just the news video and widgets. At yet lower resolutions, one could provide just the news video.

Similarly, more widgets could be added to such layouts when the videos or containing websites were scaled larger, e.g., across multiple (adjacent) displays.

Such layouts of widgets could be varied for scenarios where the vertical number of pixels exceeded the horizontal. That is, such layouts of widgets could be varied when smartphones were rotated between landscape and portrait orientations.

With these features under discussion, Web developers and end-users could resize video playback components, and/or their containing browser windows, including while videos were playing, and, as they did so, video players would automatically switch video streams as needed. These features would enable text layouts and font-sizes to be varied for readability and would enable effectively reflowable video content.

A conceptually related W3C technology is that of CSS Media Queries.

If it isn't already possible to implement these features, is there interest here in making these features possible, making it possible to provide multiple video track options such that each option would be able to specify a range of resolutions that it should be selected for?

Thank you. I look forward to discussing these ideas with you.

[rajneeshksoni]

I am not sure about DASH.
HLS has the concept of alternative renditions which allow sending the different stream for same content.
https://datatracker.ietf.org/doc/html/draft-pantos-http-live-streaming-23#section-4.3.4.2.1

The scenarios described above fit well with it.

[AdamSobieski]

Quoting from Section 4.3.4.1.1:

A set of one or more EXT-X-MEDIA tags with the same GROUP-ID value and the same TYPE value defines a Group of Renditions. Each member of the Group MUST be an alternative rendition of the same content; otherwise playback errors can occur.

Quoting from Section 4.3.4.2.1:

Every alternative Rendition associated with an EXT-X-STREAM-INF tag MUST meet the constraints for a Variant Stream described in Section 6.2.4.

Quoting from Section 6.2.4:

Each Variant Stream MUST present the same content.

Under discussion are renditions with potentially different layouts of contained content, elements, or widgets and with potentially different text layouts and/or font sizes.

Perhaps I'm overthinking the meaning of "same content"? If not, maybe there is a quick workaround like adding an attribute to something to indicate that it is not a "same content" alternative rendition but that, instead, it is a "reflowed" alternative rendition?