Update index.bs with ChapterInformation #308

jiajiabingcheng · 2023-12-13T21:51:31Z

Added a ChapterInformation attribute in the MediaMetadata to carry the Youtube video chapter data:1) title of the section 2)timestamp of this section 3)screenshot image data of this section

Preview | Diff

steimelchrome

There's a section of examples near the bottom of the spec. I think we should also add an example of adding chapter information

steimelchrome · 2023-12-15T00:52:36Z

index.bs

+
+The <dfn dict-member for="ChapterInformation">startTime</dfn> <a>dictionary member</a>
+is used to specify the {{ChapterInformation}} object's {{MediaImage/startTime}} in
+seconds. It should always be positive.


Can it also be zero? I'd guess most first chapters would have a startTime of zero

Changed it to "should not be negative."

index.bs

jiajiabingcheng · 2023-12-20T02:24:02Z

Updated the example section. Also updated the explainer.md in this pr.

index.bs

ChapterInformation will be added to the MediaMetadata Github pr: w3c/mediasession#308 This cl implements it on the blink layer. Bug: b/316045178 Change-Id: Ic06140e1aed1094d3db68fd167c9acf14c3cd2fa Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5124302 Commit-Queue: Alex Newcomer <newcomer@chromium.org> Reviewed-by: Xianzhu Wang <wangxianzhu@chromium.org> Reviewed-by: Mohamed Heikal <mheikal@chromium.org> Reviewed-by: Joe Mason <joenotcharles@google.com> Cr-Commit-Position: refs/heads/main@{#1240183}

explainer.md

index.bs

explainer.md

youennf · 2024-01-14T19:55:40Z

index.bs

 };

 dictionary MediaMetadataInit {
  DOMString title = "";
  DOMString artist = "";
  DOMString album = "";
  sequence&lt;MediaImage> artwork = [];
+  sequence&lt;ChapterInformation> chapterInfo = [];


If we introduce a ChapterInformationInit, this could be a ChapterInformationInit instead (which would allow passing either ChapterInformationInit or ChapterInformation here).

Done. Shall I also use ChapterInformationInit in the MediaMetadata interface declaration(L951)?

marcoscaceres · 2024-01-15T04:35:59Z

index.bs

+        better to do this with IDL primitives instead of JS - see
+        https://www.w3.org/Bugs/Public/show_bug.cgi?id=29004 -->
+        <li>
+          Call <a lt="freeze">Object.freeze</a> on <var>image</var>, to prevent


This seems not great... yeah, maybe this should be an interface.

Have changed ChapterInformation to an interface in this pr. For MediaImage , it might be better to submit a separate pr under the corresponding issue.

marcoscaceres · 2024-01-15T04:37:30Z

index.bs

+getting, it MUST return the result of the following steps:
+  <ol>
+    <li>
+      Let <var>frozenArtwork</var> be an empty list of type {{MediaImage}}.


Still kinda not great that we have ImageResource and MediaImage...

Are they referring to the same thing? I see we are using MediaImage in MediaMetadata, so I also use it in the ChapterInformation.

Co-authored-by: youennf <youennf@users.noreply.github.com>

youennf

@jan-ivar, @ChunMinChang , since you have thoughts in The freezing in the artwork getter is broken #237.

explainer.md

youennf · 2024-01-25T08:45:18Z

index.bs

 };

 dictionary MediaMetadataInit {
  DOMString title = "";
  DOMString artist = "";
  DOMString album = "";
  sequence&lt;MediaImage> artwork = [];
+  sequence&lt;ChapterInformationInit> chapterInfo = [];


I do not see ChapterInformationInit being defined here.

It is defined at L1184, can I use it here?

This whole section is somehow redundant with web idl.
Since we use ChapterInformationInit, it seems to make sense to add it there as well, since it seems this would be the only one missing here.

Let's add ChapterInformationInit for consistency here

index.bs

explainer.md

index.bs

jan-ivar · 2024-01-26T21:46:57Z

index.bs

+
+The <dfn dict-member for="ChapterInformation">startTime</dfn> <a>dictionary member</a>
+is used to specify the {{ChapterInformation}} object's {{MediaImage/startTime}} in
+seconds. It should not be negative.


Avoid "should" and describe what MUST happen if it is negative.

This should be done in a ChapterInformation(init) constructor (similar to MediaMetadata(init)), which should throw TypeError on invalid input, as well as run the convert artwork algorithm.

index.bs

jan-ivar · 2024-01-26T22:04:27Z

index.bs

+The <dfn dict-member for="ChapterInformation">title</dfn> <a>dictionary member</a> is
+used to specify the {{ChapterInformation}} object's {{MediaImage/title}}. 
+
+The <dfn dict-member for="ChapterInformation">startTime</dfn> <a>dictionary member</a>
+is used to specify the {{ChapterInformation}} object's {{MediaImage/startTime}} in
+seconds. It should not be negative.
+
+The <dfn dict-member for="ChapterInformation">artwork</dfn> <a>dictionary member</a>
+is used to specify the {{ChapterInformation}} object's {{MediaImage/artwork}}. On 
+getting, it MUST return the result of the following steps:


Dictionary members cannot have getters or setters.

This section appears to conflate the dictionary members with the interface attributes, which is confusing. We should describe them separately like we do for MediaMetadata.

I have changed it to an interface now as another comment suggested. I also followed the MediaMetadata 's description and rewrote this part in the latest commit :]

jan-ivar · 2024-01-26T22:23:31Z

index.bs

+      Let <var>frozenArtwork</var> be an empty list of type {{MediaImage}}.
+    </li>
+    <li>
+      For each <var>entry</var> in the {{ChapterInformation}}'s <a
+      for="ChapterInformation">artwork images</a>, perform the following steps:
+      <ol>
+        <li>
+          Let <var>image</var> be a new {{MediaImage}}.


Creating new objects in a getter is an anti-pattern, especially in a frozen array. It violates § 6.1. Attributes should behave like data properties. See also crbug 94665.

I understand this is copied from https://w3c.github.io/mediasession/#dom-mediametadata-artwork but that is also wrong and redundant since the MediaMetadata(init) already runs "the convert artwork algorithm with init’s artwork as input and set metadata’s artwork images as the result if it succeeded."

IOW we ran convert (instantiated objects) on set, so no need to run it again (instantiating more objects) on every get. The attribute getter can simply return artwork images.

Updated to:
"The artwork attribute reflects the ChapterInformation's artwork images. On getting, it MUST return the ChapterInformation's artwork. On setting, it MUST run the convert artwork algorithm with the new value as input, and set the ChapterInformation's artwork images as the result if it succeeded."
Let me know what you think.

index.bs

jan-ivar · 2024-01-26T22:28:16Z

index.bs

+On setting, it MUST run the <a>convert artwork algorithm</a> with the new value as
+<var>input</var>, and set the {{ChapterInformation}}'s <a for="ChapterInformation">artwork 
+images</a> as the result if it succeeded.


This needs to be in the constructor algorithm

Co-authored-by: Jan-Ivar Bruaroey <jan-ivar@users.noreply.github.com>

jiajiabingcheng · 2024-02-06T03:35:06Z

Hey @jan-ivar, thanks for the review. Can you PTAL again at the latest patchset?

explainer.md

index.bs

Co-authored-by: François Daoust <fd@tidoust.net>

chrisn · 2024-02-07T13:06:25Z

index.bs

+  constructor, when invoked, MUST run the following steps:
+
+  <ol>
+    <li>


... and NaN (similar to the steps for setPositionState)

jiajiabingcheng · 2024-02-13T00:17:00Z

Hey @chrisn, @jan-ivar, thanks for the previous review. Can you PTAL again at the latest patchset?

index.bs

Co-authored-by: Chris Needham <chrisn@users.noreply.github.com>

jiajiabingcheng · 2024-02-21T23:10:19Z

Hey @chrisn, @jan-ivar, thanks for the previous review. Can you PTAL again at the latest patchset?

chrisn

I added one minor comment, but apart from that, it all looks good to me. Thank you!

explainer.md

index.bs

jan-ivar

LGTM with some nits and the fix from @tidoust! Thanks!

index.bs

jan-ivar · 2024-02-22T20:39:53Z

index.bs

+      Run the <a>convert artwork algorithm</a> with <var>init</var>'s
+      {{ChapterInformationInit/artwork}} as <var>input</var> and set


This looks good but requires fixing the convert artwork algorithm algorithm to expect artwork instead of input (right now it still says "For each entry in input’s artwork, perform the following steps:") (plus fixing the other call to that algorithm).

Can we rephrase this paragraph to:
"Run the convert artwork algorithm with init’s artwork as input and create a frozen array from the result, then set it to chapterInfo’s artwork images." ?

Do we need to change the other places? For this sentence "When the convert artwork algorithm with input parameter is invoked, the user agent MUST run the following steps:", isn't input the init's artwork list which need to be parsed with the algorithm? And then it can generate the output list?

Clicking on the link in "For each entry in input’s artwork" reveals the expected type of input to be MediaMetadataInit. I.e. A's B is a deref. To take a sequence, e.g. artwork instead of {artwork}, it needs to say "For each entry in input".

Changed to "For each entry in input (which is a MediaImage list), perform the following steps:"

Co-authored-by: François Daoust <fd@tidoust.net>

Co-authored-by: Jan-Ivar Bruaroey <jan-ivar@users.noreply.github.com>

jiajiabingcheng · 2024-02-27T23:15:38Z

Thanks @chrisn and @jan-ivar for the approval!

Hi @youennf, can you PTAL at the latest patchset? Let me know if anything is missing :] Thanks!

youennf · 2024-03-06T08:34:05Z

explainer.md

+[Exposed=Window]
+interface ChapterInformation {
+  attribute DOMString title;
+  attribute double startTime;


Should these two be readonly?

youennf · 2024-03-06T08:39:02Z

index.bs

+
+dictionary ChapterInformationInit {
+  DOMString title = "";
+  double startTime = 0;


What happens if there are two chapter information with the same startTime?
Should we define which one is selected using the sequence order?
Also, it seems we somehow expect that the web page provides a sequence ordered by startTime but I would guess it is fine to not do so.

I believe it is unlikely that there will be more than one chapter with the same start time. However, if this does occur, we can simply use the chapter list as normal and sort it by start time after we have obtained it. Because the image sources are associated with each chapterInformation, there should be no problems regardless of the order in the list. Then in the UI, some chapters with the same start time will be displayed next to each other.

Agreed this is an edge case, let's make sure that even these corner cases get implemented in an interoperable manner. So if we have to use list order to disambiguate, let's define it here.

I'm thinking the website should be able to set whatever information is with the media. Even if there are two or more chapters with the same start time, there should be no order from them because they all refer to the same point in the video. Also, would it be better to leave this up to the client side? When the client receives a list of chapters, they can order them by start time first. Then, it is up to them whether or not to display them in any order. WDYT?

Hey youennf, we're moving forward with submitting this pull request. Please feel free to open an issue later if you still have any additional questions about this change. Thanks :]

youennf · 2024-03-06T08:45:25Z

index.bs

 };

 dictionary MediaMetadataInit {
  DOMString title = "";
  DOMString artist = "";
  DOMString album = "";
  sequence&lt;MediaImage> artwork = [];
+  sequence&lt;ChapterInformationInit> chapterInfo = [];


This whole section is somehow redundant with web idl.
Since we use ChapterInformationInit, it seems to make sense to add it there as well, since it seems this would be the only one missing here.

youennf · 2024-03-06T08:50:49Z

index.bs

+      Run the <a>convert artwork algorithm</a> with <var>init</var>'s
+      {{ChapterInformationInit/artwork}} as <var>input</var> and set
+      <var>chapterInfo</var>'s <a for="ChapterInformation">artwork images</a> to the
+      result of [=Create a frozen array|creating a frozen array=] from the result.


This is good as is I think but it could be made slightly clearer by splitting in two steps:

Let artWork be the result of running the convert artwork algorithm ...

Set chapterInfo.... to the result of creating a frozen array from artWork.

jiajiabingcheng · 2024-03-07T01:17:45Z

@youennf Thanks for the review!
I will add the chapter_information_init.idl under blink soon!
PTAL at the latest patchset and see if you have more comments :]

jiajiabingcheng · 2024-03-19T20:54:37Z

Hi @youennf, can you PTAL at the latest patchset and see if you have more comments? The corresponding change is already landed under blink/.

SHA: c7a47fb Reason: push, by steimelchrome Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update index.bs with ChapterInformation

2e39d54

This was referenced Dec 13, 2023

Extend MediaMetadata to capture video chapter information #273

Open

update the explainer based on a recent MediaMetadata change #309

Closed

jiajiabingcheng added 2 commits December 14, 2023 14:37

Update index.bs

40b6030

Update index.bs

1c90b91

steimelchrome reviewed Dec 15, 2023

View reviewed changes

jiajiabingcheng added 2 commits December 15, 2023 15:19

Update index.bs resolve comments

58fcc52

Update index.bs

ac87a02

steimelchrome reviewed Dec 19, 2023

View reviewed changes

index.bs Outdated Show resolved Hide resolved

jiajiabingcheng added 2 commits December 19, 2023 17:45

Update index.bs: add examples

690f342

Update explainer.md

af180ff

steimelchrome reviewed Dec 20, 2023

View reviewed changes

index.bs Outdated Show resolved Hide resolved

index.bs Outdated Show resolved Hide resolved

Update index.bs

bf940ad

youennf reviewed Jan 14, 2024

View reviewed changes

marcoscaceres reviewed Jan 15, 2024

View reviewed changes

jiajiabingcheng and others added 5 commits January 22, 2024 16:05

Update index.bs

9d60ac7

Co-authored-by: youennf <youennf@users.noreply.github.com>

Update index.bs

3e3ee8c

Update explainer.md

283feab

Update explainer.md

2869d20

Update index.bs

a1825ea

youennf reviewed Jan 25, 2024

View reviewed changes

jiajiabingcheng added 3 commits January 25, 2024 14:44

Update index.bs

140971c

Update index.bs

27cc4db

Update explainer.md

a43f58f

jan-ivar suggested changes Jan 26, 2024

View reviewed changes

jiajiabingcheng and others added 2 commits January 26, 2024 15:59

Update index.bs

7690976

Co-authored-by: Jan-Ivar Bruaroey <jan-ivar@users.noreply.github.com>

Update index.bs

edeaf6d

Co-authored-by: Jan-Ivar Bruaroey <jan-ivar@users.noreply.github.com>

tidoust reviewed Feb 6, 2024

View reviewed changes

explainer.md Show resolved Hide resolved

index.bs Show resolved Hide resolved

jiajiabingcheng and others added 2 commits February 6, 2024 13:35

Update explainer.md

ea885c3

Co-authored-by: François Daoust <fd@tidoust.net>

Update index.bs

9959bcf

Co-authored-by: François Daoust <fd@tidoust.net>

chrisn requested changes Feb 7, 2024

View reviewed changes

index.bs

constructor, when invoked, MUST run the following steps:

<ol>

<li>

Copy link

Member

chrisn Feb 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... and NaN (similar to the steps for setPositionState)

chrisn reviewed Feb 13, 2024

View reviewed changes

index.bs Show resolved Hide resolved

Update index.bs

8bf0611

Co-authored-by: Chris Needham <chrisn@users.noreply.github.com>

chrisn reviewed Feb 22, 2024

View reviewed changes

explainer.md Outdated Show resolved Hide resolved

chrisn reviewed Feb 22, 2024

View reviewed changes

index.bs Outdated Show resolved Hide resolved

jan-ivar approved these changes Feb 22, 2024

View reviewed changes

jiajiabingcheng and others added 7 commits February 22, 2024 13:23

Update index.bs

76e40d7

Co-authored-by: François Daoust <fd@tidoust.net>

Update index.bs

1b8b564

Co-authored-by: Jan-Ivar Bruaroey <jan-ivar@users.noreply.github.com>

Update explainer.md

966b2cc

Update explainer.md

009a61b

Update index.bs

ae06c70

Update index.bs

2100bfc

Update index.bs

3f4cfa1

jiajiabingcheng requested a review from chrisn February 27, 2024 01:15

chrisn approved these changes Feb 27, 2024

View reviewed changes

youennf reviewed Mar 6, 2024

View reviewed changes

jiajiabingcheng added 2 commits March 6, 2024 17:10

Update index.bs

63c64c7

Update explainer.md

dbf1354

steimelchrome approved these changes Apr 10, 2024

View reviewed changes

steimelchrome merged commit c7a47fb into w3c:main Apr 10, 2024
1 check passed

github-actions bot added a commit that referenced this pull request Apr 10, 2024

Update index.bs with ChapterInformation (#308)

1a87b1e

SHA: c7a47fb Reason: push, by steimelchrome Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

		Run the <a>convert artwork algorithm</a> with <var>init</var>'s
		{{ChapterInformationInit/artwork}} as <var>input</var> and set

Update index.bs with ChapterInformation #308

Update index.bs with ChapterInformation #308

Conversation

jiajiabingcheng commented Dec 13, 2023 • edited by pr-preview bot

steimelchrome left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jiajiabingcheng commented Dec 20, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

youennf left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jan-ivar Jan 26, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jiajiabingcheng commented Feb 6, 2024

Choose a reason for hiding this comment

jiajiabingcheng commented Feb 13, 2024

jiajiabingcheng commented Feb 21, 2024

chrisn left a comment

Choose a reason for hiding this comment

jan-ivar left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jan-ivar Feb 23, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jiajiabingcheng commented Feb 27, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jiajiabingcheng commented Mar 7, 2024

jiajiabingcheng commented Mar 19, 2024

jiajiabingcheng commented Dec 13, 2023 •

edited by pr-preview bot

jan-ivar Jan 26, 2024 •

edited

jan-ivar Feb 23, 2024 •

edited