Skip to content

Commit

Permalink
Manifest-icon-like metadata artworks. (#129)
Browse files Browse the repository at this point in the history 
Make the artwork spec more like WebApp manifest icons, which has `src`,
`type` and `sizes` fields.

Fixes #58
  • Loading branch information
@xxyzzzq
xxyzzzq authored Jul 4, 2016
1 parent 6f5ccf9 commit 869835c
Show file tree
Hide file tree
Showing 2 changed files with 428 additions and 46 deletions.
212 changes: 205 additions & 7 deletions mediasession.bs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,13 @@ urlPrefix: https://html.spec.whatwg.org/multipage/; spec: HTML
type: dfn
urlPrefix: infrastructure.html
text: case-sensitive; url: #case-sensitivity-and-string-comparison
text: ASCII case-insensitive; url: #ascii-case-insensitive
text: reflect
text: limited to only known values
text: remove an element from a document; url: #remove-an-element-from-a-document
text: in parallel
text: unordered set of unique space-separated tokens; url: #unordered-set-of-unique-space-separated-tokens
text: document base url
urlPrefix: embedded-content.html
text: media element
text: media element load algorithm
Expand All @@ -67,6 +70,8 @@ urlPrefix: https://html.spec.whatwg.org/multipage/; spec: HTML
text: task
text: task source
text: DOM manipulation task source
urlPrefix: semantics.html
text: link; for: HTMLLinkElement; url:#the-link-element
type: method
urlPrefix: embedded-content.html
text: play(); for: HTMLMediaElement; url: #dom-media-play
Expand All @@ -81,14 +86,19 @@ urlPrefix: https://html.spec.whatwg.org/multipage/; spec: HTML
text: networkState; for: HTMLMediaElement; url: #dom-media-networkstate
text: NETWORK_LOADING; for: HTMLMediaElement; url: #dom-media-network_loading
text: NETWORK_IDLE; for: HTMLMediaElement; url: #dom-media-network_idle
urlPrefix: semantics.html
text: sizes; for: HTMLLinkElement; url: #attr-link-sizes;
type: event
urlPrefix: embedded-content.html
text: pause; url: #event-media-pause
text: play; url: #event-media-play
urlPrefix: https://url.spec.whatwg.org/;
urlPrefix: https://url.spec.whatwg.org/; spec: URL
type: dfn; urlPrefix: #concept-
text: url parser
text: event listener
type: dfn
text: absolute URL; url: #syntax-url-absolute
text: relative URL; url: #syntax-url-relative
urlPrefix: https://fetch.spec.whatwg.org/; spec: FETCH
type: dfn; urlPrefix: #concept-
text: fetch
Expand Down Expand Up @@ -132,6 +142,9 @@ urlPrefix: http://www.w3.org/TR/page-visibility/; spec: PAGE-VISIBILITY
type: enum; urlPrefix: #pv-page-
text: visible
text: hidden
urlPrefix: https://www.w3.org/TR/appmanifest/; spec: appmanifest
type: dfn
text: image object; url: #dfn-image-object
urlPrefix: https://heycam.github.io/webidl/
type: exception
text: InvalidStateError
Expand Down Expand Up @@ -289,8 +302,7 @@ attribute must return the <a>media session</a>'s <a>kind</a>.

The <dfn attribute for="MediaSession"><code>metadata</code></dfn> attribute, on
getting, must return the <a>media session</a>'s <a lt="media session
metadata">metadata</a>. On setting, the <a>media session</a>'s <a lt="media
session metadata">metadata</a> must be set to the new value.
metadata">metadata</a>.

The <dfn method for="MediaSession"><code>activate()</code></dfn> method, when
invoked, must run these steps:
Expand Down Expand Up @@ -1123,15 +1135,22 @@ interface MediaMetadata {
readonly attribute DOMString title;
readonly attribute DOMString artist;
readonly attribute DOMString album;
[SameObject] readonly attribute FrozenArray&lt;MediaImage> artwork;
};

dictionary MediaMetadataInit {
DOMString title = "";
DOMString artist = "";
DOMString album = "";
sequence&lt;MediaImageInit> artwork = [];
};
</pre>

A {{MediaMetadata}} object has a <dfn for="MediaMetadata">title</dfn>, an <dfn
for="MediaMetadata">artist</dfn>, an <dfn for="MediaMetadata">album</dfn> and a
FrozenArray of <dfn for="MediaMetadata" title="artwork image">artwork
images</dfn>.

The <dfn constructor
for="MediaMetadata"><code>MediaMetadata(<var>init</var>)</code></dfn>
constructor, when invoked, must run the following steps:
Expand All @@ -1141,16 +1160,164 @@ constructor, when invoked, must run the following steps:
Let <var>metadata</var> be a new {{MediaMetadata}} object.
</li>
<li>
Set <var>metadata</var>'s {{MediaMetadata/title}}, {{MediaMetadata/artist}},
and {{MediaMetadata/album}} attributes to the values of <var>init</var>'s
{{MediaMetadataInit/title}}, {{MediaMetadataInit/artist}}, and
{{MediaMetadataInit/album}} members, respectively.
Set <var>metadata</var>'s {{MediaMetadata/title}} to <var>init</var>'s
{{MediaMetadataInit/title}}.
</li>
<li>
Set <var>metadata</var>'s {{MediaMetadata/artist}} to <var>init</var>'s
{{MediaMetadataInit/artist}}.
</li>
<li>
Set <var>metadata</var>'s {{MediaMetadata/album}} to
<var>init</var>'s {{MediaMetadataInit/album}}.
</li>
<li>
Set <var>metadata</var>'s {{MediaMetadata/artwork}} using <var>init</var>'s
{{MediaMetadataInit/artwork}} by calling the
<a><code>MediaImage(init)</code></a> constructor.
</li>
<li>
Return <var>metadata</var>.
</li>
</ol>

The <dfn attribute for="MediaMetadata"><code>title</code></dfn> attribute must
return the {{MediaMetadata}} objects's <a>title</a>.

The <dfn attribute for="MediaMetadata"><code>artist</code></dfn> attribute must
return the {{MediaMetadata}} objects's <a>artist</a>.

The <dfn attribute for="MediaMetadata"><code>album</code></dfn> attribute must
return the {{MediaMetadata}} objects's <a>album</a>.

The <dfn attribute for="MediaMetadata"><code>artwork</code></dfn>
attribute must return the {{MediaMetadata}} objects's <a for="MediaMetadata"
title="artwork image">artwork images</a>, as a FrozenArray of {{MediaImage}}s.
The <a attribute for="MediaMetadata">artwork</a> attribute can be empty.

When a user agent wants to display the {{MediaSession}}'s artwork as specified
in it's <var>metadata</var>, it is recommended to run the following <dfn>fetch
steps</dfn>:

<ol>
<!-- XXX https://www.w3.org/Bugs/Public/show_bug.cgi?id=24055 -->
<li>
If <var>metadata</var>'s <a attribute
for="MediaMetadata"><code>artwork</code></a> is empty, then terminate these
steps.
</li>
<li>
If the platform supports displaying media artwork, select a <dfn>prefered
artwork image</dfn>
from <var>metadata</var>'s <a attribute
for="MediaMetadata"><code>artwork</code></a>.
</li>
<li>
<a title="fetch">Fetch</a> <a>prefered artwork image</a>'s
{{MediaImage/src}}.

Then, <a>in parallel</a>:

<ol>
<li>
Wait for the <a>response</a>.
</li>
<li>
If the <a>response</a>'s <a>internal response</a>'s
<a lt="response type">type</a> is <i>default</i>, attempt to decode the
resource as image.
</li>
<li>
If the image format is supported, use the image as the artwork for
display. Otherwise the <a>fetch steps</a> fail and terminate.
</li>
</ol>
</li>
</ol>

If no artwork images are fetched in the <a>fetch steps</a>, the user agent may
have fallback behavior such as displaying an default image as artwork.

<h2 id="the-mediaimage-interface">The {{MediaImage}} interface</h2>

<pre class="idl">

[Constructor(MediaImageInit init)]
interface MediaImage {
readonly attribute USVString src;
readonly attribute DOMString sizes;
readonly attribute DOMString type;
};

dictionary MediaImageInit {
USVString src = "";
DOMString sizes = "";
DOMString type = "";
};
</pre>

A {{MediaImage}} object has a <dfn for="MediaImage">source</dfn>, a list of
<dfn for="MediaImage">sizes</dfn>, and a <dfn for="MediaImage">type</dfn>.

The <dfn constructor
for="MediaImage"><code>MediaImage(<var>init</var>)</code></dfn>
constructor, when invoked, must run the following steps:

<ol>
<li>
Let <var>metadata</var> be a new {{MediaImage}} object.
</li>
<li>
Set <var>metadata</var>'s {{MediaImage/src}} to <var>init</var>'s
{{MediaImageInit/src}}. If the URL is a
<a>relative URL</a>, it must be resolved to an <a>absolute URL</a> using the
<a>document base URL</a>.
</li>
<li>
Set <var>metadata</var>'s {{MediaImage/sizes}} to <var>init</var>'s
{{MediaImageInit/sizes}}.
</li>
<li>
Set <var>metadata</var>'s {{MediaImage/type}} to <var>init</var>'s
{{MediaImageInit/type}}.
</li>
<li>
Return <var>metadata</var>.
</li>
</ol>


The MediaImage <a attribute for="MediaImage">src</a>, <a attribute
for="MediaImage">sizes</a> and <a attribute for="MediaImage">type</a>
inspired from the <a lt="image object">image objects</a> in Web App Manifest.

The <dfn attribute for="MediaImage">src</dfn> attribute must return the
{{MediaImage}} object's <a for="MediaImage">source</a>. It is a URL from which
the user agent can fetch the image's data.

The <dfn attribute for="MediaImage">sizes</dfn> attribute must return the
{{MediaImage}} object's <a for="MediaImage">sizes</a>. It follows the spec of <a
attribute for="HTMLLinkElement"><code>sizes</code></a> attribute in HTML
<a for="HTMLLinkElement"><code>link</code></a> element, which is a string
consisting of an <a>unordered set of unique space-separated tokens</a> which are
<a>ASCII case-insensitive</a> that represents the dimensions of an image. Each
keyword is either an <a>ASCII case-insensitive</a> match for the string "any",
or a value that consists of two valid non-negative integers that do not have a
leading U+0030 DIGIT ZERO (0) character and that are separated by a single
U+0078 LATIN SMALL LETTER X or U+0058 LATIN CAPITAL LETTER X character. The
keywords represent icon sizes in raw pixels (as opposed to CSS pixels). When
multiple image objects are available, a user agent may use the value to decide
which icon is most suitable for a display context (and ignore any that are
inappropriate). The parsing steps for the <a attribute
for="MediaImage">sizes</a> attribute must follow <a attribute
for="HTMLLinkElement" lt="sizes">the parsing steps for HTML <code>link</code>
element <code>sizes</code> attribute</a>.

The <dfn attribute for="MediaImage">type</dfn> attribute must return the
{{MediaImage}} object's <a for="MediaImage">type</a>. It is a hint as to the
media type of the image. The purpose of this attribute is to allow a user agent
to ignore images of media types it does not support.

<h2 id="extensions-to-the-htmlmediaelement-interface">Extensions to the
{{HTMLMediaElement}} interface</h2>

Expand Down Expand Up @@ -1353,8 +1520,38 @@ When the user agent is to <dfn>resume a web audio object</dfn> for a given
title: "Episode Title",
artist: "Podcast Host",
album: "Podcast Title",
artwork: [{src: "podcast.jpg"}]
});
</pre>

Alternatively, providing multiple <a for="MediaMetadata" title="artwork
image">artwork images</a> in the metadata can let the user agent be able to
select different artwork images for different display purposes and better fit
for different screens:

<pre class="lang-javascript">
audio.session.metadata = new MediaMetadata({
title: "Episode Title",
artist: "Podcast Host",
album: "Podcast Title",
artwork: [
{src: "podcast.jpg", sizes: "128x128", type: "image/jpeg"},
{src: "podcast_hd.jpg", sizes: "256x256"},
{src: "podcast_xhd.jpg", sizes: "1024x1024", type: "image/jpeg"},
{src: "podcast.png", sizes: "128x128", type: "image/png"},
{src: "podcast_hd.png", sizes: "256x256", type: "image/png"},
{src: "podcast.ico", sizes: "128x128 256x256", type: "image/x-icon"}
]
});
</pre>

For example, if the user agent wants to use an image as icon, it may choose
<code>"podcast.jpg"</code> or <code>"podcast.png"</code> for a
low-pixel-density screen, and <code>"podcast_hd.jpg"</code>
or <code>"podcast_hd.png"</code> for a high-pixel-density screen. If the user
agent want to use an image for lockscreen background,
<code>"podcast_xhd.jpg"</code> will be prefered.

</div>

<div class="example">
Expand Down Expand Up @@ -1386,6 +1583,7 @@ When the user agent is to <dfn>resume a web audio object</dfn> for a given
title: event.target == audio1 ? "Chapter 1" : "Chapter 2",
artist: "An Author",
album: "A Book",
artwork: [{src: "cover.jpg"}]
});
}

Expand Down
Loading

0 comments on commit 869835c

Please sign in to comment.