"Media Source" concept

[erincandescent]

Currently, the draft leaves it somewhat unclear how an agent is supposed to locate and choose the media resource associated with objects (e.g. an image accompanying a "note", or the video behind an actual video object)

I do not attempt to fully clarify that issue here. Instead, I propose a mechanism by which we may do so: The "Media Source" object. This is based upon evolving and clarifying the "Media Link" object from AS1.

Media Source objects

Media Sources are JSON Objects which define an "embedded rich content" resource; they serve a purpose analogous to the element in HTML. They have the following attributes, which bear the same name and definition as the corresponding attribute in HTML5 where applicable:

• src: contains the URI of the media file in question. All Media Sources must contain a src attribute.
• type: If present, must contain a valid MIME type, which is the same as the MIME type of the resource pointed to by src. All Media Sources should provide a type attribute.
• width and height: If present, these properties declare the width and height, in pixels, of the linked media. Sources with a visual component should provide width and height attributes; sources without a visual component must not provide width and height attributes.

(Open question: do we also provide the media attribute from HTML?)

A Media Source object may be abbreviated to a string which contains the value of the src property.

Media Source lists

A Media Source list is a JSON array of zero or more Media Source objects, all of which correspond to different encodings of the same resource. The list should be sorted from most preferred version to least preferred (where the most preferred version need not correspond to the highest quality version); clients may implement their own preference system.

A Media Source list may be abbreviated to a single Media Source object, which may itself be abbreviated. This corresponds to a single element array.

[jasnell]

Not clear how the spec is unclear on this. For instance, let's look at the two examples you give:

1. "an image accompanying a "note""

The current spec vocabulary includes a well defined "image" property which would be used as:

{
  "objectType": "note",
  "content": "this is a note",
  "image": "http://example.org/foo.png"
}

Or, if you wanted to associate more metadata with that, such as alternative text... you could do...

{
   "objectType": "note",
  "content": "this is a note",
  "image": {
    "id": "http://example.org/foo.png",
    "displayName": "A picture of a Foo"
  }
}

How to get that picture information seems pretty clear to me. What am I missing?

1. "or the video behind an actual video object"

Again, this seems pretty clear to me:

{
  "objectType": "video",
  "id": "http://example.org/foo.mpg",
  "url": "http://example.org/foo.mpg",
  "mediaType": "video/mpeg",
  "displayName": "A video of my pet Foo"
}

I'm not certain what is unclear on this.

As far as alternate formats is concerned... suppose I have multiple encodings of my video above. Using the current syntax, that would be as simple as:

{
  "objectType": "video",
  "id": "http://example.org/foo.mpg",
  "url": "http://example.org/foo.mpg",
  "mediaType": "video/mpeg",
  "displayName": "A video of my pet Foo",
  "alternate": [
    {
      "id": "http://example.org/foo.avi",
      "mediaType": "video/avi"
    },
    {
      "id": "http://example.org/foo.mkv",
      "mediaType": "video/mkv"
    }
  ]
}

Again, not certain what is lacking here.

[erincandescent]

1. Which property is preferred for the video file, "id" or "url"? Presumably "id", because "url" per current documentation is basically a webpage link
2. What if both "id" and "url" are already employed for their existing purposes?
3. What are the "alternate"s alternatives to? I don't think they're really alternatives to the "conceptual object". If anything, the "alternate" relation in AS2 should only really apply to alternative representations of the AS2 document describing the object - e.g. some hypothetical XML serialization or something
4. Are the different variants really supposed to be fully fledged Objects? Please, hell, no. You have no idea how difficult that makes processing the things. I really do not want duplicate objects sprouting up everywhere in my database, never mind the semantic hell that brings.
5. Particularly for videos, applications will probably want to be able to discriminate between videos of differing resolutions; hence the width and height properties

[jasnell]

1. "id" provides the permanent identifier for the object, which may or may not be dereferenceable. It is equivalent to the "@id" property in JSON-LD. The "url" property would be used to provide the default/preferred representation. If the "url" is not specified, an implementation can attempt to fallback to the "id".
2. See above. In AS2, "url" is not defined as only pointing to a webpage link. That's a change relative to AS1
3. When I have {"objectType": "video"}, then I view that as one alternative (logical) representation of the video itself. The "alternate" provides the other alternate representations.
4. Keep in mind that I have implemented all of this. I know exactly how it impacts processing. Implementations are free to interpret the JSON however they'd like. There is precious little difference between:

{
  "image": {
    "id": "http://example.org/foo.png"
  }
}

and

{
  "image": {
    "src": "http://example.org/foo.png"
  }
}

Map that however you'd like to your own implementation.

5...

{
  "objectType": "video",
  "displayName": "A video of my pet Foo",
  "url": "http://example.org/foo_smaller.mpg",
  "height": "100",
  "width": "100",
  "alternate": {
      "id": "http://example.org/foo_larger.mpg",
      "height": 400,
      "width": 400
    }  
}

height and width are already defined in the vocabulary.

[erincandescent]

1. "id" provides the permanent identifier for the object, which may or may not be dereferenceable. It is equivalent to the "@id" property in JSON-LD. The "url" property would be used to provide the default/preferred representation. If the "url" is not specified, an implementation can attempt to fallback to the "id".
   See above. In AS2, "url" is not defined as only pointing to a webpage link. That's a change relative to AS1

This returns us to the issue of finding the webpage representing an object again, or is that another alternate? Where do I put the link to the webpage identifying an image? How do I differentiate between this and using url for the webpage? Why do I need different logic to find the webpage for different object types? What are the semantics of all of these "alternate" objects being introduced?

When I have {"objectType": "video"}, then I view that as one alternative (logical) representation of the video itself. The "alternate" provides the other alternate representations.

What are the semantics of this alternate object? What if it has a "content" member? What if it's of a different object type? How does this tie in with the alternate link relation, which is specified in terms of MIME types?

Keep in mind that I have implemented all of this. I know exactly how it impacts processing. Implementations are free to interpret the JSON however they'd like. There is precious little difference between:

No, there is a massive difference! If the image points to an object, then now my code has to go and chase that image object out of the database. If the alternatives point to objects, I need to go and chase all of those alternatives out of the database.

For various very good reasons I store my ActivityStreams content normalised - i.e. every object is a separate DB entry, just storing the id of any object it references. Each of these "alternate" objects, each of these "image" objects now becomes a separate object. This really doesn't make sense to me - most articles don't have images as separately considered objects, for example. The different encodings of a video aren't really distinct objects. Hell, what are the semantics of commenting on an alternate?

This is why I find the proliferation of objects in AS2 disturbing; because it's this thing which makes the spec nice and simple and beautiful and makes many kinds of implementation absolute hell.

[jasnell]

Let's suppose I have an object that represents an article... and let's suppose that we're following the guidelines currently described here: http://jasnell.github.io/w3c-socialwg-activitystreams/activitystreams2.html#link-best-practices

{
  "objectType": "article",
  "content": "<p>This is not a very good article.</p>",
  "id": "http://example.org/articles/1",
  "url": "http://example.org/articles/1",
  "mediaType": "text/html"
}

It ought to be pretty obvious how you navigate to the HTML version of the article. Let's suppose that article has a PDF alternate...

{
  "objectType": "article",
  "content": "<p>This is not a very good article.</p>",
  "id": "http://example.org/articles/1",
  "url": "http://example.org/articles/1",
  "mediaType": "text/html",
  "alternate": {
    "id": "http://example.org/articles/1.pdf",
    "url": "http://example.org/articles/1.pdf",
    "mediaType": "application/pdf"
  }
}

Again, how we get the PDF alternate should be obvious. Now, let's suppose you have a video object...

{
  "objectType": "video",
  "mediaType": "image/mpeg",
  "id": "http://example.org/foo.mpg",
  "url": "http://example.org/foo.mpg",
  "alternate": {
    "id": "http://example.org/player?v=foo.mpg",
    "url": "http://example.org/player?v=foo.mpg",
    "mediaType": "text/html"
  }
}

In either case, the process of locating the text/html page to display is the same: If you want the text/html, look to see if the default url will give you what you want, and if not, look at the available alternates.

One disconnect that I can see in the way you and I are discussing this is that while I see AS2 as purely syntactic model, you appear to be taking the next step further into viewing it as a semantic model. When I say "object"... I literally mean it's just a JSON object. That's why I argue that there's no "real" difference between "id": "http://.../foo" and "src": "http://.../foo". Syntactically they're effectively identical. That's where the AS2 spec currently focuses.

If the WG feels that a semantic model is better, then that can be worked on as an alternative.

[jasnell]

I believe this is addressed by the current vocab proposal. See the definition of the Link class.