Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Representing alternate post types #51

Open
cleverdevil opened this issue May 19, 2017 · 21 comments

Comments

Projects
None yet
4 participants
@cleverdevil
Copy link

commented May 19, 2017

I'm very excited by what I'm seeing in JSONFeed, as it greatly simplifies both the generation of and the consumption of feeds. On my website, I have implemented initial support for JSONFeed:

That said, in my implementation process, I discovered that I would have liked to have a way to express post types like Photos, Likes, Reposts, Bookmarks, Checkins, and other post types that are common in IndieWeb sites that leverage Microformats2, etc.

Currently, I'm doing this using an _indieweb extension in the JSON. Here are a few samples:

For a Checkin:

{
  "_indieweb": {
    "address": "310 Vista del Mar Ste C, Redondo Beach, CA, United States, 90277",
    "lat": 33.818167005599,
    "long": -118.38519045213,
    "placename": "Riviera Barber Shop",
    "type": "checkin"
  },
  "author": {
    "avatar":
    "https://cleverdevil.io/file/2fa19f964fb8970faaf20b909c69d6cb/thumb.png",
    "name": "Jonathan LaCour",
    "url": "https://cleverdevil.io/profile/cleverdevil"
  },
  "content_text": "Checked into Riviera Barber Shop",
  "date_published": "2017-05-19T18:26:35+00:00",
  "id": "https://cleverdevil.io/view/340d808fc59ec35418ca18c66ab0087f",
  "title": "Checked into Riviera Barber Shop",
  "url": "https://cleverdevil.io/2017/checked-into-riviera-barber-shop-1"
}

Here is an example for a "Like" of some content elsewhere on the web:

{
  "_indieweb": {
    "like-of": "https://twitter.com/cashlock/status/864999780178477056",
    "type": "like"
  },
  "author": {
    "avatar": "https://cleverdevil.io/file/2fa19f964fb8970faaf20b909c69d6cb/thumb.png",
    "name": "Jonathan LaCour",
    "url": "https://cleverdevil.io/profile/cleverdevil"
  },
  "content_html": "...",
  "date_published": "2017-05-18T00:24:55+00:00",
  "external_url": "https://twitter.com/cashlock/status/864999780178477056",
  "id": "https://cleverdevil.io/view/34e874491f8b60b1ae8273f3b9bec08f",
  "title": "Like: Christian Ashlock on Twitter: \"@cleverdevil https://t.co/RW3ddHzeIG\"",
  "url": "https://twitter.com/cashlock/status/864999780178477056"
},

I am not sure it makes sense for every different type of post to be represented, but it would be lovely if at least the basics were represented, or if there was an optional guideline or spec for extensions to enable feed readers to be able to represent and differentiate long-form posts, status updates, photos, likes, bookmarks, and reposts.

@manton

This comment has been minimized.

Copy link
Collaborator

commented May 25, 2017

Thanks for writing this up, @cleverdevil! I think this is going to be a common request as well, and I'm looking forward to seeing what other people do for post types. I'm going to point people to this issue when they ask about it.

For compatibility with Microformats, shouldn't the checkin have e.g. latitude instead of lat, to match p-latitude? Some inconsistency between the formats is probably unavoidable, though (like hyphens vs. underscores).

@cleverdevil

This comment has been minimized.

Copy link
Author

commented May 28, 2017

FYI, I've updated the pull request to add this to Known, and have relocated the feeds (original comment edited for clarity). I've also changed the lat to latitude and the long to longitude. I'm sure that there are other inconsistencies, but until the extensions for this type of data become more fleshed out, this will do!

@cleverdevil

This comment has been minimized.

Copy link
Author

commented May 28, 2017

FYI, the pull request has been merged into Known, so it now features JSON Feed support.

It'd be great to get some feedback on the _indieweb extensions and see if maybe some agreement/consensus can develop.

@voxpelli

This comment has been minimized.

Copy link

commented May 29, 2017

@cleverdevil _indieweb sounds odd as an extension name. There's no such thing as an indieweb format.

Maybe you're thinking Microformats 2 or some alternate representation of it, like jf2?

Microformats 2 is widely used within the Indieweb community but isn't exclusively used within the community, it's also used elsewhere and as such a name pointing out that it's mf2-based would be preferable. Also: the indieweb community is using other types of formats as well and is a fan of plurality, so naming a format plainly just indieweb doesn't describe the format that well.

I would suggest to use either the mf2 JSON representation that Micropub uses or if you're aiming for something simpler, the simplified jf2 JSON representation – and then just embed that data within a _mf2 or _jf2 extension

@manton

This comment has been minimized.

Copy link
Collaborator

commented May 29, 2017

The extension name doesn't necessarily need to describe the format. For example, you could imagine extensions for companies or web sites. _indieweb is kind of consistent with that. (I'd suggest _microformats but that makes it sound like an array of objects, so that's not quite right either.)

@voxpelli

This comment has been minimized.

Copy link

commented May 29, 2017

For reference, had my my social feed on-the-fly converted to jf2.

A like extracted form that feed looks like:

{
    "name": "Liked\n        \n          http://keithjgrant.com/replies/2017/03/66842/",
    "url": "https://voxpelli.com/social/2017/03/55604/",
    "like-of": "http://keithjgrant.com/replies/2017/03/66842/",
    "published": "2017-03-02T15:26:44+00:00",
    "type": "entry",
    "uid": "https://voxpelli.com/social/2017/03/55604/"
}

So making that a _jf2 key and maybe removing redundant keys would result in something very similar to the original suggestion, without inventing a new format:

"_jf2": {
  "like-of": "http://keithjgrant.com/replies/2017/03/66842/",
  "type": "entry"
}

Also for reference, one can also find a representation of the full mf2 json of the feed.

@cleverdevil

This comment has been minimized.

Copy link
Author

commented May 29, 2017

@voxpelli I don't have an issue changing the _indieweb key to something else, or changing the attributes I've added so far to match other, existing formats like jf2. In fact, that makes a lot of sense to me! That said, I'd hesitate to call the key _jf2, because its not actually jf2, its just something similar to jf2. Maybe I should just change it to something more generic, like _meta?

Is there somewhere that shows a list of all of the possible properties that can be specified in jf2 ( everything from likes and bookmarks, to locations, etc.)? If so, I can quickly go through and update Known's implementation to match as many properties as possible, along with changing the extension key name to something more appropriate.

@voxpelli

This comment has been minimized.

Copy link

commented May 29, 2017

@cleverdevil jf2 is still as far as I know a work in progress and it be great if JSON Feed and jf2 could start to converge with one another rather than drift further apart.

According to the wiki the primary development happens at http://dissolve.github.io/jf2/, lead by @dissolve

I think it would be great if the Microformats/IndieWeb community could keep the number of JSON formats fairly low – having the original full Microformats 2 one that eg. Micropub uses and then this simplified jf2 one for use cases like this. It makes it a lot easier to map data across different sources without needless and possibly complex conversions between representations.

@dissolve

This comment has been minimized.

Copy link

commented May 29, 2017

Jf2 is just a straight preprocessed version of mf2's Json equivalent format. Which means it's vocabulary unaware. In other words, and entry could be a single value or a array of values. I was just looking at more in depth comparison today actually.

From my understanding, the reason people don't want to use mf2 directly, is sort of 3 fold.
1: they don't want to deal with a parser library (this is sort of a poor argument in my opinion, since you need a Json parser anyway, even in JavaScript you should not be processing it directly).
2: mf2's Json equivalent is a overly verbose for the common case. Everything is always an array even though the majority of the time, it's a single value. This is what jf2 was really an attempt to solve. Many did this conversion implicitly and so it was an attempt to give people a hint on how they can store internally in a simpler structure that is equivalent to the original mf2.
3: you do not know that values will always be there or if things will be an array, a string, or a structure. Realistically, you have to deal with cases where things are missing or misformatted. However, you need some for of baseline of what is supposed to be there. This is what's missing. My plan is to create a profile of jf2 that closely matches jsonfeed, noting what fields MUST/SHOULD be present. It would still use microformats2 vocabulary, so extension can be done there, and any parser must not halt on unknown values. This allows for the additional fidelity that you lose when you covert to a different, more limited vocabulary.

With a few name changes, it would basically mean you could parse Mastodon feeds (for example) directly to jf2/feed with very little additional processing (all of which would be specified in the profile.

@cleverdevil

This comment has been minimized.

Copy link
Author

commented May 30, 2017

My plan is to create a profile of jf2 that closely matches jsonfeed, noting what fields MUST/SHOULD be present. It would still use microformats2 vocabulary, so extension can be done there, and any parser must not halt on unknown values. This allows for the additional fidelity that you lose when you covert to a different, more limited vocabulary.

Cool. So, it sounds like @dissolve has a plan to come up with a jf2 profile. Once that's done, let us know, and I am happy to push a new update to the Known JSON Feed implementation that takes into account the fields that are present. Does that sound reasonable to you?

@dissolve

This comment has been minimized.

Copy link

commented May 30, 2017

Here is an initial pass at what i am talking about. Let me know your opinions. http://dissolve.github.io/jf2/#profiles

@manton

This comment has been minimized.

Copy link
Collaborator

commented May 30, 2017

Thanks for writing this up, @dissolve. In the context of the _indieweb extension above, one area that might be worth discussing is how type is used. The JF2 profile says it must always be "entry", compared to also allowing "checkin" or "like" as @cleverdevil used. If it follows MF2, wouldn't any of the h- names also be valid types? (Disclaimer: I'm just asking questions and listening at this point. Not advocating for any particular approach.)

@dissolve

This comment has been minimized.

Copy link

commented May 30, 2017

No need for the disclaimer @manton, at this point its all just experimentation. Thanks for looking it over.

Type in JF2 is derived from h-* so they are all h-entry items. h-like and h-checkin don't actually exist.
Figuring out a post type below that would be another level thats sort of outside of it. http://tantek.github.io/post-type-discovery/ could be used from there to figure out some x-post-type field for example. But the problem with post types is they can really get jumbled up if something has a location, has a photo, but no text. is it a checkin with a photo or is it a photo with a location? And what happens if you add a 'like-of' to it? Since JF2 says you have to ignore anything you don't understand, you could add a field like this without breaking anything, but that field currently doesn't exist in MF2.

That said, this would not then work for a feed of h-reviews or h-cards. That could certainly be relaxed, but I wasn't sure that would be useful or not. My biggest concern was having h-cards and h-entries mixed in when someone was doing a feed and didn't actually tag the h-card as an author.

@cleverdevil

This comment has been minimized.

Copy link
Author

commented May 30, 2017

Personally, I like having a type field, in spite of the fact that it can be ignored. Sure, consumers could go through the "post type discovery" algorithm that Tantek outlines, but it seems like a simple "type" specifier could really help. I don't think it necessarily prevents people from including location attributes on a review post type, for example, but it could enable consumers to more easily segment content into different views, or to alter presentation based upon what the publisher intended the post type to be.

I like the start, @dissolve. I still think it'd be really nice to have a central place to see all of the fields for every known vocabulary, as that would definitely help me to properly implement everything in Known as a good reference implementation. Are you planning on addressing that in your doc, or should that be separate?

@dissolve

This comment has been minimized.

Copy link

commented May 30, 2017

@cleverdevil certainly i think having x-post-type (my suggestion) is fine, its just not something you can depend on (for those that auto generate from the mf2) unless its something added via mf2.
thats why i suggest the x-* until its something that sees common usage and gets added to

I'm not sure quite what you mean at the end there. Its using
http://microformats.org/wiki/h-entry
or possibly
http://microformats.org/wiki/h-feed

You could certainly propose post-type there which would be interesting to see, I would certainly be fine publishing that as well.
Might be interesting to see with some of the joking types like http://indieweb.org/snark or @benwerd's
http://indieweb.org/chicken

@cleverdevil

This comment has been minimized.

Copy link
Author

commented May 30, 2017

Thanks, @dissolve, I think the h-entry page on microformats.org has all of the details I'm looking for. By my read, it means that the extension (and its implementation in Known), which I believe I will call _meta for now, should be updated to generate the following properties:

  • in-reply-to - the URL which the entry is considered reply to
  • rsvp - "yes", "no", "maybe", "interested"
  • like-of - the URL which the post is considered a “like” (favorite, star) of.
  • repost-of - the URL which the post is considered a “repost” of.
  • bookmark-of - indicates this post is a bookmark of another URL.
  • featured - a representative photo or image for the entry, e.g. primary photo for an article or subject-cropped photo, suitable for use in a link-preview.
  • latitude - latitude of the location of the entry
  • longitude - latitude of the location of the entry
  • location - debating how to implement this one... Known has lat, long, address, and placename. I'll obviously toss the lat and long into the associated properties above, but the location property as specified should be an h-adr. I might just include address and placename in addition to latitude and longitude, and skip location. Open to feedback, of course :)

I'll likely still include the type for the aforementioned reasons, but it could be ignored, and the data could be run through the post discovery algorithm, if desired.

Thoughts?

@manton

This comment has been minimized.

Copy link
Collaborator

commented May 30, 2017

For featured, this is probably the same as JSON Feed's image field. I think you could reference it there and leave it out of _meta.

@cleverdevil

This comment has been minimized.

Copy link
Author

commented May 31, 2017

I've got an initial implementation of this I completed last night here:

cleverdevil/Known@9fc50f5

I'll be submitting a PR to Known after some testing.

@cleverdevil

This comment has been minimized.

Copy link
Author

commented Jun 24, 2017

FYI, this pull request has been merged into Known, so now all Known sites will support this extension, along with JSON Feed.

@manton

This comment has been minimized.

Copy link
Collaborator

commented Jun 24, 2017

Excellent, thanks @cleverdevil!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.