Federation

[craigmaloney]

Federation is a honkin' great idea. Let's do more of that.

[craigmaloney]

marked for 1.0

[mitechie]

Need an API for installs to call each other:

• api call to list/verify user
• list bookmarks, by tag, recent, etc
• provide rss feeds of those so the app can just subscribe to rss feeds and update that locally
• Dare we try to do a search on friend's bookmarks? Seems scary performance-wise

[craigmaloney]

Wondering if we could create something where the transactions are logged, ie:

• Timestamp
• shaNsum of current bookmark
• operation code (add, edit bookmark, edit tags, delete bookmark)
• payload (new bookmark, new tags)
• shaNsum of new bookmark

Then, when another remote instance calls for the latest changes, it could gather what it needs, and be off on it's merry way.

Going to look into what Status.net does for their API calls.

[mitechie]

So what all are we expecting the federation to do then? Is it actually sharding/storing the changes on each site? I guess that's what status.net would do, but hadn't thought about copying the actual urls across. Just providing some basic 'I can see your latest stuff over here' thing.

But yea., if we do copy all out then things like search/etc become more feasible.

I have some notes on creating sha-like strings based on the urls so that when we go multi-user we can create a bmark table and share those among users to reduce db storage/overhead. It's much like what the url shorteners do out there. Generate a short url string and use it as the bookmark url id across the app.

[mitechie]

And just for the record, note-like urls can usually be found:
http://rick.bmark.us//tags/bookie

https://github.com/stochastic-technologies/shortuuid

and for federation we need to check out this standard:

http://ostatus.org/

[craigmaloney]

I've overloaded the term federation to mean acting as an endpoint for stuff like status.net, as well as allowing different bookmark sites to communicate with each other. iirc, Each status.net site keeps a local store of the content that it's tracking. I'm not 100% sure, though, and will investigate.

Ideally, I'd like something that would allow anyone to access bookmarks via the twitter API, as well as a bookie API. That way, if someone wanted to follow by bookmark stream, they could follow user@bmark.us, and it would just work. If I wanted to socially interact with someone else's bookmarks, I could subscribe to another user's bookie instance, and have their public-facing content available to me.

[silopolis]

This would be a killer feature for the "socialisation" of Bookie and it's addition to projects like FreedomBox.

[anarcat]

the new cool kid in town for federation is http://pump.io, at least that's what http://mediagoblin.org will be using for sharing content amongst instances.

[jelmer]

Are there any other server implementations of pump.io? The only implementation I can find seems to have a fairly long dependency list, including nodejs.

[mitechie]

Federation seems interesting, but I'm not sure how it fits at this time. What we might do, is look at using the GSoC student's project this summer, that will be implementing saving bookmarks from external sources, to auto save bookmarks from another user's account. Perhaps that available on another Bookie instance would suffice?

I guess, what's everyone's wishlist functionality for federation?

I'm definitely not going to go loading nodejs deps and such into Bookie. I've fought a long battle to get install/setup simpler over time and bringing nodejs and npm into the picture isn't on my todo.

[jelmer]

I'd be interested in "following" the bookmarks from friends.

Installing nodejs is definitely also a no-go for me personally.

[mitechie]

Ok, yea I think we can do that sans federation though. Bookie provides RSS feeds of almost any URL (search/tag filter) and the ability to maybe add some feed into your stream of bookmarks might get the job done in a simpler fashion.

[jelmer]

Personally, being able to subscribe to an RSS feed with links would be sufficient.

[craigmaloney]

There's a program I'm using called "Spigot" which has a python library that supports sending content to pump.io. If there's a desire for Pump.io support this would be my preferred method. :)

[anarcat]

so the pump.io protocol doesn't depend on nodejs by itself. this is why i mentionned mediagoblin: those guys are working on a Python interface to the pump.io API: http://pypump.org/en/latest/

if you don't want to add that as a dependency, you can use the API directly: https://github.com/e14n/pump.io/blob/master/API.md

as that document explains, it's really just a matter of exposing a few things: an activity stream, some webfinger magic and oauth. i don't know how i feel about it personnally, but seeing mediagoblin join in seems to make me think it's promising...

for me, typical use-case scenarios would be:

1. i am a fan of george, and i want to see all the bookmarks he posts - i go on his page and click "subscribe", which asks me for my bookie url for registration. after i register, i see his posts in my stream of bookmarks
2. i want to share a bookmark with george: there's a "share" button on my bookie which allows me to share a resource with him

this is the idea here: to integrate other install's stream of bookmarks into my install without having to host them myself. it's like what RSS feeds provide, but it's as if bookie was an RSS reader...