Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Ruby implementation of the OStatus protocol suite.
Ruby
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
lib
spec
.gitignore
.travis.yml
Gemfile
LICENSE
README.md
Rakefile
proudhon.gemspec

README.md

Proudhon

Build Status

Proudhon is a collection of classes that parse and generate XML and enable federation in Ruby using the OStatus protocol suite. It provides WebFinger, Atom, ActivityStreams, RSS, PubSubHubbub, Salmon, PortableContacts, GeoRSS and Diaspora compatibility.

Also: http://en.wikipedia.org/wiki/Pierre-Joseph_Proudhon

Installation

gem install proudhon

Usage

require 'proudhon'

Webfinger

We'll start with the Webfinger protocol.

The Webfinger protocol provides identities for your users. It is crucial for discovery and federation. The queries use an e-mail like format, so it works in a per-domain basis. For example, Diaspora offers accounts like yourname@joindiaspora.com, which can be easily queried, first by fetching http://joindiaspora.com/.well-known/host-meta then by fetching the endpoint itself.

Here's the spec: http://tools.ietf.org/html/draft-jones-appsawg-webfinger-00

HostMeta

The /.well-known/host-meta file tell your clients where to find your webfinger endpoint.

We provide a generator:

Proudhon::HostMeta.to_xml('http://domain.com/webfinger/?q={uri}')

This command will generate the following xml:

<?xml version="1.0" encoding="UTF-8"?>
<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0">
  <hm:Host xmlns:hm="http://host-meta.net/xrd/1.0">domain.com</hm:Host>
  <Link type="application/xrd+xml" template="http://domain.com/webfinger/?q={uri}" rel="lrdd">
    <Title>Resource Descriptor</Title>
  </Link>
</XRD>

The {uri} is mandatory. This is a placeholder for the account name.

You should use the XML to answer a GET query to /.well-known/host-meta. Here's how to do it in Sinatra:

get '/.well-known/host-meta' do
    Proudhon::HostMeta.to_xml('http://my-domain.com/webfinger/?q={uri}')
end

Of course, you can add more link tags by using a block:

puts Proudhon::HostMeta.to_xml('http://domain.com/webfinger/?q={uri}') { |xml|
  xml.Link(:rel => 'something', :href => 'http://site.com/')
}

Which renders:

<?xml version="1.0" encoding="UTF-8"?>
<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0">
  <hm:Host xmlns:hm="http://host-meta.net/xrd/1.0">domain.com</hm:Host>
  <Link type="application/xrd+xml" template="http://domain.com/webfinger/?q={uri}" rel="lrdd">
    <Title>Resource Descriptor</Title>
  </Link>
  <Link href="http://site.com/" rel="something"/>
</XRD>

Webfinger

The host-meta contains a path to your webfinger endpoint, which receives a parameter and returns XML. Here's how to generate the XML:

finger = Proudhon::Finger.new(:links => { :hcard => "http://domain.com/user/username" })
finger.to_xml

Which renders:

<?xml version="1.0" encoding="UTF-8"?>
<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0">
  <Link href="http://domain.com/user/username" rel="http://microformats.org/profile/hcard"/>
</XRD>

You can follow the example above to serve it in Sinatra:

get '/webfinger/?q=:uri' do |uri|
  user = Users.find_by_uri(uri)
  finger = Proudhon::Finger.new(:links => { :hcard => "http://domain.com/user/#{params[:uri]}" })

  finger.to_xml
end

Of course, you'll want to return more than just an hCard. Here's a non-extensive list of data you can put on the hash above:

- OStatus specific (To be explained later):
  - :updates_from - Atom feed containing Activities
  - :replies - The user's Salmon endpoint for replies (deprecated but still used by some providers)
  - :mention - The user's Salmon endpoint for mentions (deprecated but still used by some providers)
  - :salmon - The user's Salmon endpoint
  - :magic_key - The user's Magic Key. A public key used to validate the Salmons.

- Not OStatus, but might still be important to the protocol:
  - :atom - Atom feed for subscription
  - :oauth - OAuth endpoint of the user (in case you're using WebFinger as a login strategy)
  - :hcard - HTML page containing an hCard
  - :profile - HTML profile page
  - :poco - HTML page containing PortableContacts
  - :described_by - User's FOAF

- Diaspora-specific
  - :diaspora_key
  - :diaspora_seed
  - :diaspora_guid

The updates_from is mantatory. It provides an Atom Feed that will contain the activies of the user. If you're dealing only with 100% public data, you can implement federation by using only this Atom Feed (coupled with PubSubHubbub).

The replies, mention, salmon and magic_key are, in our case, what makes Salmon work. If you want Salmons to work, you'll need at least a :salmon endpoint and a serialized :magic_key. We'll see about that in a minute.

Querying Webfinger

A remote user can easily be queried:

Proudhon::Finger.fetch('user@domain.com')

You'll get an object of the Finger type. It contains some parameters and a hash of links. Webfinger is used most of the time to help validating Salmons (even though you can use the user's Atom Feed for that) and when 'following' a remote user.

Atom

Atom feeds are a huge part of OStatus. They are where the bulk of user information is contained. They are so important that some implementations ditch Webfinger and go for an Atom-only implementation. Instead of "following" users by the email-like address, you can optionally "follow" by their URL, and, by reading the , you get the URL to the user's Atom Feed. However, if you choose to use Webfinger, they should be referenced on the "updates_from" above.

Atom feeds in OStatus are no different than normal Atom Feeds. Refer to the Atom specs and to the Proudhon documentation for more information about which fields to use and such.

Here's how to query Atom feeds:

atom = Proudhon::Atom.from_uri('http://search.twitter.com/search.atom?q=ruby')

You'll get an Atom object, that can be easily queried. Its structure mirrors the structure of a real Atom feed. Check the documentation to get more information.

atom.title
=> "ruby - Twitter Search"
atom.entries.first.title
=> "Ruby is an amazing language"

And here's how to generate:

atom = Proudhon::Atom.new(:title => 'My Feed', :id => '1234', :entries => [ Proudhon::Entry.new(:title => 'Test', :verb => :post) ])
atom.to_xml

And here's your freshly generated Atom feed (namespaces removed for legibility):

<?xml version="1.0" encoding="UTF-8"?>
<feed ...>
  <id>1234</id>
  <title>My Feed</title>
  <entry>
    <title>Test</title>
    <activity:verb>post</activity:verb>
  </entry>
</feed>

You can serve the generated XML to your users via a feed endpoint. XML constructors on Proudhon follow struct-like semantics on the constructor, so you can use "map" to transforme between a complex database query return and a Proudhon::Atom object, for example. Or you can do things in a more stateful way.

Atom Little-Helpers

Activity Streams, GeoRSS and PortableContacts (POCO) are fields embedded inside Atom documents containing very important semantic metadata concerning the feed entries. PubSubHubbub is (sort of) another addition to Atom that helps distributing feeds between various subscribers without putting the burden on the generator.

Those four tools are also used outside of OStatus. In fact, you can implement federation just with those four, without touching Webfinger and Salmon.

We also included Pingbacks in our implementation, they have a simple implementation and they're already widely used around the web, so it's always a plus.

Activity Streams

Activity Streams are small pieces of information that are coupled with Atom Feeds. They provide standardized verbs for social network communication and represent what your users do. You'll get them automatically if you query a feed with activities.

You can generate them by using the Proudhon::Activity class. They go in the :activity parameter inside each Entry of the Atom Feed.

atom = Proudhon::Atom.new(:title => 'My Feed', :id => '1234', :entries => [ Proudhon::Entry.new(:title => 'Test',
  :verb => :post, :activity => Proudhon::Activity.new(:type => :post, :content => 'Test') ) ])
atom.to_xml

The result (namespaces removed for legibility):

<?xml version="1.0" encoding="UTF-8"?>
<feed ...>
  <id>1234</id>
  <title>My Feed</title>
  <entry>
    <title>Test</title>
    <activity:verb>post</activity:verb>
    <activity:object>
      <content>Test</content>
    </activity:object>
  </entry>
</feed>

Notice the activity:verb and activity:object. Those two are the core elements of Activity Streams, but there's even more. You can look up the specification at http://activitystrea.ms/ . Also, check out the Proudhon HTML documentation to see in what places it can be used.

GeoRSS-Simple

GeoRSS is a universal way to tell a location in an Atom Feed. It is understood by lots of Feed Readers and OStatus implementations. It's pretty easy to use, just a parameter away:

atom = Proudhon::Atom.new(:author => Proudhon::Author.new(:geo_point => '45.256 -71.92' ))
atom.to_xml

And here's the generated XML (namespaces removed for legibility):

<?xml version="1.0" encoding="UTF-8"?>
<feed ...>
  <author>
    <georss:point>45.256 -71.92</georss:point>
  </author>
</feed>

PortableContacts

PortableContacts can be used too, and they go inside Proudhon::Author.

atom = Proudhon::Atom.new(:author => Proudhon::Author.new(:poco_display_name => 'your name'))
atom.to_xml

The result:

<?xml version="1.0" encoding="UTF-8"?>
<feed ...>
  <author>
    <poco:displayName>your name</poco:displayName>
  </author>
</feed>

The following fields are available:

  • poco_id
  • poco_display_name
  • poco_name
  • poco_nickname
  • poco_published
  • poco_updated
  • poco_birthday
  • poco_anniversary
  • poco_gender
  • poco_note
  • poco_preferred_username
  • poco_utc_offset
  • poco_connected

PubSubHubbub

In a federated social newtork, you have multiple users in multiple servers, and you have to keep them updated on what the other users are posting. For instance, suppose I just published a nice story on server 1. How does server 2, your subscriber, knows I posted something? Well, the subscriber could "poll" the publisher every N minutes. But this is problematic in two ways: data can appear up to N minutes late on the subscriber, and if you have too many remote users, you'll end up spending way too many network resources.

The other solution is for the publisher to "push" data into subscribers. Much better, but then the network burden is on the publisher. If you have too many subscribers, you'll need massive network resources on the publisher. It would be much better if we had a separated "mailman" to handle that for us.

PubSubHubbub solves that. It uses a third party server, a "hub", if you may, to transmit the data. Whoever generates Atom Feeds can be a producer. Whoever wants to receive notifications is a subscriber.

With Proudhon, you just need a "hub" server. There are many servers, free and commercial. Since the Protocol is quite simple, you can even roll out your own.

Publishing

In an Atom Feed, you can enable PubSubHubbub by simply the :hub link on your feed:

atom = Proudhon::Atom.new
atom.links[:hub] = 'http://pubsubhubbub.appspot.com/'
...

Most readers also relies on the :self link to figure out where the feed will be hosted, so please include it.

atom.links[:self] = "http://example.com/feed.atom"

After you update a feed, just tell your Hub that it has been updated.

atom.publish
=> ""

That's it. Being a publisher is very easy.

Subscribing

Subscribing is a bit harder, because you need Callbacks to confirm subscriptions and coordinated the messages that arrive.

Just subscribing and unsubscribing is very easy by itself. Here's how it works:

atom.subscribe(your_callback)
=> ""
atom.unsubscribe(your_callback)
=> ""

But notice that you have to pass a parameter called your_callback. It must be an URL that will receive the Atom feeds.

After subscribing or unsubscribing you will to immediately receive a query asking for confirmation in your callback. This is useful to prevent spamming or DDOSing.

The request contains at least three parameters:

  • hub.mode - the command you called: "subscribe" or "unsbuscribe"
  • hub.topic - the URL of the Atom Feed you asked for
  • hub.challenge - the challenge code that must be echoed back

Here's how you would implement a subscriber endpoint in Sinatra:

get "/callback" do
  # Validate those before returning
  topic = params['hub.topic']
  mode = params['hub.mode']

  # If validated, return challenge code to confirm
  if validate_subscription(topic, mode)
    params['hub.challenge']
  end
end

After you validate a subscription, data will start being pushed into your callback. You can receive it by using a POST handler. Your data will be in the request body.

In Sinatra, again:

post '/callback' do
  your_data = request.body.read
end

That's it. By using Atom Feeds and PubSubHubbub, you can effectively have realtime updates of public data between federated nodes.

Salmon

"Salmons" are Activities (just like you saw above) from users outside your network that are of interest to your server, like a reply or a post from a remote user that is followed by a local user.

In practice, a Salmon is a signed Atom Entry that's pushed into a remote server. It is mostly used for replies and mentions, and private communication.

Accepting Salmons

You need and endpoint to accept "salmon slaps". You can use the Webfinger method above, but most implementations rely on the Atom Feed. Just use the :salmon link on your feed:

atom = Proudhon::Atom.new(:links => { :salmon => 'https://website.com/salmon_endpoint' })

Your endpoint is just a POST endpoint with semantics similar to the PubSubHubbub. Here's how you'd do it in Sinatra:

post '/salmon_endpoint' do
  salmon_xml = request.body.read
  process_salmon(salmon_xml)
end

Here's how you can process the XML of the Salmon:

salmon = Proudhon::Salmon.new(salmon_xml)
salmon.content
=> ...

But Salmons rarely carry textual data - they're mostly Atom Feed Entries, so there's also a helper function to convert a Salmon into an Atom Entry:

salmon.to_entry
=> ...

With those two commands you just have to parse the Atom entry and notify users, insert data into your database, etc.

Pingback

Pingback is as easy as PubSubHubbub. Just pick an entry and call the method.

atom.entry.first.pingback(atom.links[:pingback], "http://my.url.com/my/article")
=> ""

Validation

However, how do you verify the provenance of your salmons? This is normally done via WebFinger or via an third-party Validation Service. It is a bit of a grey area when it comes to OStatus, because there's a lot of confusion on the implementations.

Our tests passes against the data on the specs, but we still can't interop with StatusNet, MiniMe or other OStatus implementations yet.

I strongly suggest that you wait until the code and the specifications gets stable, or help me with my code.

But here's how to verify. It should work between implementations using this library:

finger = Proudhon::Finger.new('glassx@identi.ca')
magic_key = finger.links[:magic_public_key]
public_key = Proudhon::MagicKey.new(magic_key)
salmon.verify(public_key)
=> true

Sending Salmons

You will also need to send new Salmons to other networks. First, you need an Atom Feed entry:

entry = Proudhon::Entry.new(:id => 'tag:example.com,2009:cmt-0.44775718',
  :cntent => 'This is a reply', :title => 'This is a reply',
  :author => Proudhon::Author.new(:name => 'Somebody'))

You also need a user-specific private key. Here's how to generate one on-the-fly. Keep in mind that you have to store it in a database!

key = OpenSSL::PKey::RSA::generate(512)

Then, you convert the entry to a Salmon:

salmon = entry.to_salmon

And deliver it:

salmon.deliver('https://website.com/salmon_endpoint', key)
=> true

That's it.

Diaspora Salmon

Diaspora is an emerging Social Network that uses its own kind of Salmon. It's not compatible with normal Salmons, and doesn't use Atom Feed Entries. The data is cryptographed.

It's a bit more complicated - you need the private key of the user receiving the message to decrypt the message itself.

salmon = Proudhon::Diaspora.new(salmon_message, private_key)
salmon.name
=> "Alexander Hamiltom"
salmon.uri
=> "acct:tom@localhost"
salmon.type
=> "status_message"
salmon.fields
=> {:public=>"false", :raw_message=>"I wanna know", :guid=>"84b5d8cc342062e4", :diaspora_handle=>"tom@localhost", :created_at=>"2011-03-28 16:06:40 UTC"}

The fields you'll receive and the "type" of the message are available on the our HTML documentation.

To create a message you need your private key and the target user's public key:

salmon = Proudhon::Diaspora.new
salmon.name = "Alexander Hamiltom"
salmon.uri = "acct:tom@localhost"
salmon.type = "status_message"
salmon.fields = {:public=>"false", :raw_message=>"I wanna know", :guid=>"84b5d8cc342062e4", :diaspora_handle=>"tom@localhost", :created_at=>"2011-03-28 16:06:40 UTC"}
salmon.to_xml(sender_private_key, receiver_public_key)
=> [...]

There you have it. It's basically serialized data going back and forth, but the private/public keys complicate the matter.

RSS

RSS feeds are still widely available on the Web, since most readers still accept it. This library includes an implementation.

Proudhon can parse RSS feeds, too:

rss = Proudhon::RSS.from_uri('http://feeds.mashable.com/mashable')
=> ...
rss.channel.title
=> 'Mashable!'
rss.channel.items.first.title
=> "Will the iPad 3 Have a Futuristic OLED Screen?"

And generate:

rss = Proudhon::RSS.new(:channel => Proudhon::Channel.new(:author => 'Myself', :items => [ Proudhon::Item.new(:title => 'Test') ]))
puts rss.to_xml
=> "<?xml version="1.0" encoding="UTF-8"?>\n<rss version="2.0">\n  <channel>\n    <author>Myself</author>\n    <item>\n      <title>test</title>\n    </item>\n  </channel>\n</rss>\n"
Something went wrong with that request. Please try again.