Home

jalada edited this page Sep 13, 2010 · 5 revisions
Clone this wiki locally

Welcome to the Hydrometeor wiki!

Hydrometeor is a long-polling webserver based on a publisher-subscriber model. It is written in Erlang and makes use of the Mochiweb library. Where possible it is written using OTP principles.

Hydrometeor is not designed to serve static files. In fact, it cannot. Instead it is designed to serve channel subscription requests and send channel messages to subscribers immediately upon receipt of them.

Features

  • By utilising Erlang’s lightweight processes, connections use very little memory.
  • Erlang and REST API for publishers supporting sending messages, and deleting channels.
  • Ability to save state to disk between restarts.
  • REST long-polling API for subscribing to realtime messages based on channel.
  • Multiple channel subscriptions in a single connection.
  • Multiple messages per long-poll response (catch-up facility).
  • Ability to poll for a ‘backlog’ of size n.
  • Maximum channel ‘age’ – prunes channels that haven’t been accessed by clients to save on memory/the need to delete channels manually.
  • Theoretical (untested) horizontal scaling using Distributed Erlang.
  • Round-robin channels (AMQP style).

Coming soon

  • Who knows…

Why Hydrometeor?

I’m the Lead Developer for Twitterfall and we’ve been using the Perl-based Meteor Server for quite some time. It has a great TCP API for publishers, can scale horizontally really easily, and has proved itself when dealing with thousands of concurrent connections. It’s exceeded our expectations quite a lot. There have been times however when we’ve needed some different features and Meteor’s method of ‘streaming’ becomes unreliable. As a result, we have sometimes had to turn to other solutions such as RabbitMQ, or some quickly hacked together Erlang (all our backend is written in Erlang) but none ever quite fit the bill.

So I decided to write my own web server to replace Meteor, hence the name Hydrometeor. It’s not affiliated with Twitterfall, it’s my own project. It’s open source (though I have yet to append the license anywhere).