Scott's ideas

From the original README:

• split project into chronologic-client and chronologic-server?

• add thin dependency?

• server should catch exceptions and return error codes

• privacy: does a checkin get published to the spot feed even if the user is private? if I look at a spot feed, shouldn't I see my private friends?

  • perhaps the event should have a private ('subscribers only') flag, so it can be excluded from results if there's not a subscription between the requester and the event creator
  • optional metadata flag: private
  • option passed to timeline: private
  • what if one timeline has a high percentage of private events? it'll make pagination much harder
  • some kind of intersection function would be very handy

• consider using elastic load balancing across a few chronologic nodes

• test with millions of objects/events/subscriptions

• consider writing new events to any explicitly given timelines, but backgrounding the fanout to subscribers

• support for browers to request timelines directly from chronologic server, via auth token

• evented ruby server

  • we can use yajl's streaming encoder
  • could use evented cassandra
  • would require thin or similarly EM-based rack server
  • http://github.com/raggi/async_sinatra
  • http://macournoyer.com/blog/2009/06/04/pusher-and-async-with-thin/
  • http://rainbows.rubyforge.org/ ?

• etag/if-modified-since

• consider storing full copies of events in timelines, so that reads don't require joins/multigets

• web UI

  • stats (total events, total timelines, avg fanout, total objects, total subscriptions, hourly graphs, response time stats, node health)
  • live stream
  • recently updated timelines
  • add-object form
  • add-subscription form
  • add-event form (with dropdowns for all timelines, objects, events, etc)

• redis for real-time notifications, queuing, cache, stats?

  • volatile redis hashes might be good for caching frequently used objects
  • when a timeline changes, preemptively cache it, so that even cold requests are fast
  • could still use a TTL so that feeds timelines that are never requested and never change don't eat up space
  • or get fancy, and try to prioritize timelines that are frequently requested

• client-side memcached (or redis)

  • store etag or last-mod values, and response body, so server can say not-modified
  • also consider supporting short Expires headers

• real-time / websockets / APS notifications support?

  • http://stackoverflow.com/questions/2999430/any-success-with-sinatra-working-together-with-eventmachine-websockets
  • POST to /channels
  • browsers can subscribe to channels via websockets
  • other agents could subscribe in order to fire off APS or PSHB pings

• client should support node failure gracefully

  • support retries and falling back to another node (unless load balancing is sufficient)

• enforce the requirement that :data values be keys

• allow timelines to be (optionally) capped, and to remove parentless events that are no longer represented in a timeline

• better documentation

  • pagination
  • how to build a new timeline or re-build one
  • creating atom/activitystreams feeds
  • streamlined cassandra installation
  • how to run on heroku (is there any hosted cassandra?)
  • rails-specific idioms (how to config, use observers to create events)
  • how to create a top-news feed
  • create github site
  • add rdoc comments
  • how to do event clustering (so your friends-feed can't be overrun by one hyper checker-inner).

• PSHB support?

  • the app requests notification when a timeline changes, and fires the hub notification then
  • not efficient: it'd be better if we could ping the hub with all the changes at once (user, spot, area, all friends, etc)
  • maybe you can tell Chronologic which timelines should cause hub pings and let it manage them
  • or maybe chronologic itself should provide hub functionality

• get some specs running

• alternate implementations

  • node-based server (need cassandra lib)
  • avro or some other interface?
  • any need for a broadcast system like zeromq? amqp?
  • mongo backend?
  • riak backend?
  • erlang server?

• enqueue fanout? or will it be fast enough? what if cassandra is temporarily down?

  • even though writes are fast, maybe they should still be triggered by a queue, so that we can group together notifications, publishing, etc., and pause cassandra sometimes.

• when/why should key be UUIDs?

• set up gem

• consider that testing for key existence is very fast in cassandra