No description, website, or topics provided.
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
README.rdoc

README.rdoc

How to develop a simple horizontally scalable, highly available system

We will show how to apply these methods to groupieguide.com (your typical Rails site).

Goals

  • Replicates across multiple datacenters

  • Easy setup

  • Fast enough

  • Allows for easy / simple updates

  • Data is safe. Backups and restores are simple. Incremental backups are possible.

  • High visibility into components

  • Writing the software for the system doesn't require someone smarter than me

  • Easy to test

  • Not tied to a vendor (i.e. EC2, Rackspace, Google Apps)

  • Easy to setup a development environment

  • Cheap. Doesn't require lots of fast hardware. Runs on small machines.

  • Show how you can migrate your standard Rails app to this architecture

Initial Incomplete Ideas

  • Cassandra for datastore

  • All (or most) services use a common communication method (probably Thrift)

  • Use chef for configuration management

  • Rails for frontend

  • Chef-deploy for deployments

  • Use recent version of ubuntu

  • rsyslog for logging

  • pacemaker or heartbeat for virtual IPs

Questions

  • Not sure about monitoring

  • How to do datacenter switchover? Low DNS TTL – issues?

  • How to break application into smaller services without introducing tons more complexity

  • Storing backups

  • What use cases will Cassandra not support?

  • Storing files / uploads

  • Full-text searching?

Action Items!

  1. Research what it would take to migrate GroupieGuide to Cassandra. Write up findings.

  2. Research virtual IP failover systems (pacemaker, heartbeat, wackamole). Do tests.

  3. Research how to best split GroupieGuide up into services, while keeping development easy. Also make sure that the system can be automatically tested via integration tests.

  4. Setup the chef cookbooks for deploying this system.

  5. Setup rsyslog for monitoring and centralized logging.

  6. Research how to best do backups and restores.

  7. Setup monitoring service.

  8. Research and setup full-text searching.

  9. Research and setup best way to do file uploads (and storing other files).