Rationale: If you run Cassandra locally on the same hosts where your program is running, then you have no kind of single-point of failure. But if you run your system with a load-balancer in front, then the SPOF is pretty obvious. This change let cassanderl connect to multiple hosts automatically and also fault out hosts which fails. Design: A new process, the watchdog is introduced. The watchdog is queried by dispatchers in cassanderl. The dispatchers are handed hosts in a round-robin fashion. If a host fails, this is reported into the watchdog which then times out the given host for a little while.
This change picks a calling convention which allows one to tell cassanderl/dispcount how to behave. In other words, we provide a callback interface with which we can instruct whether or not the connection is still valid and to what extent it is valid.
When cassanderl is starting up, it needs to be dependent on `thrift` and `cassandra_thrift`. None of these start anything but the dependency needs to be there so the reltool can automatically figure out the dependency. It also means errors will make the application startup fail, which is what we want over a half-running system.