Autoscaling Triggers
Autoscaling is one of the most difficult (and thus useful) features that are found in scalable software; difficult because it’s often not just one piece of software that needs to be scaled. In a simple three-tier web deployment model, it’s three pieces of software: the load balancer, the application servers, and the database. Each of these need to be handled differently and have their own quirks.
Let’s start off with an example to show you what the software stack looks like in AppScale. Suppose you deploy a single Google App Engine app to AppScale. That results in us deploying your app with the following technologies:
- Load balancer: Nginx for static file serving and SSL, and HAProxy for health checks
- App Server: Application servers running GAE apps (CherryPy for Python/Go/PHP, and Jetty for Java)
- Database: Cassandra and ZooKeeper
We place Nginx on the head node by itself, and it proxies traffic to HAProxy on the head node as well. Every 20 seconds, each slave node queries HAProxy to see how much traffic is coming in on each app. If more than 7 requests are queued up, waiting to be served, we try to add an AppServer on that node. We add up to 10 AppServers per node with CPU and memory willing, and once all AppServer nodes are “full” (either 10 AppServer, maxed CPU, or maxed RAM) a new node is added.
Downscaling happens in a similar fashion. Here, if the head node notices that there are zero requests enqueued with HAProxy, it removes an AppServer off the first slave node. Again, it never removes the last AppServer, but at that point, it moves onto the next node and starts removing AppServers there. If all nodes are down to their last AppServer, then it removes a node. Like in the upscaling case, we no longer have to get a confirmation from a second node, as the one HAProxy has a global view on the traffic coming to all nodes.
If you know you're going to deal with a set amount of traffic you can optionally use the "appengine" flag in your AppScalefile to set a static number of application servers per application. This option will disable autoscaling and its slight overhead of monitoring HAProxy stats.